Physics Hacker

Recently my harddisk crashed and I used this opportunity to reinstall my system. Everything went fine, but getting my user certificates back turned out to be a bit tricky. For Grid Computing and administrating a Grid Site I need my certificates to be stored in the browser. As I recently renewed them, I haven’t done a backup of them (shame on me) so I couldn’t just reimport them in my newly installed Firefox 4 (beta). Here is what I did to get the certificates back from my old Firefox profile:

1. Find out where your profile folder is located. This Mozillazine Page might help.
2. Quit Firefox if it is still running
3. Go to the profile folder of your new Firefox installation and backup the files cert8.db and key3.db
4. Copy key3.db from your old profile folder to your new one if you want to restore your private/user certificates
5. Copy cert8.db from your old profile folder to your new one if you want to restore all other certificates you had installed in Firefox (e.g. root certificates and host certificates)

If you need to find out the number of CPUs or CPU cores or cores per CPU of your system, you could look it up in /proc/cpuinfo but it’s quite hard to figure out the right parameters. A good overview on the parameters for different system configurations can be found here. If you want to put the actual numbers in variables, here is a nice way to do it:

export CORES_PER_CPU=`grep -c "physical id.*: 0" /proc/cpuinfo`
export CPU_TOTAL=`grep -c "core id.*: 0" /proc/cpuinfo`
export CORE_TOTAL=`grep -c processor /proc/cpuinfo`

/proc/cpuinfo shows an entry for each CPU core. The physical id is incremented for each physical CPU. If the entry has the same physical id as another core, the core belongs to the same CPU. Therefore counting the number of entries with physical id set to 0 results in the number of cores per CPU. The core id is incremented for each core on a physical CPU. Therefore counting the number of entries with core id set to 0 results in the number of physical CPUs. The total number of cores can be retrieved quite easily by counting the number of processor entries.

Unfortunately the above method does not work on all systems. I noticed on some systems with single core processors, that the values core id and physical id are not present.

I searched for official documentation on the proc filesystem, but only found the following document which doesn’t describe the cpuinfo values:

• http://www.kernel.org/doc/Documentation/filesystems/proc.txt

If someone happens to know a better documentation I would be glad if he/she would share it with me!

Debugging the SSL handshake can be lots of pain, especially if the SSL commands are done by components not under your control. Fortunately there is a tool called ssldump which lets you monitor the complete SSL handshake. The following command prints out detailed information about the SSL handshake (on interface eth0):

ssldump -a -A -H -i eth0

If you want to sneak at the encrypted traffic you need to tell ssldump where to find the hostkey (e.g. hostkey.pem or similar) of the machine

ssldump -N -d -k $PATH_TO_HOSTKEY/hostkey.pem -A -H -i eth0

Replace $PATH_TO_HOSTKEY and hostkey.pem accordingly.

More information can be found here or in the manpage of ssldump.

I recently wanted to start Mac OS X Applications from the console. Sometimes it is possible to start an application directly if it’s possible to find the executable in the application folder. But sometimes, espaccially for application installers there is no executable. In this case it is possible to start the application (or installer) by using the open command. e.g. to start TextEdit:

open /Applications/TextEdit.app

The open command does the same as clicking on a file / application. So you could open a pdf document like this:

open document.pdf

Sometimes an installer requires to run as a superuser. Starting the installer with

sudo open /Applications/INSTALLER_APP

unfortunately does not work, because only open will run as a superuser, but not the installer itself. To start the installer as superuser the following might help:

• pkg installer:

sudo ./MyApplication.app/Contents/MacOS/Installer

• other installers (no linebreak!)

sudo /System/Library/Frameworks/Carbon.framework/Versions/A/Support/LaunchCFMApp \
  ./MyApplication.app/Contents/MacOS/application

We had to install a few Desktop computers with an up to date operating system like Ubuntu, because neither Scientific Linux 5.x, nor SUSE Linux Enterprise was supporting the chipset of our new workstations. But Ubuntu does.

To make life easier for the normal users and to have a homogeneous computing environment we decided to write a kickstart file for Ubuntu. Ubuntu has support for kickstart files, but the documentation is really outdated and quite incomplete:

• Kickstart Compatibility
• Automatic Installation

We needed LDAP Authentication which is currently not working with Ubuntu kickstart. The trick is to include the LDAP configuration into the %post section of the kickstart file.

Creating a basic kickstart file

We started creating a kickstart file using the GUI utility on a working Ubuntu installation (you may use an Ubuntu Live CD for this)

system-config-kickstart

which can be installed on an Ubuntu system with

apt-get install system-config-kickstart

You need to add a meta package like ubuntu-desktop to the kickstart file created by system-config-kickstart in the %packages section and you might want to add other packages like nfs-common:

%packages
ubuntu-desktop
nfs-common

Configuration of LDAP in the kickstart file

The GUI utility provides some LDAP options which unfortunately do not work with Ubuntu 8.04. So we needed to add the LDAP configuration to the post section of the kickstart file.

During the LDAP installation (with apt-get install) some configuration options have to be filled in interactively. To do this automatically, you can use the preseed mechanism (see here for more information on preseed under Ubuntu 8.04).

At first you have to install LDAP on your own machine (or use a livecd) and configure it interactively:

apt-get install ldap-auth-client --assume-yes
auth-client-config -a -p lac_ldap

After that run the followin command to get all the LDAP options

debconf-get-selections | grep ldap

Now you should see something like this:

ldap-auth-config	ldap-auth-config/bindpw	password
ldap-auth-config	ldap-auth-config/rootbindpw	password
ldap-auth-config	ldap-auth-config/binddn	string	cn=proxyuser,dc=example,dc=net
ldap-auth-config	ldap-auth-config/dbrootlogin	boolean	false
ldap-auth-config	ldap-auth-config/rootbinddn	string	cn=manager,dc=example,dc=net
ldap-auth-config	ldap-auth-config/pam_password	select	md5
ldap-auth-config	ldap-auth-config/move-to-debconf	boolean	true
ldap-auth-config	ldap-auth-config/ldapns/ldap-server	string	ldap_server_name
ldap-auth-config	ldap-auth-config/ldapns/base-dn	string	dc=your,dc=domain,dc=tld
ldap-auth-config	ldap-auth-config/override	boolean	true
ldap-auth-config	ldap-auth-config/ldapns/ldap_version	select	3
ldap-auth-config	ldap-auth-config/dblogin	boolean	false

Now include these information into your kickstart file with the preseed option. The result should look similar to this:

preseed --owner ldap-auth-config ldap-auth-config/bindpw password
preseed --owner ldap-auth-config ldap-auth-config/rootbindpw password
preseed --owner ldap-auth-config ldap-auth-config/binddn string cn=proxyuser,dc=example,dc=net
preseed --owner ldap-auth-config ldap-auth-config/dbrootlogin boolean false
preseed --owner ldap-auth-config ldap-auth-config/rootbinddn string  cn=manager,dc=example,dc=net
preseed --owner ldap-auth-config ldap-auth-config/pam_password select  md5
preseed --owner ldap-auth-config ldap-auth-config/move-to-debconf boolean true
preseed --owner ldap-auth-config ldap-auth-config/ldapns/ldap-server string  ldap_server_name
preseed --owner ldap-auth-config ldap-auth-config/ldapns/base-dn string  dc=your,dc=domain,dc=tld
preseed --owner ldap-auth-config ldap-auth-config/override boolean true
preseed --owner ldap-auth-config ldap-auth-config/ldapns/ldap_version select  3
preseed --owner ldap-auth-config ldap-auth-config/dblogin boolean false

In the %post section of the kickstart file add the following to install and configure LDAP

%post --interpreter=/bin/bash
apt-get install ldap-auth-client --assume-yes
auth-client-config -a -p lac_ldap

Other useful stuff

There are some other useful things you can put into the %post section of the kickstart file:

You might want to moun the home directories like this:

echo "host:/export/home /home  nfs    defaults  0 0"  >> /etc/fstab

If you want to enable auto update on a regular base you can use the package cron-apt

apt-get install cron-apt --assume-yes

By default the cron job just downloads the updates. To automatically install the updates you have to strip the -d option from the apt-get command. This can be done as following:

sed -e 's/ -d / /g' /etc/cron-apt/action.d/3-download > /etc/cron-apt/action.d/3-download2
mv /etc/cron-apt/action.d/3-download2 /etc/cron-apt/action.d/3-download

You might want to set the rootmail user or add users to the sudoers list:

### ROOTMAIL
echo "root:           rootmail@your.domain.de" >> /etc/aliases
### SUDOERS
echo "username ALL=(ALL) ALL" >> /etc/sudoers

The final kickstart file

Here you can find an example kickstart file for a x64 system which you can adopt to your personal needs.

Manually creating a password under linux for e.g. /etc/shadow or kickstart isn’t really easy. I searched for a while until I found the userdbpw utility. On Debian Systems (including Ubuntu) it can be installed with

apt-get install courier-authlib-userdb

For most distributions the md5 algorithm is used to create passwords. The userdbpw command for md5 passwords is:

userdbpw -md5

More information on userdbpw can be found in its manpage.

So Summer is getting hot and fans are starting to roar. So if you appreciate some silence in the office and just happen to have a Fujitsu Siemens ESPRIMO with P4 on your desk this is for you.

This works for SLC 4 on Fujitsu Siemens ESPRIMO with P4

Install sysutils:

yum install sysfsutils

load module p4_clockmod:

modprobe p4_clockmod

check available schedulers and frequencies:

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies

if needed load “ondemand” scheduler:

modprobe cpufreq_ondemand;

check current frequency and scheduler:

cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

make “ondemand” the active scheduler:

echo ondemand > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

make changes permanent:

echo “modprobe p4_clockmod” >> /etc/rc.modules

echo “modprobe cpufreq_ondemand” >> /etc/rc.modules

chomod u+x /etc/rc.modules

echo “devices/system/cpu/cpu0/cpufreq/scaling_governor=ondemand >> /etc/sysfs.conf

I stole some of this from this german Howto on CPU scaling.

Running larger sites demands for scripting node installation.

I like to use python for the task (surprise surprise). As it happens the mayor package managment tool for Red Hat, CentOS and Scientific Linux “yum” is itself written in python.

Since yum lacks documentation on how to use inside your own python code I have googled the web and found this nice page: Deciphering the Yum API.

For some reasons the code didn’t work for me on Scientific Linux 4 (Red Hat 3.4.6-9) which still uses python2.3. So I fiddled around a bit and got it working. Here are the updated examples.

Listing packages:

import yum

yb = yum.YumBase()
yb.doConfigSetup()
yb.doTsSetup()
yb.doRpmDBSetup()
for pkg in yb.rpmdb.getPkgList():
  print pkg

Searching packages:

In newer versions of yum there seems to be YumBase.searchGenerator, wich should be prefered for performance and memory footprint. However in the enterprise class distros i have to deal with, I could not use it. So I present a solution with YumBase.searchPackages.

import yum

yb = yum.YumBase()
yb.doConfigSetup()
yb.doRepoSetup()
yb.doSackSetup()
yb.doTsSetup()
yb.doRpmDBSetup()
fields = ['name']        # fields to look at
criteria = ["k3b"]       # strings to find in fields
matches = yb.searchPackages(fields, criteria)
for match in matches:
  print match

Ask if a certain package is installed:

import yum

yb = yum.YumBase()
yb.doConfigSetup()
yb.doTsSetup()
yb.doRpmDBSetup()
# prints 1 if installed else 0
print yb.rpmdb.installed('vim-enhanced')

Installing packages:

Install the editor joe. Note that we do not use the “standard” YumBase but the “command line” YumBaseCli. This class provides the installPkgs() function which magically sorts out which package we want by just saying “joe”. Note also that by using YumBaseCli in this way, we are bypassing some argument checks the command line yum would have done. To make sure yum doesn’t ask any questions we do “ybc.conf.setConfigOption(‘assumeyes’,True)”.
Beware of the consequences.

import sys

sys.path.append('/usr/share/yum-cli')

import cli

ybc = cli.YumBaseCli()
ybc.doConfigSetup()
ybc.doTsSetup()
ybc.doRpmDBSetup()
ybc.installPkgs(['joe'])
ybc.buildTransaction()
ybc.conf.setConfigOption('assumeyes',True)
ybc.doTransaction()