ksh93 libshell

In a number of posts about a year ago I discussed how to develop custom builtins for Korn Shell 93 (ksh93) using libshell and published APIs. You can also use these same APIs to access ksh93 functionality from within a C application. This post provides three simple examples of how to do this.

The first example simply takes two numbers as arguments on the command line, multiplies the numbers and prints out the result on stderr.
#include <shell.h>
#include <nval.h>

int main(int argc, char *argv[])
{
Namval_t *np;
Sfdouble_t res;

Shell_t *shp = sh_init(argc, argv, 0);
sh_trap("((xresult=$0*$1))", 0);

np = nv_open("xresult", shp->var_tree, 0);
res = nv_getnum(np);
sfprintf(sfstderr,"%Lg\n", res);
nv_close(np);

return(0);
}
Here is how to compile this example using static libraries. I built these examples in my home directory but the ksh93 build environment is at /work/ksh93, hence the use of -I and -L options. Note the large number of AST (Advanced Software Toolkit) and ksh93 libraries you have to specify in order to resolve all the symbols.
gcc -o example1 -I/work/ksh93/arch/linux.i386-64/include/ast example1.c -L/work/ksh93/arch/linux.i386-64/lib -lshell -lm -ldll -lcmd  -ldl -last 
For example:

$ ./example1 3 4
12
$
The next example breaks the text string string into its component parts and prints them out to stdout. Note that "foo\\ bar" is one word, not two words.
#include <shell.h>
#include <stdio.h>
#include <nval.h>

int
main(int argc, char *argv[])
{
char tmp[512];
char string[] ="hello world 'my name is Finnbarr' foo\\ bar";

sprintf(tmp, "set - %s && for i in \"$@\";\n do\n echo \"$i\"\ndone", string);

Shell_t *shp = sh_init(argc, argv, 0);
sh_trap(tmp, 0);

return(0);
}
Here is the output:
./example2
hello
world
my name is Finnbarr
foo bar
The next example is similar to the previous example, except that it creates an array and then uses nv_getval() to access the contents of the created array and print the contents to stderr.
#include <shell.h>
#include <stdio.h>
#include <nval.h>

int
shell(int argc, char* argv[], char *str)
{
Namval_t *np, *np_sub;
char tmp[512];

sprintf(tmp, "set - %s && for i in \"$@\";\n do\n aname+=(\"$i\")\ndone", str);

Shell_t *shp = sh_init(argc, argv, 0);
sh_trap(tmp, 0);

np = nv_open("aname", shp->var_tree, 0);
nv_putsub(np, NULL, ARRAY_SCAN);
np_sub = np;

do {
// copy out the arguments to wherever here.
fprintf(stderr, "%d: subscript='%s' value='%s'\n", np_sub, nv_getsub(np_sub), nv_getval(np_sub));
} while (np_sub && nv_nextsub(np_sub));

nv_close(np);

return(0);
}

int
main(int argc, char *argv[])
{
char string[] ="hello world 'my name is simon' foo\\ bar";

shell(argc, argv, string);
}
Here is the output:
$ ./example1
37212336: subscript='0' value='hello'
37212336: subscript='1' value='world'
37212336: subscript='2' value='my name is simon'
37212336: subscript='3' value='foo bar'
$
As you can see, running shell script snippets from within a C application is not difficult. For more information on these APIs read the ksh93 shell(3) and nval(3) man pages.
 

Microsoft SUA JavaScript Shell

Recently I needed to test some JavaScript code on a Microsoft Vista Ultimate operating system using the command line. This is something that I have done before on Linux platforms using a JavaScript shell but had never done on a Microsoft platform. For those who are not familiar with a JavaScript shell, it is a command line interface to a JavaScript engine. Similar to Python or Ruby, the JavaScript shell has two modes of operation. You can use it as an interactive shell, in which you type JavaScript code at a prompt and get instant results, or you can invoke a JavaScript program.

The easiest way that I know of to build a JavaScript shell is to download and build the SpiderMonkey JavaScript engine which comes with a JavaScript Shell. SpiderMonkey is one of two JavaScript engines which the Mozilla software project supports. It is used in the Firefox browser and elsewhere. The other JavaScript engine is Rhino. Rhino is written using the Java language whereas SpiderMonkey is a pure C language implementation which conforms to ECMA-262 Edition 3. Included in SpiderMonkey is a JavaScript shell. Source code for both JavaScript engines is readily available on the Mozilla website.

For Microsoft Vista I had a choice of either building a native Windows executable which could be accessed via the Vista command prompt (similar to the old DOS prompt) or, because Vista Ultimate comes with Subsystem for Unix-based Applications (SUA), a shell which was accessable via a more familar Unix-like environment. I choice to go the SUA route.

Many people are unaware of Microsoft's SUA offering. Basicly it is a full-featured POSIX-compliant subsystem for Windows NT-based operating systems. For their own reasons, Microsoft limits the availablity of SUA to certain products such as Vista Enterprise and Ultimate editions and Windows Server 2008. It comes with hundreds of utilities including the GNU compiler collection, Korn shell, and development headers and libraries. See here for further information.

SUA is a subsystem which runs natively on top of the Windows kernel. which was orginally developed by Softway Systems. The original name for the product was OpenNT but that was later changed to Interix. Interix and most of the development team was acquired by Microsoft in 1999 and the product was renamed Services For Unix (SFU) in 2005. With the release of Microsft Vista, it was integrated into the operating system as a separate installable component and was renamed Subsystem for Unix-based Applications. There is an active user community around SUA which is hosted by Interop Systems and sponsored by Microsoft.

This is the Makefile which I hacked together to build a static js executable from the SpiderMonkey 1.8 sources for use with Microsoft Vista SUA.

JSVERSION = js-1.8.0

CC = gcc

OS_ARCH := $(subst /,_,$(shell uname -s | sed /\ /s//_/))
OS_RELEASE := $(shell uname -r)
OS_CONFIG := $(OS_ARCH)$(OS_RELEASE)
OBJDIR = ./$(OS_CONFIG)

DEFINES = -DXP_UNIX -DPOSIX_SOURCE -DHAVE_LOCALTIME_R

LIBDIR := lib
INCLUDES += -I$(OBJDIR)
RANLIB = ranlib
LDFLAGS = -lm

OPTIMIZER =
INTERP_OPTIMIZER =

CFLAGS = $(OPTIMIZER) $(DEFINES) $(INCLUDES)
INTERP_CFLAGS = $(INTERP_OPTIMIZER) $(DEFINES) $(INCLUDES)

PREDIRS += editline
DEFINES += -DEDITLINE

JS_CFILES = jsapi.c jsarena.c jsarray.c jsatom.c jsbool.c jscntxt.c jsdate.c \
jsdbgapi.c jsdhash.c jsdtoa.c jsemit.c jsexn.c jsfun.c jsgc.c jshash.c \
jsinterp.c jsinvoke.c jsiter.c jslock.c jslog2.c jslong.c jsmath.c \
jsnum.c jsobj.c jsopcode.c jsparse.c jsprf.c jsregexp.c jsscan.c \
jsscope.c jsscript.c jsstr.c jsutil.c jsxdrapi.c jsxml.c prmjtime.c

JS_HFILES = jsarray.h jsatom.h jsbool.h jsconfig.h jscntxt.h jsdate.h \
jsemit.h jsexn.h jsfun.h jsgc.h jsinterp.h jsiter.h jslibmath.h \
jslock.h jsmath.h jsnum.h jsobj.h jsopcode.h jsparse.h jsarena.h \
jsclist.h jsdhash.h jsdtoa.h jshash.h jslong.h jstypes.h jsprvtd.h \
jspubtd.h jsregexp.h jsscan.h jsscope.h jsscript.h jsstr.h jsutil.h \
jsxdrapi.h jsxml.h

API_HFILES = jsapi.h jsdbgapi.h

OTHER_HFILES = jsbit.h jscompat.h jscpucfg.h jsotypes.h jsstddef.h jsopcode.tbl \
jsproto.tbl js.msg jsshell.msg jskeyword.tbl prmjtime.h resource.h


ifdef JS_HAS_FILE_OBJECT
JS_CFILES += jsfile.c
JS_HFILES += jsfile.h
endif

LIB_CFILES = $(JS_CFILES)
PROG_CFILES = js.c

LIB_OBJS = $(addprefix $(OBJDIR)/, $(LIB_CFILES:.c=.o))
PROG_OBJS = $(addprefix $(OBJDIR)/, $(PROG_CFILES:.c=.o))

CFILES = $(LIB_CFILES) $(PROG_CFILES)
OBJS = $(LIB_OBJS) $(PROG_OBJS)

LIBRARY = $(OBJDIR)/libjs.a
PROGRAM = $(OBJDIR)/js
TARGETS = $(PROGRAM)

EDIT_LIBRARY = $(OBJDIR)/libedit.a
EDIT_DIR = ./editline
EDIT_CFILES = $(EDIT_DIR)/editline.c $(EDIT_DIR)/sysunix.c
EDIT_OBJS = $(OBJDIR)/editline.o $(OBJDIR)/sysunix.o
EDIT_CFLAGS = -DANSI_ARROWS -DHAVE_TCGETATTR -DHIDE -DUSE_DIRENT -DSYS_UNIX \
-DHAVE_STDLIB -DUNIQUE_HISTORY $(DEFINES) -I$(EDIT_DIR)


define MAKE_OBJDIR
if test ! -d $(@D); then rm -rf $(@D); mkdir -p $(@D); fi
endef


all:
$(MAKE) $(TARGETS)

$(PROGRAM): $(PROG_OBJS) $(LIBRARY) $(EDIT_LIBRARY)
$(CC) -o $@ $(CFLAGS) $(PROG_OBJS) $(LIBRARY) $(EDIT_LIBRARY) $(LDFLAGS)

$(OBJDIR)/%.o: %.c %.h
@$(MAKE_OBJDIR)
$(CC) -o $@ -c $(CFLAGS) $*.c

$(OBJDIR)/editline.o: $(EDIT_DIR)/editline.c $(EDIT_DIR)/editline.h
@$(MAKE_OBJDIR)
$(CC) -o $@ -c $(EDIT_CFLAGS) $(EDIT_DIR)/editline.c

$(OBJDIR)/sysunix.o: $(EDIT_DIR)/sysunix.c $(EDIT_DIR)/editline.h
@$(MAKE_OBJDIR)
$(CC) -o $@ -c $(EDIT_CFLAGS) $(EDIT_DIR)/sysunix.c

$(OBJDIR)/%.o: %.c
@$(MAKE_OBJDIR)
$(CC) -o $(OBJDIR)/jscpucfg jscpucfg.c
$(OBJDIR)/jscpucfg > $(OBJDIR)/jsautocfg.h
$(CC) -o $(OBJDIR)/jskwgen jskwgen.c
$(OBJDIR)/jskwgen $(OBJDIR)/jsautokw.h
$(CC) -o $@ -c $(CFLAGS) $*.c

$(OBJDIR)/jsinterp.o: jsinterp.c jsinterp.h
@$(MAKE_OBJDIR)
$(CC) -o $@ -c $(INTERP_CFLAGS) jsinterp.c

$(EDIT_LIBRARY): $(EDIT_OBJS)
$(AR) rv $@ $?
$(RANLIB) $@

$(LIBRARY): $(LIB_OBJS)
$(AR) rv $@ $?
$(RANLIB) $@

clean:
rm -rf $(OBJS)

clobber:
rm -rf $(OBJDIR)

tarball:
mkdir -p ./$(JSVERSION)/editline
cp *.[ch] ./$(JSVERSION)
cp *.tbl ./$(JSVERSION)
cp *.js ./$(JSVERSION)
cp *.msg ./$(JSVERSION)
cp Makefile ./$(JSVERSION)
cp README ./$(JSVERSION)
cp editline/*.[ch] ./$(JSVERSION)/editline
cp editline/README ./$(JSVERSION)/editline
cp editline/editline.3 ./$(JSVERSION)/editline
tar cvf $(JSVERSION).tar ./$(JSVERSION)
gzip $(JSVERSION).tar
rm -rf ./$(JSVERSION)

Note you need to use gcc 4.2 instead of gcc 3.2 otherwise you will get an undefined _JS_DHashTableOperate symbol due to an obsure gcc bug. However with gcc 4.2, you will see a large number of compiler warnings relating to alignment is greater than maximum object file. Fortunately you can safely ignore them. By the way, I choose to build a static executable because building shared libraries on SUA using gcc is somewhat problematic.



Once you have build an executable, a simple smoketest is to use the executable to execute the prefect.js JavaScript script which is included with the SpiderMonkey source code. You should get output similar to the following:

$ ./Interix6.0/js perfect.js

A number is 'perfect' if it is equal to the sum of its
divisors (excluding itself).

The perfect numbers up to 500 are:
6 = 1 + 2 + 3
28 = 1 + 2 + 4 + 7 + 14
496 = 1 + 2 + 4 + 8 + 16 + 31 + 62 + 124 + 248
That's all.
$

Well, this should be enough to enable you to build your own JavaScript shell on Microsoft's Subsystem for Unix-based Applications. Good Luck!
 

Inverting Large Images Using CUDA

This is a simple example of how to invert a very large image, stored as a vector using nVidia's CUDA programming environment and an EVGA GeForce GTX 260 graphics card.

A GeForce GTX 260 card should have only 24 cores on it, but the nVidia SDK deviceQuery utility reports that there are 27 multiprocessors on my GeForce GTX 260 graphics card and I am able to use all 27 of them.

$ deviceQuery
CUDA Device Query (Runtime API) version (CUDART static linking)
There is 1 device supporting CUDA

Device 0: "GeForce GTX 260"
CUDA Driver Version: 2.30
CUDA Runtime Version: 2.30
CUDA Capability Major revision number: 1
CUDA Capability Minor revision number: 3
Total amount of global memory: 938803200 bytes
Number of multiprocessors: 27
Number of cores: 216
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 16384 bytes
Total number of registers available per block: 16384
Warp size: 32
Maximum number of threads per block: 512
Maximum sizes of each dimension of a block: 512 x 512 x 64
Maximum sizes of each dimension of a grid: 65535 x 65535 x 1
Maximum memory pitch: 262144 bytes
Texture alignment: 256 bytes
Clock rate: 1.35 GHz
Concurrent copy and execution: Yes
Run time limit on kernels: Yes
Integrated: No
Support host page-locked memory mapping: Yes
Compute mode: Default (multiple host threads can use this device simultaneously)
The application is fairly trivial. It reads in an image file in PPM P6 format, stores the image in a vector of integers, one per pixel, with 24-bits used to store the RGB values. Depending on which command line option is selected ( -c invert using CPU, -g invert using GPU), it then inverts the image vector either using the CPU or the GPU (Graphical Processing Unit). Finally it writes the image out to an output file in PPM P6 format.

It uses a block size of 16 and a single kernel called myCudaInvertPixel which works on 16 rows of the image at the time via tiling.

#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>
#include <climits>

#include <unistd.h>
#include <time.h>
#include <string.h>
#include <stdio.h>

#include <cuda.h>
#include <cuda_runtime_api.h>

#define BLOCK_SIZE 16
// #define DEBUG 0

// ------------ prototypes --------------
__global__ void myCudaInvertPixel(int *, int);


class CCL {

int width;
int height;

public:

CCL() { width = height = 0; }
~CCL() { width = height = 0; }

void cuda_ccl(vector& img, int w)
{
width = w;
height = img.size()/w;

#ifdef DEBUG
cout << "Image width=" << width << " height=" << height << endl;
#endif
int result = cuda_invert(img, width, height);

return;
}

private:
int cuda_invert(vector&, int, int);

};


int
CCL::cuda_invert(vector& image, int width, int height)
{
int deviceCount = 0;
int driverVersion = 0, runtimeVersion = 0;
int globalMem;
int image_size = width * height * sizeof(int);

int *cuda_image_ptr;
cudaDeviceProp deviceProp;
cudaError_t err;

int strake_size = 0;
int last_strake_size = 0;
int strakes = 0;
int this_strake_size = 0;
int offset = 0;

if (cudaGetDeviceCount(&deviceCount) != cudaSuccess) {
printf("ERROR: cudaGetDeviceCount failed. CUDA Driver and Runtime version may be mismatched.\n");
return 1;
}

if (deviceCount == 0) {
printf("ERROR: No device supporting CUDA found\n");
return 1;
}

cudaGetDeviceProperties(&deviceProp, 0);
globalMem = deviceProp.totalGlobalMem;

#ifdef DEBUG
printf("Number of CUDA devices: %d\n\n", deviceCount);
printf("Device %d: \"%s\"\n", 0, deviceProp.name);
#if CUDART_VERSION >= 2020
cudaDriverGetVersion(&driverVersion);
cudaRuntimeGetVersion(&runtimeVersion);
printf(" CUDA Driver Version: %d.%d\n", driverVersion/1000, driverVersion%100);
printf(" CUDA Runtime Version: %d.%d\n", runtimeVersion/1000, runtimeVersion%100);
#endif
printf(" CUDA capability revision number: %d.%d\n", deviceProp.major, deviceProp.minor);
#if CUDART_VERSION >= 2000
printf(" Number of multiprocessors: %d\n", deviceProp.multiProcessorCount);
printf(" Number of cores: %d\n", 8 * deviceProp.multiProcessorCount);
printf(" Maximum number of threads per block: %d\n", deviceProp.maxThreadsPerBlock);
#endif
printf(" Clock rate: %.2f GHz\n", deviceProp.clockRate * 1e-6f);
printf(" Global memory: %d\n", globalMem);
printf(" Shared memory per block: %d\n", deviceProp.sharedMemPerBlock);
printf(" Registers per block: %d\n\n", deviceProp.regsPerBlock);
#endif

cudaSetDevice(0); // not needed - default is 0.

int rowBytes = width * 4; // size of a row
int maxrow = BLOCK_SIZE; // max rows that can be handled in global memory
strake_size = maxrow * rowBytes; // memory to allocate
strakes = image_size/strake_size + ((image_size % strake_size == 0) ? 0 : 1); // there X strakes necessary
last_strake_size = image_size - (strake_size * (strakes - 1)); // figure out size of last strake (may be smaller)
#if DEBUG
printf(" Image size %d\n", image_size);
printf(" Image vector size %d\n", image.size());
printf(" Row size %d\n", rowBytes);
printf(" Maxximum rows %d\n", maxrow);
printf(" Strake size %d\n", strake_size);
printf(" Strakes %d\n", strakes);
printf(" Last strake size %d\n\n", last_strake_size);
#endif

// uncomment if you want to using CUDA's elapsed timer
// float elapsed_time_ms = 0.0f;
// cudaEvent_t start, stop;
// cudaEventCreate( &start);
// cudaEventCreate( &stop);
// cudaEventRecord(start, 0);

dim3 dimBlock(BLOCK_SIZE, BLOCK_SIZE);
dim3 dimGrid(((width*4)/dimBlock.x) + (((width*4)%dimBlock.x)?1:0), BLOCK_SIZE);

cudaMalloc((void**)&cuda_image_ptr, strake_size);
if ((err = cudaGetLastError()) != cudaSuccess) {
fprintf(stderr, "ERROR: Cuda: %s\n", cudaGetErrorString(err));
exit(2);
}

for (int i = 0; i < strakes; i++) {
if (i > 0 && i == strakes - 1) {
this_strake_size = last_strake_size;
} else {
this_strake_size = strake_size;
}
offset = 0 + i *(strake_size/4);

cudaMemcpy(cuda_image_ptr, &image[offset], this_strake_size, cudaMemcpyHostToDevice);
if ((err = cudaGetLastError()) != cudaSuccess) {
fprintf(stderr, "ERROR: Cuda: %s\n", cudaGetErrorString(err));
exit(2);
}

// invoke CUDA kernel
myCudaInvertPixel<<<dimGrid, dimBlock>>>(cuda_image_ptr, width);
if ((err = cudaGetLastError()) != cudaSuccess) {
fprintf(stderr, "ERROR: Cuda: %s\n", cudaGetErrorString(err));
exit(2);
}

// not needed here - cudaThreadSynchronize();

cudaMemcpy( &image[offset], cuda_image_ptr, this_strake_size, cudaMemcpyDeviceToHost);
if ((err = cudaGetLastError()) != cudaSuccess) {
fprintf(stderr, "ERROR: Cuda: %s\n", cudaGetErrorString(err));
exit(2);
}
}

cudaFree(cuda_image_ptr);

// uncomment if you want to using CUDA's elapsed timer
// cudaEventRecord(stop, 0);
// cudaEventSynchronize(stop);
// cudaEventElapsedTime( &elapsed_time_ms, start, stop);
// printf("Elapsed time per CUDA: %f\n", elapsed_time_ms);

return 0;
}


// CUDA kernel definition - invert pixel
__global__ void myCudaInvertPixel(int *pixel, int width)
{
int ix = blockIdx.x * blockDim.x + threadIdx.x;
int iy = blockIdx.y * blockDim.y + threadIdx.y;
int idx = ix + iy*width;

if (ix <= width || iy >= BLOCK_SIZE) return;

pixel[idx] = 0x00FFFFFF - pixel[idx];
}


class Timer {

private:

timespec starttime, endtime;
float elapsed;

float difftime(timespec start, timespec end) {
float temp;

temp = (end.tv_sec - start.tv_sec) +
(float) (end.tv_nsec - start.tv_nsec) / 1000000000.0;

return temp;
}

public:

Timer() { elapsed = 0.00; }

void reset() { elapsed = 0.00; }

void start() {
elapsed = 0.00;
clock_gettime(CLOCK_MONOTONIC, &starttime);
}

void stop() {
clock_gettime(CLOCK_MONOTONIC, &endtime);
elapsed = elapsed + difftime(starttime, endtime);
}

void restart() { clock_gettime(CLOCK_MONOTONIC, &starttime); }

void print() { cout << "Elapsed time: " << elapsed << endl; }

};

class PPM {

protected:

int width;
int height;
int maxColor;
vector <int> image;
friend class CCL;

public:

PPM() {
width = 0;
height = 0;
maxColor = 0;
}

int read(char *filename);

int write(char* filename);

void printsize() {
cout << "Image height: " << height << endl;
cout << "Image width: " << width <<< endl;
cout << "Image vector size: " << image.size() << endl;
}

void invertpixelNoCuda() {
for (vector<int>::iterator it = image.begin(); it != image.end(); ++it) {
*it = 0x00FFFFFF - *it;
}
}

void invertpixelCuda() {
CCL ccl;
ccl.cuda_ccl(image, width);
}

int getwidth() { return width; }

};


int
PPM::read(char *filename)
{
FILE *fp = fopen(filename, "r");
int rgb, x, y;
unsigned char red, green, blue;
char buf[256];
int *pwidth = &width;
int *pheight = &height;

if (fp== NULL) return 1;

do {
fgets(buf, 256, fp);
} while(buf[0] == '#');

if (buf[0] != 'P' || buf[1] != '6') {
fclose(fp);
return 1;
}

do {
fgets(buf, 256, fp);
} while(buf[0] == '#');

sscanf(buf, "%u %u", pwidth, pheight);
fscanf(fp, "%u\n", &maxColor);

image.resize(height*width);
for (y = 0; y < height ; y++) {
for (x = 0; x < width; x++) {
fscanf(fp, "%c%c%c", &red, &green, &blue);
rgb = ((int)blue & 0x000000ff) << 16 |
((int)green & 0x000000ff ) << 8 |
((int)red & 0x000000ff) | 0x00000000;
image[y*width + x] = rgb;
}
}

fclose(fp);
return 0;
}


int
PPM::write(char* filename)
{
FILE *fp;
int y, x;
unsigned char red=0, green=0, blue=0;
int rgb;

if ((fp = fopen(filename, "w")) == NULL) {
fprintf(stderr, "ERROR: Cannot open output file\n");
return 1;
}

fprintf(fp, "P6\n");
fprintf(fp, "%d %d\n", width, height);
fprintf(fp, "255\n");

for (y = 0; y < height; y++) {
for (x = 0; x < width; x++) {
rgb = image[y*width + x];
blue = (rgb & 0x00FF0000) >> 16;
green = (rgb & 0x0000FF00) >> 8;
red = (rgb & 0x000000FF);
fprintf(fp, "%c%c%c", red, green, blue);
}
}

fclose(fp);

return(0);
}


void
usage(char *name,
int mode)
{
cerr << "Usage: " << name << " [-c] [-d] [-g] infile outfile" << endl;
exit(mode);
}


int
main(int argc,
char *argv[])
{
int c;
int cflag = 0, dflag = 0, gflag = 0;
PPM ppm;
Timer timer;

if (argc == 1)
usage(argv[0], 0);

opterr = 0;
while ((c = getopt( argc, argv, "cdgh")) != -1) {
switch (c) {
case 'c':
cflag = 1;
break;
case 'd':
dflag = 1;
break;
case 'g':
gflag = 1;
break;
case '?':
case 'h':
usage(argv[0], 0);
break;
}
}

if (optind != argc-2) {
cerr << "ERROR: An input file and an output file must be specified" << endl;
usage(argv[0], 1);
}

if (ppm.read(argv[optind++])) {
cerr << "ERROR: Failed to read input file" << endl;
exit(1);
}

timer.start();

if (dflag) ppm.printsize();

if (cflag) ppm.invertpixelNoCuda();

if (gflag) ppm.invertpixelCuda();

timer.stop();

if (ppm.write(argv[optind])) {
cerr << "ERROR: Failed to write output file" << endl;
exit(1);
}

timer.print();

exit(0);
}

Well that is all there is to this application. To compile it you need to have at least the nVidia CUDA 2.3 SDK installed and the latest nVidia graphics card driver.

If you are new to CUDA programming, I suggest you download the source code, compile it and play around with it on your own computer. The man pages for Linux do not work (they are all mixed up) but good up-to-date documentation can be found online at nVidia CUDA Library Documentation.

You might also want to play around with the block (dimBlock) and grid (dimGrid) dimensions. nVidia has a useful Microsoft Excel Occupancy Calculator Spreadsheet.

P.S. I tested the application on images up to a 32000 x 32000 pixels on RHEL5.4 using the CUDA 2.3 SDK and an EVGA GeForce GTX 260 graphics card with 896Mb of memory.

Connected Component Labeling 1


Recently, I became interested in connected component labeling (CCL) algorithms because of a competition on TopCoder. Over the next few weeks I plan to explore various algorithms including parallel algorithms written to take advantage of nVidia GPUs via the CUDA programming model.

To simply things and enable me to concentrate on the algorithms, all images will be in PPM P6 format and will be loaded into a vector (image) of integers. The image vector will be processed by the CCL algorithm. The transformed image vector will then be written out to a PPM P6 file for viewing by an external application such as GIMP.

Here is an example program which contains a PPM class for reading, writing, inverting, and displaying some information about a PPM file. An image is read into the image vector with the red, green and blue components stored in specific bits of each integer. It also contains a simple timer class which uses clock_getttime() to retrieve the monotomic time.

#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>
#include <unistd.h>
#include <time.h>

using namespace std;

class Timer {

private:

timespec starttime, endtime;
float elapsed;

float difftime(timespec start, timespec end)
{
float temp;

temp = (end.tv_sec - start.tv_sec) +
(float) (end.tv_nsec - start.tv_nsec) / 1000000000.0;

return temp;
}

public:

Timer() {
elapsed = 0.00;
}

void reset() {
elapsed = 0.00;
}

void start() {
elapsed = 0.00;
clock_gettime(CLOCK_MONOTONIC, &starttime);
}

void stop() {
clock_gettime(CLOCK_MONOTONIC, &endtime);
elapsed = elapsed + difftime(starttime, endtime);
}

void restart() {
clock_gettime(CLOCK_MONOTONIC, &starttime);
}

void print() {
cout << "Elapsed time: " << elapsed << endl;
}
};


class PPM {

protected:

int width;
int height;
int maxColor;
vector image;

public:

PPM()
{
width = 0;
height = 0;
maxColor = 0;
}

int readfile(char *filename);

int writefile(char* filename);

void printsize()
{
cout << "Image height: " << height << endl;
cout << "Image width: " << width << endl;
cout << "Image vector size: " << image.size() << endl;
}

void invertpixel()
{
for (vector::iterator it = image.begin(); it != image.end(); ++it) {
*it = 0x00FFFFFF - *it;
}
}
};

int
PPM::readfile(char* filename)
{
ifstream fin;
char buf[256];
unsigned char red, green, blue;
int i, j, rgb;

fin.open(filename);
if (!fin.is_open()) return 1;

fin.getline(buf, 256);
if (buf[0] != 'P' || buf[1] != '6') {
fin.close();
return 1;
}
while (fin.peek()=='#') fin.getline(buf, 256);

fin >> width >> height >> maxColor;
fin.getline(buf, 256); // discard rest of line

image.resize(height*width);
for (i = 0; i < height; i++) {
for (j = 0; j < width; j++) {
fin >> red >> green >> blue;
rgb = ((int)blue & 0x000000ff) << 16 |
((int)green & 0x000000ff ) << 8 |
((int)red & 0x000000ff) | 0x00000000;
image[i * width + j] = rgb;
}
}

fin.close();
return 0;
}

int
PPM::writefile(char* filename)
{
ofstream fout;
stringstream ss;
unsigned char red, green, blue;
int i, j, rgb;

fout.open(filename);
if (!fout.is_open()) return 1;

fout << "P6" << endl;
ss << width << " " << height;
fout << ss.str() << endl;
fout << "255" << endl;

for (i = 0; i < height; i++) {
for (j = 0; j < width; j++) {
rgb = image[i * width + j];
red = (rgb & 0x000000FF);
green = (rgb & 0x0000FF00) >> 8;
blue = (rgb & 0x00FF0000) >> 16;
fout << red << green << blue;
}
}

fout.close();
return 0;
}
void
usage(char *name,
int mode)
{
cerr << "Usage: " << name << " [-d] infile outfile" << endl;
exit(mode);
}

int
main(int argc,
char *argv[])
{
int c, result = 0;
int dflag = 0, iflag = 0;
PPM ppm;
Timer timer;

if (argc == 1)
usage(argv[0], 0);

opterr = 0;
while ((c = getopt( argc, argv, "di")) != -1) {
switch (c) {
case 'd':
dflag = 1;
break;
case 'i':
iflag = 1;
break;
case '?':
case 'h':
usage(argv[0], 0);
break;
}
}

if (optind != argc-2) {
cerr << "ERROR: An input file and an output file must be specified" << endl;
usage(argv[0], 1);
}


timer.start();
if (ppm.readfile(argv[optind++])) {
cerr << "ERROR: Failed to read input file" << endl;
exit(1);
}

if (dflag) ppm.printsize();

if (iflag) ppm.invertpixel();

if (ppm.writefile(argv[optind])) {
cerr << "ERROR: Failed to write output file" << endl;
exit(1);
}

timer.stop();
timer.print();

exit(0);
}

Here is an alternative way of reading and writing an image file using mmap()

#include <fcntl.h>
#include <sys/mman.h>
#include <string.h>
#include <stdio.h>

int
PPM::readfile(char *filename)
{
FILE *fp;
struct stat statbuf;
char buf[256];
int *pwidth = &width;
int *pheight = &height;

if ((fp = fopen(filename, "r")) == NULL) return 1;

do {
fgets(buf, 256, fp);
} while(buf[0] == '#');

if (buf[0] != 'P' || buf[1] != '6') {
fclose(fp);
return 1;
}

do {
fgets(buf, 256, fp);
} while(buf[0] == '#');

sscanf(buf, "%u %u", pwidth, pheight);
fscanf(fp, "%u\n", &maxColor);

image.resize(height*width);
long offset = ftell(fp);
int fd = fileno(fp);
fstat(fd, &statbuf);

unsigned char* data = (unsigned char*)mmap(0, statbuf.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (data == MAP_FAILED) {
fprintf(stderr, "ERROR: mmap failed. %s\n", strerror(errno));
fclose(fp);
return 1;
}

unsigned char* data_offset = (unsigned char*)(data + offset);
unsigned char red, green, blue;
int i = 0, o = 0;
for (int y = 0; y < height * width; y++) {
blue = data_offset[i++];
green = data_offset[i++];
red = data_offset[i++];
image[o++] = red << 16 | green << 8 | blue;
}

munmap(data, statbuf.st_size);
close(fd);

return 0;
}

int
PPM::writefile(char* filename)
{
unsigned int rgb;
int result;

int fd = open(filename, O_RDWR | O_CREAT | O_TRUNC, (mode_t)0600);
if (fd == -1) {
fprintf(stderr, "ERROR: Cannot create output file\n");
return 1;
}

char buf[256];
sprintf(buf, "P6\n%d %d\n255\n", width, height);
int bufsize = strlen(buf);

int mmap_size = bufsize + (height * width * 3);
if (((result = lseek(fd, mmap_size - 1, SEEK_SET)) == -1) ||
((result = write(fd, "", 1)) == -1)) {
fprintf(stderr, "ERROR: Cannot write null at end of output file\n");
close(fd);
return 1;
}

unsigned char* data = (unsigned char*)mmap(0, mmap_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
if (data == MAP_FAILED) {
fprintf(stderr, "ERROR: mmap failed. %s\n", strerror(errno));
close(fd);
return 1;
}

int o = 0;
for (int i = 0; i < bufsize; i++)
data[o++] = buf[i];

for (int i = 0; i < height * width; i++) {
rgb = image[i];
data[o++] = rgb & 0xFF;
data[o++] = (rgb >> 8) & 0xFF;
data[o++] = (rgb >> 16) & 0xFF;
}

munmap(data, mmap_size);
close(fd);

return(0);
}

Note the need to seek to the end of the new file and write a null before calling mmap() when writing a file. Also MAP_SHARED must be used instead of MAP_PRIVATE otherwise your file will contain nothing but zeros.

Well, that is all I have time for at present. In my next post on this subject I will implement a 4-connected thresholding union-find algorithm.
 

Coming Soon - Software Engineering PE

In August 2009 the NCEES Board of Directors approved moving forward with the development of a Principles and Practice of Engineering examination, commonly known as the PE (Professional Engineer) exam, for the discipline of software engineering.

NCEES is the umbrella organization for the engineering and surveying jurisdictional licensing boards in the United States. It develops and administers examinations used for engineering licensure. The prerequisite for NCEES to consider initiating a PE examination in a new discipline includes written requests from no fewer than 10 state licensing boards that can demonstrate a need for the examination in their jurisdictions. The requests must include proof of such need, estimate of usage, and evidence that knowledge areas and skills are not adequately measured in an existing examination.

In addition, no new discipline can be added to the examination program unless there is an ABET-accredited degree in that discipline. The jurisdictional licensing boards requesting the examination were those with significant amounts of software engineering industry and those that have institutions granting ABET-accredited degrees in software engineering.

Over the past decade there have been several efforts to establish a path to professional practice licensure for software engineers in the United States. These efforts failed for various reasons. The main obstacle was the lack of a reasonable number of ABET-accredited programs offering an undergraduate degree in software engineering. This is no longer tha case as there are now a number of ABET-accredited software engineering undergraduate degree programs available in the United States.

In a report from the IEEE-USA Licensure and Registration committee, it was noted that software engineering professional licensure affects both US IEEE members and society at large. The central issue is one of protection of public safety, health and welfare, since the purpose of licensure is to establish competence to practice in an area and not to measure the level of an individual’s expertise above the competency threshold. A large number of technologies in existence today are directly impacted by software engineering, including safety critical artifacts such as real-time controllers and logistics systems. In addition other countries such as Canada require licensing of software engineers.

In July 2007, a group of stakeholder organizations including members of NSPE (National Society of Professional Engineers), IEEE, and TBPE (Texas Board of Professional Engineers) established the Software Engineering Licensure Consortium (SELC.) SELC developed a software engineering licensure needs document that was distributed to licensing boards. It also conducted a survey to determine need and interest in establishing a software engineering licensure path.

What comes next? The next phase is known as a Professional Activities and Knowledge Study (PAKS). The PAKS process will include the development of surveys and meetings with licensed engineers who practice software engineering that will ultimately result in a specification of the content for the software engineering licensure examination. Once the specification is developed, a committee of software engineers is formed to develop examination questions under the oversight of NCEES. After NCEES receives and approves the committee’s software engineering PE examination, it will be each up to individual licensing boards whether they license software engineers in their state or territory.

Professional licensure for an engineering discipline is established by each U.S. jurisdiction at their discretion. The existence of the software engineering PE examination does not automatically mean that each jurisdiction will approve a software engineer license since each board makes such a decision based on the needs (and politics) of their individual jurisdictions.

Hopefully the PAKS process will only take a year or two and the resulting examination will be approved by NCEES without delay. Meanwhile software engineers should start canvasing their state engineering licensing boards to approve licenses in this discipline once the software engineering PE examination is available.

Project Plymouth

Plymouth is the codename for a freedesktop.org project started in 2007 by Ray Strobe of Redhat to develop a graphical application to display a flicker free animated splash screen during the boot process while logging console text output to a log file. Fedora 10 (Cambridge) was the first release of Fedora to contain Plymouth. Development work is actively ongoing and the current release is 0.71.

Plymouth is intended to be a replacement for rhgb (Red Hat Graphical Boot) which is currently used by Red Hat to provide a graphical boot display. If rhgb is part of the kernel command line, rhgb is started early in the boot process by /etc/sysinit. rhgb starts an X server for display :1 on one virtual terminal so that it avoids conflict with the regular X server which may be starting for display :0 on another virtual terminal. It also creates a Unix domain socket (/etc/rhgb/temp/rhgb-socket) so that boot scripts can communicate with it. As boot scripts execute, they can use rhgb-client to send messages to rhgb, which then updates the text and progress display. When the system is finished booting, rhgb-client is invoked with the --quit option to send a terminate request to rhgb. The user is then switched to the X server used by the display manager. Unfortunately the sequence of switching from text mode to rhgb's X server to text mode to the display manager's X server can cause significant screen flickering. Another major drawback of rhgb is that boot messages are not logged.

Plymouth is designed to work on systems with DRM (Direct Rendering Manager) kernel modesetting drivers. DRM is a component of the Direct Rendering Infrastructure project. It consists of two kernel modules, a generic DRM driver, and another which has support for the specific graphics card hardware. This pair of drivers allows a userspace client direct access to the graphics card hardware. See here for further information on DRM mode setting. The idea behind Plymouth is that very early on in the boot process the native video display mode for the system is set by a kernel mode setting driver. In turn Plymouth uses that mode, and that mode remains the same during the entire boot process up to and after an X server starts. For systems without kernel modesetting drivers, there is a fallback text mode which is the familiar tricolor blue/white/black progress bar. Plymouth also drops back to this text mode if the default plugin fails for whatever reason.

Kernel modesetting drivers are still in active development and somewhat buggy. Currently, only Radeon R500 and higher series graphics cards support kernel modesetting by default. There is work in progress to provide kernel modesetting support for R100 and R200 graphics cards. Intel kernel modesetting drivers exist but are not turned on by default. Support for kernel modesetting in the nVidia graphic cards via the Nouveau driver is still experimental. If you end up with nothing but a black screen during boot up, or a screen with nothing but random noise on it, try adding nomodeset to the kernel command line to disable kernel mode setting.

If there is no suitable kernel modesetting driver available for your particular graphics card or you want to set an explicit mode, you can add the string vga=XXX to the kernel command line. The kernel command line option vga=ask invokes the the built-in vesa framebuffer driver, displays a list of supported modes and asks you to select a mode. It then boots the kernel using this mode. The kernel command line option vga=mode, where mode is either a 4 digit hexadecimal with a leading zero and no letter 'x' or a 3 digit decimal number, enables you to set a specific mode.

How can you tell what particular modes are available and which will work best for you? This really depends on the type of graphics card that you have in your system, and the amount of video memory available. The only way is to experiment with different modes.

The following table shows the mode numbers you can input at the vga= prompt using hexadecimals



and here is the same table using decimal numbers.



Note that 8 bits = 256 colors, 15 bits [5:5:5] = 32,768 colors, 16 bits [5:6:5] = 65,536 colors and 24 bits [8:8:8] = 16.8 million colors. Additional modes are at the discretion of the graphics card manufacturer, as the VESA 2.0 specification only defines modes up to 0x31F. For more information about VESA modes, see this article about VESA BIOS Extension compliant graphic cards.

Plymouth works with themes which are analogous to screensavers that are displayed at boot time. Fedora 11 shipped with three graphical themes solar, fade-in and spinfinity, and two non-graphical themes text and details. The text theme is the default theme which is displayed if another theme fails for whatever reason.

The terminology and technology around themes and plugins has evolved as the project progressed. The version of Plymouth that shipped in Fedora 10 was based on a plugin system where each splash screen had to be coded from scratch. This problem was recognized and for Fedora 11 Plymouth went through a major rewrite whereby it now supports themes which in turn use standard plugins. Thus theme developers can now focus on the theme graphics rather than having to do raw coding.

Currently there are five themes in the Fedora repositories. Charge is the default theme for Fedora 11 (Leonidas). Spinfinity is a throbber that moves in a path shaped like the infinity sign. Fade-In shows the Fedora logo fading in and out in a star field. details shows the classic scrolling output from the boot process. text is the fallback bottom of the screen tricolor theme. Solar, my personal favorite to date and the default theme for Fedora 10, was not included in Fedora 11. It displays a planet with exploding pulsars.

To install all the Plymouth themes in the Fedora repositories:

# yum -y install plymouth-theme-*

Installed Plymouth themes can be listed use the plymouth-set-default-theme script:

# /usr/sbin/plymouth-set-default-theme --list
charge
details
fade-in
spinfinity
text

Theme files are stored in the /usr/share/plymouth/themes subdirectory.

# ls /usr/share/plymouth/themes/
charge default.plymouth details fade-in spinfinity text

Note that default.plymouth is a symbolic link to the actual designed default theme.

There are two types of plugins: splash and control. There can only be one splash plugin in use at a time. A splash plugin is what draws the splash screen, asks for a password, displays messages, and more. A theme calls a splash plugin to do the actual work. For example, here is a listing of the files associated with the charge theme.

$ ls /usr/share/plymouth/themes/charge
box.png progress-01.png progress-07.png progress-13.png throbber-00.png throbber-06.png throbber-12.png
bullet.png progress-02.png progress-08.png progress-14.png throbber-01.png throbber-07.png throbber-13.png
charge.plymouth progress-03.png progress-09.png progress-15.png throbber-02.png throbber-08.png throbber-14.png
entry.png progress-04.png progress-10.png progress-16.png throbber-03.png throbber-09.png throbber-15.png
lock.png progress-05.png progress-11.png progress-17.png throbber-04.png throbber-10.png
progress-00.png progress-06.png progress-12.png progress-18.png throbber-05.png throbber-11.png

The theme configuration file that is read by plymouthd is the name of the theme with a .plymouth extension. In this case it is charge.plymouth.

$ cat /usr/share/plymouth/themes/charge/charge.plymouth
[Plymouth Theme]
Name=Charge
Description=A theme that features the shadowy hull of a Fedora logo charge up and and finally burst into into full form.
ModuleName=two-step

[two-step]
ImageDir=/usr/share/plymouth/themes/charge
HorizontalAlignment=.5
VerticalAlignment=.5
Transition=none
TransitionDuration=0.0
BackgroundStartColor=0x416fa7
BackgroundEndColor=0x4b83c1

This theme calls the two-step plugin to do the actual work of displaying the theme. The two-step plugin expects a certain number and type of image files with specific names. Various directives can be passed to plugins; the number and type being plugin specific. For example different type of transitions can be specified for the two-step plugin using the Transition directive, i.e. fade-over, cross-fade and merge-fade.

Some plugins did not ship with Fedora 11. One such plugin is the label plugin. It is not part of initrd but is loadable once the root filesystem is mounted. It is implicitly loaded when a splash plugin attempts to display text. After label is loaded, it uses pango and cairo to handle message localization.

Another such plugin is script which supports a scripting language for themes. It supports two basic objects, i.e. Image and Sprite. If you are familiar with Javascript or the C language you should be comfortable with the syntax and idiom. Note that the scripting language is undergoing rapid development at present with a view to making it more object orientated so you may have to read the git logs or the source code to figure out what is or is not supported.

For examples of scripted themes, I recommend you look at the sources for the Vizta or the Dandelion themes. These themes were both developed by Charlie Brie, a research assistant at the University of Manchester UK, who is the main developer behind the scripting language. If you want to try these themes out on Fedora 11, you will have to import the sources from the Plymouth Git tree, configure, rebuild and install on your system.

The two main binaries involved in Plymouth are /sbin/plymouthd, a daemon that does most of the actual work by displaying the splash screen and logging the boot session, and /bin/plymouth which is the interface to /sbin/plymouthd. Unfortunately no man page is supplied for /bin/plymouth but there is some information in the /usr/share/doc/plymouth-0.7.0 subdirectory. Both have a number of useful options.

$ /sbin/plymouthd --help
Boot splash control server
USAGE: plymouthd [OPTION...]
Options:
--help This help message
--attach-to-session Redirect console messages from screen to log
--no-daemon Do not daemonize
--debug Output debugging information
--mode= Mode is one of: boot, shutdown

$ /bin/plymouth --help
Boot splash control client
USAGE: plymouth [OPTION...] [COMMAND [OPTION...]...]

Options:
--help This help message
--debug Enable verbose debug logging
--newroot= Tell boot daemon that new root filesystem is mounted
--quit Tell boot daemon to quit
--ping Check of boot daemon is running
--sysinit Tell boot daemon root filesystem is mounted read-write
--show-splash Show splash screen
--hide-splash Hide splash screen
--ask-for-password Ask user for password
--ignore-keystroke= Remove sensitivity to a keystroke
--update= Tell boot daemon an update about boot progress
--details Tell boot daemon there were errors during boot
--wait Wait for boot daemon to quit

Available commands:
ask-for-password Ask user for password
ask-question Ask user a question
message Display a message
watch-keystroke Become sensitive to a keystroke
pause-progress Pause boot progress bar
unpause-progress Unpause boot progress bar
report-error Tell boot daemon there were errors during boot
quit Tell boot daemon to quit

Options for ask-for-password command:
--command= Command to send password to via standard input
--prompt= Message to display when asking for password
--number-of-tries= Number of times to ask before giving up (requires --command)
--dont-pause-progress Don't pause boot progress bar while asking

Options for ask-question command:
--command= Command to send the answer to via standard input
--prompt= Message to display when asking the question
--dont-pause-progress Don't pause boot progress bar while asking

Options for message command:
--text= The message text

Options for watch-keystroke command:
--command= Command to send keystroke to via standard input
--keys= Keys to become sensitive to

Options for quit command:
--retain-splash Don't explicitly hide boot splash on exit

In Fedora /usr/bin/rhgb-client is a symbolic link to /usr/bin/plymouth

One way to experiment with Plymouth is to invoke it from runlevel 2 or 3. For example, here is a simple script to display the default theme for 5 seconds, ask for a password, and ask for your name before finally quitting.

#!/bin/sh

# first check that we are in an appropriate runlevel
rlevel=$(runlevel | cut -d " " -f 2)
if [[ "$rlevel" != "2" && "$rlevel" != "3" ]]
then
echo "ERROR: You must be at runlevel 2 or 3"
exit 1
fi

echo "Testing plymouth default theme ..."
plymouthd
sleep 1

# check if the plymouthd daemon is alive
plymouth --ping
if [[ $? -eq 1 ]]
then
echo "ERROR: Plymouth daemon not running"
exit 1
fi

# show the default splash screen for 5 seconds
plymouth --show-splash
sleep 5

plymouth --ask-for-password
sleep 2

# using a command rather than an option
plymouth ask-question --prompt="What is your name?"
sleep 5

plymouth --quit

echo "Done ..."
exit 0

Note that not all plugins support every command and option at the present time. The above script works with the solar theme which uses the space-flares plugin. However this plugin does not support the message command for example. A useful option which is missing from plymouth would be an option to enumerate which commands were supported.

Plymouth is not really designed to be built from source by end users. For it to work correctly, it has to be integrated into the underlying distribution. Because it starts so early in the boot process, it needs to be added to a distribution's initrd (initial ram disk) and the distribution needs to interface with plymouthd to tell it how the boot is progressing. For example, here is the nash script in my Fedora 11 initrd. Notice how the Plymouth splash screen is called as soon as a console is available and also several other times in the script.

lsinitrd /boot/initrd-2.6.29.5-191.fc11.x86_64.img
.........................
#!/bin/nash
mount -t proc /proc /proc
setquiet
echo Mounting proc filesystem
echo Mounting sysfs filesystem
mount -t sysfs /sys /sys
echo Creating /dev
mount -o mode=0755 -t tmpfs /dev /dev
mkdir /dev/pts
mount -t devpts -o gid=5,mode=620 /dev/pts /dev/pts
mkdir /dev/shm
mkdir /dev/mapper
echo Creating initial device nodes
mknod /dev/null c 1 3
mknod /dev/zero c 1 5
mknod /dev/systty c 4 0
mknod /dev/tty c 5 0
mknod /dev/console c 5 1
mknod /dev/ptmx c 5 2
mknod /dev/fb c 29 0
mknod /dev/hvc0 c 229 0
mknod /dev/tty0 c 4 0
mknod /dev/tty1 c 4 1
mknod /dev/tty2 c 4 2
mknod /dev/tty3 c 4 3
mknod /dev/tty4 c 4 4
mknod /dev/tty5 c 4 5
mknod /dev/tty6 c 4 6
mknod /dev/tty7 c 4 7
mknod /dev/tty8 c 4 8
mknod /dev/tty9 c 4 9
mknod /dev/tty10 c 4 10
mknod /dev/tty11 c 4 11
mknod /dev/tty12 c 4 12
mknod /dev/ttyS0 c 4 64
mknod /dev/ttyS1 c 4 65
mknod /dev/ttyS2 c 4 66
mknod /dev/ttyS3 c 4 67
daemonize --ignore-missing /bin/plymouthd
/lib/udev/console_init tty0
plymouth --show-splash
echo Setting up hotplug.
hotplug
echo Creating block device nodes.
mkblkdevs
echo Creating character device nodes.
mkchardevs
echo Making device-mapper control node
mkdmnod
modprobe scsi_wait_scan
rmmod scsi_wait_scan
mkblkdevs
echo Scanning logical volumes
lvm vgscan --ignorelockingfailure
echo Activating logical volumes
lvm vgchange -ay --ignorelockingfailure vg_ultra
resume /dev/mapper/vg_ultra-lv_swap
echo Creating root device.
mkrootdev -t ext4 -o defaults,ro /dev/mapper/vg_ultra-lv_root
echo Mounting root filesystem.
mount /sysroot
cond -ne 0 plymouth --hide-splash
echo Setting up other filesystems.
setuproot
loadpolicy
plymouth --newroot=/sysroot
echo Switching to new root and running init.
switchroot
echo Booting has failed.
sleep -1
init

During the boot progress the boot status is regularly updated with strings signifying what is happening. Plugins can listen to these if they choose to but they are generally ignored in the current plugins, and are only used for calculating the boot time estimation. In Fedora 11, For example the rc.sysinit script includes several calls to plymouth to hide or show the splash screen according to whether a password is needed to access a filesystem or a filesystem is to be relabeled by selinux.

So how does Plymouth know when to quit? Actually, it has no way of knowing. It just keeps on going until it receives a quit message. In the case of Fedora 11, the /etc/event.d/quit-plymouth script sends the quit message.

# quit-plymouth - script to stop boot splash
#
# This service triggers plymouth to quit when we reach the
# end of the boot cycle. We start on 'stopping rcX' to make sure
# this completes before the getty starts.
# prefdm handles quit differently, though.

start on runlevel S
start on stopping rc2
start on stopping rc3
start on stopping rc4

script
/usr/bin/plymouth quit || :
end script

A special case is when a user boots to single user. In this case the /etc/event.d/rcS-sulogin script is executed.

# rcS-sulogin - "single-user" runlevel compatibility
#
# This task runs /bin/bash during "single-user" mode,
# then continues to the default runlevel.

start on runlevel S

stop on runlevel

console owner
script
runlevel --set S >/dev/null || true
plymouth --hide-splash || true
exec /bin/bash
end script
post-stop script
if [ "$1" = "S" ]; then
runlevel=$(/bin/awk -F ':' '$3 == "initdefault" && $1 !~ "^#" { print $2 }' /etc/inittab)
[ -z "$runlevel" ] && runlevel="3"
exec telinit $runlevel
fi
end script

What is not commonly known is that you can also use Plymouth to provide a splash screen during system shutdown or reboot. This is done in Fedora 11 via the /etc/event.d/plymouth-shutdown script. as shown below.

# plymouth-shutdown - put up shutdown splash
#
# This service triggers plymouth to put up a splash
# when leaving runlevel 5.

start on stopped prefdm

console output
script
set $(runlevel || true)
if [ "$2" != "0" ] && [ "$2" != "6" ]; then
exit 0
fi

/sbin/plymouthd --mode=shutdown || exit 1
/bin/plymouth --sysinit
/bin/plymouth --show-splash
if [ "$2" = "0" ]; then
/bin/plymouth message --text="Shutting down..."
elif [ "$2" = "6" ]; then
/bin/plymouth message --text="Restarting..."
fi
end script

Console boot messages are redirected to a pseudo-terminal which is created very early on in the boot process. These messages are buffered until filesystems are fully mounted. Then the buffer is dumped to /var/log/boot.. In either text or graphics mode, the boot messages are obscured. However you can see these messages at any time during boot sequence by hitting the ESC key.

One of the side effects of changing Plymouth themes is that you have to generate a new initrd image. Usually this is done using the mkinird script. However there is an alternative to doing this. You can modify your existing initrd image to remove any Plymouth-related files and create a second initrd image which contains just the Plymouth-related files. When you change a theme, only the Plymouth image needs to be generated. You have to modify grub.conf to load both images when booting. Here is a stanza from my grub.conf which does just that.

title Graphical Boot (Fedora 2.6.29.6-217.2.16.fc11.x86_64)
root (hd0,1)
kernel /vmlinuz-2.6.30.5-43.fc11.x86_64 ro root=/dev/mapper/vg_ultra-lv_root rhgb quiet nopat vga=0x37b 2
initrd /initrd-2.6.30.5-43.fc11.x86_64.img /initrd-plymouth.img

Here is a shell script which will generate the two images. It is based on existing scripts in the Plymouth codebase.

#!/bin/bash
#
#
# FPMurphy 9/12/2009
#

[ -z "$TMPDIR" ] && TMPDIR="/var/tmp"

[ -z "$LIBEXECDIR" ] && LIBEXECDIR="/usr/libexec"
[ -z "$DATADIR" ] && DATADIR="/usr/share"
[ -z "$PLYMOUTH_PLUGIN_PATH" ] && PLYMOUTH_PLUGIN_PATH="$(plymouth --get-splash-plugin-path)"
[ -z "$PLYMOUTH_LOGO_FILE" ] && PLYMOUTH_LOGO_FILE="/usr/share/plymouth/bizcom.png"
[ -z "$PLYMOUTH_THEME_NAME" ] && PLYMOUTH_THEME_NAME=$(plymouth-set-default-theme)

[ -z "$PLYMOUTH_IMAGE_FILE" ] && PLYMOUTH_IMAGE_FILE="/boot/initrd-plymouth.img"
[ -z "$IMAGE_FILE" ] && IMAGE_FILE="/boot/initrd-$(uname -r).img"

if [ -z "$PLYMOUTH_POPULATE_SOURCE_FUNCTIONS" ]; then
if [ -f "${LIBEXECDIR}/initrd-functions" ]; then
PLYMOUTH_POPULATE_SOURCE_FUNCTIONS="${LIBEXECDIR}/initrd-functions"
fi
if [ -f "${DATADIR}/dracut/dracut-functions" ]; then
PLYMOUTH_POPULATE_SOURCE_FUNCTIONS="${DATADIR}/dracut/dracut-functions"
fi
fi

if [ -n "$PLYMOUTH_POPULATE_SOURCE_FUNCTIONS" ]; then
source $PLYMOUTH_POPULATE_SOURCE_FUNCTIONS
fi

if [ " $(type -t inst) " != " function " ]; then
echo "Need 'inst' function, try setting PLYMOUTH_POPULATE_SOURCE_FUNCTIONS to a file that defines it" 1>&2
exit 1
fi

if [ " $(type -t set_verbose) " != " function " ]; then
function set_verbose { true; }
fi

Function usage() {
local output="/dev/stdout"
local rc=0
if [ "$1" == "error" ]; then
output="/dev/stderr"
rc=1
fi

echo "usage: plymouth_setup_initrds [ --verbose | -v ]" > $output
exit $rc
}

verbose=false
INITRDDIR=""
while [ $# -gt 0 ]; do
case $1 in
--verbose|-v)
verbose=true
;;
--help|-h)
usage normal
;;
*)
usage error
break
;;
esac
shift
done
set_verbose $verbose || :

CURRENTDIR=`pwd`
INITRDDIR=`mktemp -d ${TMPDIR}/initrd.XXXXXX`
[ -z "$INITRDDIR" ] && {
echo "mktemp failed"
exit 1
}

mkdir -p ${INITRDDIR}${DATADIR}/plymouth/themes
inst /sbin/plymouthd $INITRDDIR /bin/plymouthd
inst /bin/plymouth $INITRDDIR
inst ${DATADIR}/plymouth/themes/text/text.plymouth $INITRDDIR
inst ${PLYMOUTH_PLUGIN_PATH}/text.so $INITRDDIR
inst ${DATADIR}/plymouth/themes/details/details.plymouth $INITRDDIR
inst ${PLYMOUTH_PLUGIN_PATH}/details.so $INITRDDIR
inst ${PLYMOUTH_LOGO_FILE} $INITRDDIR
inst /etc/system-release $INITRDDIR
if [ -z "$PLYMOUTH_THEME_NAME" ]; then
echo "No default plymouth plugin is set" > /dev/stderr
exit 1
fi

PLYMOUTH_MODULE_NAME=$(grep "ModuleName *= *" ${DATADIR}/plymouth/themes/${PLYMOUTH_THEME_NAME}/${PLYMOUTH_THEME_NAME}.plymouth | sed 's/ModuleName *= *//')

if [ ! -f ${PLYMOUTH_PLUGIN_PATH}/${PLYMOUTH_MODULE_NAME}.so ]; then
echo "The default plymouth plugin (${PLYMOUTH_MODULE_NAME}) doesn't exist" > /dev/stderr
exit 1
fi

inst ${PLYMOUTH_PLUGIN_PATH}/${PLYMOUTH_MODULE_NAME}.so $INITRDDIR

if [ -d ${DATADIR}/plymouth/themes/${PLYMOUTH_THEME_NAME} ]; then
for x in ${DATADIR}/plymouth/themes/${PLYMOUTH_THEME_NAME}/* ; do
[ ! -f "$x" ] && break
inst $x $INITRDDIR
done
fi

if [ -L ${DATADIR}/plymouth/themes/default.plymouth ]; then
cp -a ${DATADIR}/plymouth/themes/default.plymouth $INITRDDIR${DATADIR}/plymouth/themes
fi

# generate the initrd-plymouth image
if [ -f ${PLYMOUTH_IMAGE_FILE} ]; then
mv ${PLYMOUTH_IMAGE_FILE} ${PLYMOUTH_IMAGE_FILE}.bak
fi

echo "Generating image: $PLYMOUTH_IMAGE_FILE"
cd ${INITRDDIR}
rm -f lib*/{ld*,libc*,libdl*,libm*,libz*,libpthread*,libpng*,librt*}
rm -f usr/lib*/libpng*
find . | cpio -H newc --quiet -o | gzip -9 > ${PLYMOUTH_IMAGE_FILE}

cd ${CURRENTDIR}
rm -rf ${INITRDDIR}

# now remove all plymouth items from regular initrd
INITRDDIR=`mktemp -d ${TMPDIR}/initrd.XXXXXX`
[ -z "$INITRDDIR" ] && {
echo "mktemp failed"
exit 1
}
cd ${INITRDDIR}
zcat ${IMAGE_FILE} | cpio -i

rm -f ${INITRDDIR}/bin/plymout*
rm -f ${INITRDDIR}/lib64/libply*
rm -f ${INITRDDIR}/usr/lib64/libply*
rm -rf ${INITRDDIR}/usr/lib64/plymouth
rm -rf ${INITRDDIR}/usr/share/plymouth

echo "Generating image: ${IMAGE_FILE}"
mv ${IMAGE_FILE} ${IMAGE_FILE}.bak
findall . | cpio -H newc --quiet -o | gzip -9 > ${IMAGE_FILE}

cd ${CURRENTDIR}
rm -rf ${INITRDDIR}

exit 0

Here is a listing of the generated /boot/initrd-plymouth.img image.

$ lsinitrd /boot/initrd-plymouth.img
drwx------ 6 root root 0 Sep 12 16:43 .
drwxr-xr-x 2 root root 0 Sep 12 16:43 etc
-rw-r--r-- 1 root root 29 May 11 18:45 etc/fedora-release
lrwxrwxrwx 1 root root 14 Sep 12 16:43 etc/system-release -> fedora-release
drwxr-xr-x 4 root root 0 Sep 12 16:43 usr
drwxr-xr-x 3 root root 0 Sep 12 16:43 usr/lib64
drwxr-xr-x 2 root root 0 Sep 12 16:43 usr/lib64/plymouth
-rwxr-xr-x 1 root root 27242 Sep 12 01:42 usr/lib64/plymouth/details.so
-rwxr-xr-x 1 root root 28471 Sep 12 01:42 usr/lib64/plymouth/text.so
-rwxr-xr-x 1 root root 80032 Sep 12 01:42 usr/lib64/plymouth/space-flares.so
-rwxr-xr-x 1 root root 200218 Sep 12 01:42 usr/lib64/libplybootsplash.so.2.0.0
lrwxrwxrwx 1 root root 25 Sep 12 16:43 usr/lib64/libplybootsplash.so.2 -> libplybootsplash.so.2.0.0
drwxr-xr-x 3 root root 0 Sep 12 16:43 usr/share
drwxr-xr-x 3 root root 0 Sep 12 16:43 usr/share/plymouth
-rw-r--r-- 1 root root 5529 Sep 12 01:42 usr/share/plymouth/bizcom.png
drwxr-xr-x 5 root root 0 Sep 12 16:43 usr/share/plymouth/themes
drwxr-xr-x 2 root root 0 Sep 12 16:43 usr/share/plymouth/themes/details
-rw-r--r-- 1 root root 84 Sep 12 01:42 usr/share/plymouth/themes/details/details.plymouth
drwxr-xr-x 2 root root 0 Sep 12 16:43 usr/share/plymouth/themes/text
-rw-r--r-- 1 root root 98 Sep 12 01:42 usr/share/plymouth/themes/text/text.plymouth
lrwxrwxrwx 1 root root 20 Sep 12 16:43 usr/share/plymouth/themes/default.plymouth -> solar/solar.plymouth
drwxr-xr-x 2 root root 0 Sep 12 16:43 usr/share/plymouth/themes/solar
-rw-r--r-- 1 root root 246 Sep 12 01:42 usr/share/plymouth/themes/solar/progress_bar.png
-rw-r--r-- 1 root root 355666 Sep 12 01:42 usr/share/plymouth/themes/solar/star.png
-rw-r--r-- 1 root root 1896 Sep 12 01:42 usr/share/plymouth/themes/solar/lock.png
-rw-r--r-- 1 root root 165 Sep 12 01:42 usr/share/plymouth/themes/solar/solar.plymouth
-rw-r--r-- 1 root root 296 Sep 12 01:42 usr/share/plymouth/themes/solar/bullet.png
-rw-r--r-- 1 root root 870 Sep 12 01:42 usr/share/plymouth/themes/solar/box.png
-rw-r--r-- 1 root root 350 Sep 12 01:42 usr/share/plymouth/themes/solar/entry.png
drwxr-xr-x 2 root root 0 Sep 12 16:43 lib64
-rwxr-xr-x 1 root root 293522 Sep 12 01:42 lib64/libply.so.2.0.0
lrwxrwxrwx 1 root root 15 Sep 12 16:43 lib64/libply.so.2 -> libply.so.2.0.0
drwxr-xr-x 2 root root 0 Sep 12 16:43 bin
-rwxr-xr-x 1 root root 70256 Sep 12 01:42 bin/plymouth
-rwxr-xr-x 1 root root 110319 Sep 12 01:42 bin/plymouthd

As you can see it only contains Plymouth-related files. It does not contain the label plugin because this is loaded in using dlopen() when needed.

You can debug Plymouth by adding plymouth:debug, plymouth:debug=file:, or plymouth:debug=file:path_to_log_file on the kernel command line. The default file is /var/log/plymouth-debug.log if logging to a file is specified but no file is specified, i.e. option two. Other kernel command line options include console=/dev/what_ever_works to override the default console (/dev/tty0) and plymouth:splash=name_of_theme to override the default theme.

There are also a number of key combinations such as CTRL-L to redraw the screen, CTRL-V to toggle debug mode and CTRL-T to enable text mode. Unfortunately I was unable to get any of these key combinations to work. However the ESC key worked as expected and toggled the display between detailed and the default theme.

Plymouth does all pixel manipulation in software. There is no GPU acceleration. It does not use MMX (Matrix Math Extension) or SSE (Streaming SIMD Extension) hence there is no CPU acceleration either. A plugin that loads a lot of images or does a lot of full screen updates will be slower than one that loads a few images and just updates small parts of the screen. Much of what has been written about Plymouth in the computing press implies that the goal of Plymouth is to provide a faster boot up experience but that is not an explicit design goal of Plymouth.

Well that is about all the useful information on Plymouth that I have time to write about at present. After reading this post, I hope you have a better understanding of Plymouth and how it relates to the boot sequence. Remember however that this project is in active development. and nothing is cast in stone. For example, the inclusion of Dracut, a replacement for nash, in Fedora 12 (Constantine) may affect how Plymouth is invoked. Themes and plug-ins are also rapidly evolving.

As Plymouth is implemented in other GNU/Linux distributions such as Ubuntu, expect to see a flourishing of graphical boot themes from independent authors. I look forward to that day.
 

Ruby D-Bus and Fedora 11

Earlier this year I wrote a number of posts about monitoring and interacting with D-Bus using shell scripts. In this post I use Ruby to monitor and interact with D-Bus enabled applications. If you are unfamiliar with D-Bus, a good starting point is this Freedesktop.org tutorial by the authors of the D-Bus specification.

I used the standard out-of-the-box version of Ruby which comes with Fedora 11, i.e.

$ ruby --version
ruby 1.8.6 (2009-06-08 patchlevel 369) [x86_64-linux]

Those readers who are familiar with Ruby will recognize that this version of Ruby is quite old by Ruby standards and is in maintenance mode with Kirk Haines of Engine Yard being the lead maintainer. Hopefully a 1.9 version of Ruby will be included in the official repositories for Fedora 12.

I also installed the latest version (0.2.9 "I'm not dead") of ruby-dbus which comes as a compressed tarball rather than as a gem. This project was recently taken over by Martin Vidner, who works for Novell in their Prague development group, after a long period of no activity by the original developers. An introduction to and a tutorial on ruby-dbus is available here. It needs to be updated and expanded to include all the interfaces and methods but it is a good introduction to the general concepts.

Introspection is core to D-Bus scripting. Here is one way to introspect the D-Bus system bus and output the resulting introspection data in XML format. The introspection data format is specified by the D-Bus Specification. The introspect_data method does all the heavy lifting.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SystemBus.instance
xml = bus.introspect_data("org.freedesktop.DBus", "/org/freedesktop/DBus/Introspectable")
puts xml

Here is the output from my computer which is running Fedora 11:

<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node>
<interface name="org.freedesktop.DBus.Introspectable">
<method name="Introspect">
<arg name="data" direction="out" type="s"/>
</method>
</interface>
<interface name="org.freedesktop.DBus">
<method name="Hello">
<arg direction="out" type="s"/>
</method>
<method name="RequestName">
<arg direction="in" type="s"/>
<arg direction="in" type="u"/>
<arg direction="out" type="u"/>
</method>
<method name="ReleaseName">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="StartServiceByName">
<arg direction="in" type="s"/>
<arg direction="in" type="u"/>
<arg direction="out" type="u"/>
</method>
<method name="UpdateActivationEnvironment">
<arg direction="in" type="a{ss}"/>
</method>
<method name="NameHasOwner">
<arg direction="in" type="s"/>
<arg direction="out" type="b"/>
</method>
<method name="ListNames">
<arg direction="out" type="as"/>
</method>
<method name="ListActivatableNames">
<arg direction="out" type="as"/>
</method>
<method name="AddMatch">
<arg direction="in" type="s"/>
</method>
<method name="RemoveMatch">
<arg direction="in" type="s"/>
</method>
<method name="GetNameOwner">
<arg direction="in" type="s"/>
<arg direction="out" type="s"/>
</method>
<method name="ListQueuedOwners">
<arg direction="in" type="s"/>
<arg direction="out" type="as"/>
</method>
<method name="GetConnectionUnixUser">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="GetConnectionUnixProcessID">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="GetAdtAuditSessionData">
<arg direction="in" type="s"/>
<arg direction="out" type="ay"/>
</method>
<method name="GetConnectionSELinuxSecurityContext">
<arg direction="in" type="s"/>
<arg direction="out" type="ay"/>
</method>
<method name="ReloadConfig">
</method>
<method name="GetId">
<arg direction="out" type="s"/>
</method>
<signal name="NameOwnerChanged">
<arg type="s"/>
<arg type="s"/>
<arg type="s"/>
</signal>
<signal name="NameLost">
<arg type="s"/>
</signal>
<signal name="NameAcquired">
<arg type="s"/>
</signal>
</interface>
</node>

The following example uses the introspect method instead of the introspect_data method to enumerate a list of the available services using the ListActivatableNames method. I am going to assume that you are familiar with the Ruby language.so concepts such as looping and arrays do not need explanation.

require 'dbus'

bus = DBus::SystemBus.instance
proxy = bus.introspect("org.freedesktop.DBus", "/org/freedesktop/DBus/ListActivatableNames")

bus.proxy.ListActivatableNames[0].each do |service|
puts "#{service}"
end

This is the output:

org.freedesktop.DBus
org.freedesktop.DeviceKit.Disks
org.fedoraproject.Setroubleshootd
com.hp.hplip
org.fedoraproject.Config.Services
org.freedesktop.ConsoleKit
org.gnome.CPUFreqSelector
net.reactivated.Fprint
org.freedesktop.PackageKit
org.freedesktop.DeviceKit
org.freedesktop.NetworkManagerSystemSettings
org.gnome.ClockApplet.Mechanism
org.kerneloops.submit
org.freedesktop.PolicyKit
org.freedesktop.Gypsy
org.gnome.GConf.Defaults
fi.epitest.hostap.WPASupplicant
org.gnome.SystemMonitor.Mechanism
org.freedesktop.DeviceKit.Power
org.freedesktop.nm_dispatcher
org.opensuse.CupsPkHelper.Mechanism

You can query devices via the D-Bus interface to HAL as the following examples demonstrate.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SystemBus.instance
xml = bus.introspect_data("org.freedesktop.Hal", "/org/freedesktop/Hal/devices/computer")
puts xml

Here is what is outputted when this script is executed:

<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node>
<interface name="org.freedesktop.Hal.Device">
<method name="GetAllProperties">
<arg name="properties" direction="out" type="a{sv}"/>
</method>
<method name="SetMultipleProperties">
<arg name="properties" direction="in" type="a{sv}"/>
</method>
<method name="GetProperty">
<arg name="key" direction="in" type="s"/>
<arg name="value" direction="out" type="v"/>
</method>
......
<signal name="InterfaceLockAcquired">
<arg name="interface_name" type="s"/>
<arg name="lock_holder" type="s"/>
<arg name="num_locks" type="i"/>
</signal>
<signal name="InterfaceLockReleased">
<arg name="interface_name" type="s"/>
<arg name="lock_holder" type="s"/>
<arg name="num_locks" type="i"/>
</signal>
</interface>
.......
<interface name="org.freedesktop.Hal.Device.CPUFreq">
<method name="SetCPUFreqGovernor">
<arg name="governor_string" direction="in" type="s"/>
</method>
<method name="SetCPUFreqPerformance">
<arg name="value" direction="in" type="i"/>
</method>
<method name="SetCPUFreqConsiderNice">
<arg name="value" direction="in" type="b"/>
</method>
<method name="GetCPUFreqGovernor">
<arg name="return_code" direction="out" type="s"/>
</method>
<method name="GetCPUFreqPerformance">
<arg name="return_code" direction="out" type="i"/>
</method>
<method name="GetCPUFreqConsiderNice">
<arg name="return_code" direction="out" type="b"/>
</method>
<method name="GetCPUFreqAvailableGovernors">
<arg name="return_code" direction="out" type="as"/>
</method>
</interface>
</node>

Here is an example of how to output the previous data in a more readable form using the interfaces method..

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SystemBus.instance
proxy = bus.introspect("org.freedesktop.Hal", "/org/freedesktop/Hal/devices/computer")

proxy.interfaces.each do |interface|
puts "Interface: #{interface}"
proxy[interface].methods.each do |key,value|
puts " Method: #{key}"
end
proxy[interface].signals.each do |key,value|
puts " Signal: #{key}"
end

Here is an portion of the output: from executing this script.

Interface: org.freedesktop.Hal.Device.CPUFreq
Method: GetCPUFreqConsiderNice
Method: GetCPUFreqGovernor
Method: SetCPUFreqPerformance
Method: GetCPUFreqAvailableGovernors
Method: SetCPUFreqGovernor
Method: GetCPUFreqPerformance
Method: SetCPUFreqConsiderNice
Interface: org.freedesktop.Hal.Device.SystemPowerManagement
Method: Shutdown
Method: SuspendHybrid
Method: SetPowerSave
Method: Hibernate
Method: Reboot
Method: Suspend
Interface: org.freedesktop.Hal.Device
Method: ReleaseInterfaceLock
Method: ClaimInterface
Method: Reprobe
........
Method: SetPropertyBoolean
Signal: InterfaceLockAcquired
Signal: InterfaceLockReleased
Signal: PropertyModified
Signal: Condition
Interface: org.freedesktop.DBus.Introspectable
Method: Introspect

You can easily drill down to individual interfaces and methods. For example, here is one way to list the set of available CPU frequency governors.

#!/usr/bin/ruby
#
# <interface name="org.freedesktop.Hal.Device.CPUFreq">
# <method name="SetCPUFreqGovernor">
# <arg name="governor_string" direction="in" type="s"/>
# </method>
# <method name="SetCPUFreqPerformance">
# <arg name="value" direction="in" type="i"/>
# </method>
# <method name="SetCPUFreqConsiderNice">
# <arg name="value" direction="in" type="b"/>
# </method>
# <method name="GetCPUFreqGovernor">
# <arg name="return_code" direction="out" type="s"/>
# </method>
# <method name="GetCPUFreqPerformance">
# <arg name="return_code" direction="out" type="i"/>
# </method>
# <method name="GetCPUFreqConsiderNice">
# <arg name="return_code" direction="out" type="b"/>
# </method>
# <method name="GetCPUFreqAvailableGovernors">
# <arg name="return_code" direction="out" type="as"/>
# </method>
# </interface>
#

require 'dbus'

bus = DBus::SystemBus.instance
proxy = bus.introspect("org.freedesktop.Hal", "/org/freedesktop/Hal/devices/computer")

interface = proxy["org.freedesktop.Hal.Device.CPUFreq"]
governors = interface.GetCPUFreqAvailableGovernors
puts "Available CPU Freq Governors:"
governors[0].each do |gov|
puts " #{gov}"
end

which produces the following output:

Available CPU Freq Governors:
ondemand
userspace
performance

Lots of useful information can be be obtained from the HAL manager interface. The following script lists all the methods supported by the HAL manager interface and then uses the GetAllDevices method to enumerate the devices on my computer.

require 'dbus'

bus = DBus::SystemBus.instance
proxy = bus.introspect("org.freedesktop.Hal", "/org/freedesktop/Hal/Manager")

puts "HAL manager reports following methods:"
proxy.interfaces[0].each do |interface|
proxy[interface].methods.each do |key, value|
puts " #{key}"
end
end

interface = proxy["org.freedesktop.Hal.Manager"]
devices = interface.GetAllDevices
puts "\nHAL manager reports following devices:"
devices[0].each do |device|
puts " #{device}"
end

Here is a portion of the output when this script is executed

HAL manager reports following methods:
AcquireGlobalInterfaceLock
DeviceExists
CommitToGdl
GetAllDevicesWithProperties
FindDeviceByCapability
NewDevice
SingletonAddonIsReady
FindDeviceStringMatch
GetAllDevices
ReleaseGlobalInterfaceLock
Remove

HAL manager reports following devices:
/org/freedesktop/Hal/devices/net_ba_cf_03_4e_14_ca
/org/freedesktop/Hal/devices/volume_part7_size_115326976
/org/freedesktop/Hal/devices/computer
/org/freedesktop/Hal/devices/storage_model_DVD_Writer_1070d
...............
/org/freedesktop/Hal/devices/acpi_CPU0
/org/freedesktop/Hal/devices/acpi_CPU1
/org/freedesktop/Hal/devices/acpi_CPU2
/org/freedesktop/Hal/devices/acpi_CPU3
/org/freedesktop/Hal/devices/pci_8086_2940
/org/freedesktop/Hal/devices/pci_8086_293e
/org/freedesktop/Hal/devices/usb_device_1d6b_2_0000_00_1a_7_if0
/org/freedesktop/Hal/devices/usb_device_1d6b_2_0000_00_1a_7
/org/freedesktop/Hal/devices/pci_8086_293c
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_2_if0
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_2
/org/freedesktop/Hal/devices/pci_8086_2939
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_1_if0
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_1
/org/freedesktop/Hal/devices/pci_8086_2938
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_0_if0
/org/freedesktop/Hal/devices/usb_device_1d6b_1_0000_00_1a_0
/org/freedesktop/Hal/devices/pci_8086_2937
/org/freedesktop/Hal/devices/pci_8086_294c
/org/freedesktop/Hal/devices/pci_10de_640
/org/freedesktop/Hal/devices/pci_8086_29e1
/org/freedesktop/Hal/devices/pci_8086_29e0

The next example uses the D-Bus interface to Tomboy, a desktop note-taking application, to create a simple note, display it for 5 seconds and then delete the note.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SessionBus.instance
service = bus.service("org.gnome.Tomboy")

tomboy = service.object("/org/gnome/Tomboy/RemoteControl")
tomboy.introspect
tomboy.default_iface = "org.gnome.Tomboy.RemoteControl"
note = tomboy.CreateNamedNote("My Note")[0]
tomboy.SetNoteContents(note, "Hello World")
tomboy.DisplayNote(note)
sleep 5
tomboy.DeleteNote(note)

Note the need to introspect the tomboy object and set the default interface using the default_iface method.

You can also add one or more tags to a Tomboy note and extract comprehensive metadata in XML format as shown in the following example.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SessionBus.instance
service = bus.service("org.gnome.Tomboy")

tomboy = service.object("/org/gnome/Tomboy/RemoteControl")
tomboy.introspect
tomboy.default_iface = "org.gnome.Tomboy.RemoteControl"
note = tomboy.CreateNamedNote("My Note")[0]
tomboy.SetNoteContents(note, "Hello World")
tomboy.AddTagToNote(note, "blog example")
tomboy.DisplayNote(note)
sleep 5
xml = tomboy.GetNoteCompleteXml(note)
tomboy.DeleteNote(note)

puts xml

Here is the XML that is outputted when this script is executed:

<?xml version="1.0" encoding="utf-16"?>
<note version="0.3" xmlns:link="http://beatniksoftware.com/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size" xmlns="http://beatniksoftware.com/tomboy">
<title>My Note</title>
<text xml:space="preserve"><note-content version="0.1">Hello World</note-content></text>
<last-change-date>2009-09-07T15:44:10.2178650-04:00</last-change-date>
<last-metadata-change-date>2009-09-07T15:44:10.2212100-04:00</last-metadata-change-date>
<create-date>2009-09-07T15:44:10.2142940-04:00</create-date>
<cursor-position>37</cursor-position>
<width>450</width>
<height>360</height>
<x>1440</x>
<y>0</y>
<tags>
<tag>blog example</tag>
</tags>
<open-on-startup>False</open-on-startup>
</note>

The next example show you how to monitor D-Bus messages and act upon certain messages. You can do this using D-Bus signals. Suppose, for example, you want to monitor activity relating to Tomboy notes such as when Tomboy notes are created. First of all you need to find out what D-Bus signals Tomboy supports. This is easy to do using introspection as shown in the following script.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SessionBus.instance
service = bus.service("org.gnome.Tomboy")

tomboy = service.object("/org/gnome/Tomboy/RemoteControl")
tomboy.introspect
tomboy.default_iface = "org.gnome.Tomboy.RemoteControl"
puts "List of supported signals:"
tomboy.signals.each do |key,value|
puts " #{key}"
end

Here is the output:

List of supported signals:
NoteSaved
NoteAdded
NoteDeleted

As you can see the version of Tomboy on my computer emits three D-Bus signals - NoteAdded when a note is added, NoteSaved when a note is saved, and NoteDeleted when a note is deleted.

The following script outputs a message each time a Tomboy note is created, saved or deleted.

#!/usr/bin/ruby

require 'dbus'

bus = DBus::SessionBus.instance

match = DBus::MatchRule.new
match.type = "signal"
match.interface = "org.gnome.Tomboy.RemoteControl"
match.path = "/org/gnome/Tomboy/RemoteControl"

bus.add_match(match) do |msg, misc|
puts "#{msg.member} #{msg.params[0]}"
end

main = DBus::Main.new
main << bus
main.run

Here is what is outputted when a single Tomboy note is created, saved and then deleted.

NoteAdded note://tomboy/56587548-7535-4e21-a4b3-3ee85ffa7756
NoteSaved note://tomboy/56587548-7535-4e21-a4b3-3ee85ffa7756
NoteDeleted note://tomboy/56587548-7535-4e21-a4b3-3ee85ffa7756

Well that's all for now. There is a lot more that you can do using ruby-dbus but I will leave it up to you to go explore the possibilities. The best place to discover what is possible is to study the ruby-dbussource code. It is fairly compact and terse but logically laid out.