Showing posts with label Shell. Show all posts
Showing posts with label Shell. Show all posts

KSH93 Bit Manipulation

When programmers think about bitwise manipulation, they usually think about using C or C++ to solve their problem since both programming languages provide a rich set of features which make it easy to perform bitwise manipulation.

However it is possible to just as easily perform such operations using the ksh93 shell.  This post will explain what bitwise manipulation and number conversion facilities are available in ksh93 and provide some hopefully useful utilities and examples.

Many programmers are unaware that ksh93 has builtin support for different numeral radices (AKA arithmetic bases) as shown in the following example.
$ print $(( 10 + 20 ))
30
$ print $(( 2#10 + 20 ))
22
$ print $(( 2#10 + 4#20 ))
10
$
where base 2 is denoted by 2#, base 4 by #4 and so on.  The general case syntax is [base#]n where base is a decimal number between 2 and 36 representing the arithmetic base and n is a number in that base.  You can mix and match bases within expressions.  As you would expect, the default base, i.e. if [base#] is omitted, is decimal(10).

Here is an example of how to convert decimal 255 to base 2, base 8 and base 16 respectively using the typeset command.
$ typeset -i2 a=255
$ echo $a
2#11111111
$ typeset -i8 a=255
$ echo $a
8#377
$ typeset -i16 a=255
$ echo $a
16#ff
$
Note that ksh93 displays the base prefix before each number unless it is a decimal number, i.e. base 10. 

The bitwise manipulation operators in ksh93 are AND, OR, XOR, SHIFT LEFT and SHIFT RIGHT.  There are no SHIFT ROTATE or COMPLEMENT operators.  The following is the list of supported bitwise operators in decreasing order of precedence which, by the way, is the same as in the C programming language.





<< >>Bitwise shift left, shift right
&Bitwise AND
^Bitwise XOR
|Bitwise OR

The following shell script includes both logical and bitwise operators together with the expected output for each statement.  Notice the difference in output between a logical and bitwise operator.
#!/bin/ksh93

integer -i2 a=5
integer -i2 b=6

# expected output 1
print $(( a < b ))
# expected output 0
print $(( a > b ))
# expected output 3
print $(( a ^ b ))
# expected output 4
print $(( a & b ))
# expected output 1
print $(( a && b ))
# expected output 7
print $(( a | b ))
# expected output 1
print $(( a || b ))
A useful feature of bitwise operators is that by using the shift left and shift right bitwise operators you can mimic multiplication or division by 2 or a power of 2.
$ x=4
$ print $((x << 1))
8
$ print $(( x << 2 ))
16
$ print $(( x >> 1 ))
2
$ print $(( x >> 2 ))
1
Here is a fairly simple shell script which uses numeric base conversion and bitwise operators to calculate the network and broadcast addresses for a given IP address and subnet mask.
#!/bin/ksh93

typeset -i2 mask=255

[[ $# != 2 ]] && {
echo "Usage: $0 ipaddress subnetmask"
exit 1
}

SaveIFS=$IFS
IFS=.
typeset -a IParr=($1)
typeset -a NMarr=($2)
IFS=$SaveIFS

typeset -i2 ip1=${IParr[0]}
typeset -i2 ip2=${IParr[1]}
typeset -i2 ip3=${IParr[2]}
typeset -i2 ip4=${IParr[3]}

typeset -i2 nm1=${NMarr[0]}
typeset -i2 nm2=${NMarr[1]}
typeset -i2 nm3=${NMarr[2]}
typeset -i2 nm4=${NMarr[3]}

echo
echo " IP Address: $1"
echo " Subnet Mask: $2"
echo " Network Address: $((ip1 & nm1)).$((ip2 & nm2)).$((ip3 & nm3)).$((ip4 & nm4))"
echo "Broadcast Address: $((ip1 | (mask ^ nm1))).$((ip2 | (mask ^ nm2))).$((ip3 | (mask ^ nm3))).$(( ip4 | (mask ^ nm4)))"
echo

exit 0
Some sample output for this script.
$ ./calculate-address 10.150.12.1 255.255.255.0

IP Address: 10.150.12.1
Subnet Mask: 255.255.255.0
Network Address: 10.150.12.0
Broadcast Address: 10.150.12.255

$ ./calculate-address 10.150.12.1 255.255.254.0

IP Address: 10.150.12.1
Subnet Mask: 255.255.254.0
Network Address: 10.150.12.0
Broadcast Address: 10.150.13.255
The next example calculates the network and broadcast addresses for a Classless Inter-Domain Routing (CIDR) compliant IP address.  Note the use of the shift right operator in calculating the network mask.
#!/bin/ksh93

[[ $# != 1 ]] && {
echo "Usage: $0 ipaddress/netmask"
exit 1
}

SaveIFS=$IFS
IFS="./"
typeset -a IParr=($1)
IFS=$SaveIFS

typeset -i2 ip1=${IParr[0]}
typeset -i2 ip2=${IParr[1]}
typeset -i2 ip3=${IParr[2]}
typeset -i2 ip4=${IParr[3]}
typeset -i2 cidr=${IParr[4]}

typeset -i2 nm1=0 nm2=0 nm3=0 nm4=0

typeset -i quad=$(( cidr / 8 ))
sigbits=$(( cidr % 8 ))
if (( sigbits != 0 )); then
slot=$(( 256 - ( 256 >> $sigbits ) ))
fi

for (( i=1; i < 5; i++ ))
do
nameref nm=nm${i}
if (( quad != 0 )); then
nm=255
(( --quad ))
elif (( quad == 0 )); then
nm=slot
break
fi
done

typeset -i2 mask=255

print
print " IP Address: $((ip1)).$((ip2)).$((ip3)).$((ip4))"
print " CIDR Netmask Mask: $((nm1)).$((nm2)).$((nm3)).$((nm4))"
print " CIDR Network (Route): $((ip1 & nm1)).$((ip2 & nm2)).$((ip3 & nm3)).$((ip4 & nm4))"
print " Broadcast Address: $((ip1 | (mask ^ nm1))).$((ip2 | (mask ^ nm2))).$((ip3 | (mask ^ nm3))).$((ip4 | (mask ^ n
m4)))"
print

exit 0
Sample output:
$ ./calculate-address "192.168.21.12/18"

IP Address: 192.168.21.12
CIDR Network Mask: 255.255.192.0
CIDR Network (Route): 192.168.0.0
Broadcast Address: 192.168.63.255
While both the bash and zsh shells also provide a full set of bitwise operators, only ksh93 and zsh with their builtin support for numeric base conversions makes such bitwise operators truly useful in shell scripts.
 

Monitoring D-Bus

Because D-Bus operates for the most part under the hood invisible to the user, means of monitoring D-Bus messages is important for both activation and debugging purposes.  In this post I examine how to monitor and act on such messages.  I assume that you are relatively familiar with D-Bus.  If not, you should look at my previous posts on D-Bus to gain an understanding of the basic concepts.

Probably the most commonly used tool is the command line utility dbus-monitor which was written by Philip Blundell as part of the reference D-Bus implementation and which is standard on most D-Bus enabled platforms.  It is a very powerful utility but unfortunately it's man page is terse and fails to explain the most powerful feature of dbus-monitor i.e. watch expressions.

Suppose, for example, I want to monitor the creation, saving and deletion of Tomboy notes using D-Bus signals.  To get the list of the supported signals I could introspect Tomboy using qdbus i.e.
$ qdbus org.gnome.Tomboy /org/gnome/Tomboy/RemoteControl | grep signal
signal void org.gnome.Tomboy.RemoteControl.NoteAdded(QString uri)
signal void org.gnome.Tomboy.RemoteControl.NoteDeleted(QString uri, QString title)
signal void org.gnome.Tomboy.RemoteControl.NoteSaved(QString uri)
$
or I could use dbus-monitor to look at the signals generated when Tomboy creates, saves and deletes a note.
$ dbus-monitor type=signal interface="org.gnome.Tomboy.RemoteControl"
signal sender=org.freedesktop.DBus -> dest=:1.102 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameAcquired
string ":1.102"
signal sender=:1.63 -> dest=(null destination) path=/org/gnome/Tomboy/RemoteControl; interface=org.gnome.Tomboy.RemoteControl; member=NoteAdded
string "note://tomboy/0ae6b0b3-9394-44e9-ad4a-46726e68cebc"
signal sender=:1.63 -> dest=(null destination) path=/org/gnome/Tomboy/RemoteControl; interface=org.gnome.Tomboy.RemoteControl; member=NoteSaved
string "note://tomboy/0ae6b0b3-9394-44e9-ad4a-46726e68cebc"
signal sender=:1.63 -> dest=(null destination) path=/org/gnome/Tomboy/RemoteControl; interface=org.gnome.Tomboy.RemoteControl; member=NoteDeleted
string "note://tomboy/0ae6b0b3-9394-44e9-ad4a-46726e68cebc"
string "New Note 1"
$
As you can see Tomboy issues three signals - NoteAdded when a note is added, NoteSaved when a note is saved, and NoteDeleted when a note is deleted.  Interestingly, for some reason NoteDeleted provides both the uri and the title of the note whereas the other two signals only provide the uri.

Suppose I just want to to monitor when a Tomboy note is saved by a user.  I can use a dbus-monitor watch expression for this purpose as shown in the following example.
dbus-monitor "type='signal',sender='org.gnome.Tomboy',interface='org.gnome.Tomboy.RemoteControl',path='/org/gnome/Tomboy/RemoteControl',member='NoteSaved'"
A watch expression is essentially a message filter.  Note the syntax!  Everything between the two double quotes is part of the watch expression.

Unfortunately the syntax of a watch expression is not documented in the dbus-monitor man page.  However from looking at the source code for dbus-monitor, it appears that a watch expression consists of one or more name-value pairs, e.g. sender='org.gnome.Tomboy' separated by commas.  The following are valid names: type, sender, destination, path, interface, member and error_name.  The value of type must be one of the following: signal, method_call, method_return or error. 

Probably the most powerful feature of dbus-monitor is the fact that you are not limited to using just one watch expression at a time.  The following example simultaneously monitors all 3 Tomboy signals and uses awk to parse the output from dbus-monitor and display a meaningful message.
#!/bin/bash

OJECT="'org.gnome.Tomboy'"
IFACE="'org.gnome.Tomboy.RemoteControl'"
DPATH="'/org/gnome/Tomboy/RemoteControl'"

WATCH1="type='signal', sender=${OJECT}, interface=${IFACE}, path=${DPATH}, member='NoteAdded'"
WATCH2="type='signal', sender=${OJECT}, interface=${IFACE}, path=${DPATH}, member='NoteSaved'"
WATCH3="type='signal', sender=${OJECT}, interface=${IFACE}, path=${DPATH}, member='NoteDeleted'"

dbus-monitor "${WATCH1}" "${WATCH2}" "${WATCH3}" | \
awk '
/member=NoteAdded/ { getline; print "Created note " substr($2,7) }
/member=NoteSaved/ { getline; print "Added note " substr($2,7) }
/member=NoteDeleted/ { getline; print "Deleted note " substr($2,7) }
'
Here is the output generated when I clicked on the Tomboy icon to create a new note, waited for the automatic save and then selected the delete option to delete the note.
$ ./test
Created note //tomboy/3da026dc-f6ee-4637-8a94-bec6e2844824"
Added note //tomboy/3da026dc-f6ee-4637-8a94-bec6e2844824"
Deleted note //tomboy/3da026dc-f6ee-4637-8a94-bec6e2844824"
Well that is all for this post.  Enjoy!

P.S. I tested these examples using Fedora 10. I have no idea whether they will work on previous versions of Fedora or other Linux distributions.
 

Scripting Tomboy

Tomboy is an open source GNOME desktop note-taking application which is written in C# and utilizing the Mono runtime, Gtk# and the GtkSpell spell-checker.

The actual release of Tomboy which comes with Fedora 10 is version 0.12.0.  This includes a comprehensive D-Bus interface which makes it possible to create, modify and display Tomcat notes from your shell scripts.  This post provides an overview of the available D-Bus methods and includes a number of examples for you to experiment with.  See my previous post on D-Bus scripting if you are unfamilar with the basic concepts of D-Bus scripting.

First we will list the available objects using qdbus.
$ qdbus org.gnome.Tomboy
/
/org
/org/gnome
/org/gnome/Tomboy
/org/gnome/Tomboy/RemoteControl
Next, we list all the available signals and methods for RemoteControl.
$ qdbus org.gnome.Tomboy /org/gnome/Tomboy/RemoteControl
method QString org.freedesktop.DBus.Introspectable.Introspect()
method bool org.gnome.Tomboy.RemoteControl.AddTagToNote(QString uri, QString tag_name)
method QString org.gnome.Tomboy.RemoteControl.CreateNamedNote(QString linked_title)
method QString org.gnome.Tomboy.RemoteControl.CreateNote()
method bool org.gnome.Tomboy.RemoteControl.DeleteNote(QString uri)
method bool org.gnome.Tomboy.RemoteControl.DisplayNote(QString uri)
method bool org.gnome.Tomboy.RemoteControl.DisplayNoteWithSearch(QString uri, QString search)
method void org.gnome.Tomboy.RemoteControl.DisplaySearch()
method void org.gnome.Tomboy.RemoteControl.DisplaySearchWithText(QString search_text)
method QString org.gnome.Tomboy.RemoteControl.FindNote(QString linked_title)
method QString org.gnome.Tomboy.RemoteControl.FindStartHereNote()
method QStringList org.gnome.Tomboy.RemoteControl.GetAllNotesWithTag(QString tag_name)
method qlonglong org.gnome.Tomboy.RemoteControl.GetNoteChangeDate(QString uri)
method QString org.gnome.Tomboy.RemoteControl.GetNoteCompleteXml(QString uri)
method QString org.gnome.Tomboy.RemoteControl.GetNoteContents(QString uri)
method QString org.gnome.Tomboy.RemoteControl.GetNoteContentsXml(QString uri)
method qlonglong org.gnome.Tomboy.RemoteControl.GetNoteCreateDate(QString uri)
method QString org.gnome.Tomboy.RemoteControl.GetNoteTitle(QString uri)
method QStringList org.gnome.Tomboy.RemoteControl.GetTagsForNote(QString uri)
method bool org.gnome.Tomboy.RemoteControl.HideNote(QString uri)
method QStringList org.gnome.Tomboy.RemoteControl.ListAllNotes()
signal void org.gnome.Tomboy.RemoteControl.NoteAdded(QString uri)
signal void org.gnome.Tomboy.RemoteControl.NoteDeleted(QString uri, QString title)
method bool org.gnome.Tomboy.RemoteControl.NoteExists(QString uri)
signal void org.gnome.Tomboy.RemoteControl.NoteSaved(QString uri)
method bool org.gnome.Tomboy.RemoteControl.RemoveTagFromNote(QString uri, QString tag_name)
method QStringList org.gnome.Tomboy.RemoteControl.SearchNotes(QString query, bool case_sensitive)
method bool org.gnome.Tomboy.RemoteControl.SetNoteCompleteXml(QString uri, QString xml_contents)
method bool org.gnome.Tomboy.RemoteControl.SetNoteContents(QString uri, QString text_contents)
method bool org.gnome.Tomboy.RemoteControl.SetNoteContentsXml(QString uri, QString xml_contents)
method QString org.gnome.Tomboy.RemoteControl.Version()
As a simple example of how to use a published method, we invoke the Version method to return the version of Tomboy that we are using.
$ qdbus org.gnome.Tomboy /org/gnome/Tomboy/RemoteControl org.gnome.Tomboy.RemoteControl.Version
0.12.0
We can use dbus-send instead of qdbus as shown below but, as you can, see qdbus syntax is more compact.  Also note that we have to use the session bus.
$ dbus-send --type=method_call --session --print-reply \
--dest='org.gnome.Tomboy' /org/gnome/Tomboy/RemoteControl \
org.gnome.Tomboy.RemoteControl.Version
0.12.0
In the following example, we create the equivalant of "Hello World" using a note, display it for 5 seconds and then delete the note.
#!/bin/bash
DPATH="/org/gnome/Tomboy/RemoteControl"
INTERFACE="org.gnome.Tomboy.RemoteControl"

TMP=`qdbus org.gnome.Tomboy ${DPATH} ${INTERFACE}.CreateNamedNote "My Note" 2>/dev/null`
RESULT=$?
if [[ $RESULT != 0 ]]
then
exit 1
fi

# figure out note uri string which is of the form
# note://0xaf3356abcdefg
OID=${TMP#note:}

# set the contents of the note using the uid
qdbus org.gnome.Tomboy ${DPATH} ${INTERFACE}.SetNoteContents note:$OID "Hello World"
qdbus org.gnome.Tomboy ${DPATH} ${INTERFACE}.DisplayNote note:$OID
sleep 5
qdbus org.gnome.Tomboy ${DPATH} ${INTERFACE}.DeleteNote note:$OID
Here is a screenshot of the note created by this shell script.



Incidentally, if you create one or more notes by accident while experimenting and need to delete them, the following script will delete all notes.
#!/bin/bash
OJECT="org.gnome.Tomboy"
DPATH="/org/gnome/Tomboy/RemoteControl"
IFACE="org.gnome.Tomboy.RemoteControl"

qdbus ${OJECT} ${DPATH} ${IFACE}.ListAllNotes |
while read TMP
do
echo "$TMP"
OID=${TMP#note:}
qdbus ${OJECT} ${DPATH} ${IFACE}.DeleteNote note:$OID
done
You can also add one or more tags to a note and extract comprehensive metadata in XML format as shown in the following example.
#!/bin/bash
OJECT="org.gnome.Tomboy"
DPATH="/org/gnome/Tomboy/RemoteControl"
IFACE="org.gnome.Tomboy.RemoteControl"

TMP=`qdbus ${OJECT} ${DPATH} ${IFACE}.CreateNamedNote "My Note" 2>/dev/null`
RESULT=$?
OID=${TMP#note:}

qdbus ${OJECT} ${DPATH} ${IFACE}.SetNoteContents note:$OID "Hello World"
qdbus ${OJECT} ${DPATH} ${IFACE}.AddTagToNote note:$OID "blog example"
qdbus ${OJECT} ${DPATH} ${IFACE}.DisplayNote note:$OID
sleep 4
qdbus ${OJECT} ${DPATH} ${IFACE}.GetNoteCompleteXml note:$OID
qdbus ${OJECT} ${DPATH} ${IFACE}.DeleteNote note:$OID

# this is the output from the GetNoteCompleteXml
<?xml version="1.0" encoding="utf-16"?>
<note version="0.3" xmlns:link="http://beatniksoftware.com/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size" xmlns="http://beatniksoftware.com/tomboy">
<title>My Note</title>
<text xml:space="preserve"><note-content version="0.1">Hello World</note-content></text>
<last-change-date>2009-02-08T10:44:46.3741020-05:00</last-change-date>
<last-metadata-change-date>2009-02-08T10:44:46.3940970-05:00</last-metadata-change-date>
<create-date>2009-02-08T10:44:46.3441770-05:00</create-date>
<cursor-position>37</cursor-position>
<width>450</width>
<height>360</height>
<x>3</x>
<y>48</y>
<tags>
<tag>blog example</tag>
</tags>
<open-on-startup>False</open-on-startup>
</note>
You can also create a note in a single call to dbus-send or qdbus by passing the appropriate XML formatted details to the SetNoteCompleteXml method.
#!/bin/bash
OJECT="org.gnome.Tomboy"
DPATH="/org/gnome/Tomboy/RemoteControl"
IFACE="org.gnome.Tomboy.RemoteControl"

TMP=`qdbus ${OJECT} ${DPATH} ${IFACE}.CreateNamedNote "My Note" 2>/dev/null`
RESULT=$?
OID=${TMP#note:}

qdbus ${OJECT} ${DPATH} ${IFACE}.SetNoteCompleteXml note:$OID \
"<?xml version='1.0' encoding='utf-16'?>
<note version='0.3' xmlns:link='http://beatniksoftware.com/tomboy/link'
xmlns:size='http://beatniksoftware.com/tomboy/size'
xmlns='http://beatniksoftware.com/tomboy'>
<title>My Note</title>
<text xml:space='preserve'>
<note-content version='0.1'>Hello World</note-content>
</text>
<tags>
<tag>first tag</tag>
<tag>second tag</tag>#!/bin/bash
</tags>
<open-on-startup>False</open-on-startup>
</note>"

qdbus ${OJECT} ${DPATH} ${IFACE}.DisplayNote note:$OID
sleep 4
qdbus ${OJECT} ${DPATH} ${IFACE}.DeleteNote note:$OID
Interestingly, for some reason or other you cannot set the position or size of the resulting note.  If anybody knows how to overcome this limitation, please let me know.

Well that is all for this particular post as I have to head off to the airport for another boring long flight to Europe.  In my next post I will show you how to handle D-Bus signals in your shell scripts.
 

Scripting D-Bus

D-Bus (Desktop Bus) is a low-latency, low-overhead, easy-to-use message bus technology which supports application launch and linking.  It is primarly used on GNU/Linux desktops but has been ported to other platforms including Microsoft Windows and Apple Mac OS X.  This post provides a quick overview of D-Bus concepts, some history, and some examples of how to use D-Bus in your shell scripts.

Originally both the KDE and GNOME desktop projects used CORBA for inter-application communication.  Over time however, for various reasons, KDE migrated from CORBA to Desktop Comunications Protocol (DCOP) and GNOME migrated to Bonono.  This lead to the situation where GNU/Linux desktop distributions had to support two different inter-application lauch and linking models and many standard desktop applications could not communicate seamlessly with each other.  To ameliorate this unsatisfactory situation, D-Bus (the name was suggested by Harri Porten) was conceived and developed by Red Hat as part of the freedesktop.org project.  The design of D-Bus was heavily influenced by DCOP.  From the start, it was designed to be a replacement for the two competing technologies.   The initial source code module was created by Havoc Pennington in late 2002.  Development was quite slow with many changes to the wire protocol.  However by 2006 the specification was relatively stable.  First GNOME and then KDE made the decision to transition to D-Bus in order to support a single unified applcation linking and lauching technology on GNU/linux desktops.

In many ways D-Bus is similar to Sun Microsystems ToolTalk which is the undelying technology in Common Desktop Environment, and Microsoft's Object Linking and Embedding (OLE) technology.



The basic D-Bus protocol is a low latancy peer-to-peer or client-server binary protocol.  It is not intended for inter-machine use but rather for intra-machine use.  It works in terms of messages rather than byte streams.  A message bus is used when many-to-many communication is desired.  Normally applications communicate via such a message bus but direct application-to-application communication is also possible.

When communicating on a message bus, applications can query which other applications and services are available, as well as activate one on demand.  A daemon, or service, must be launched before any applications can connect to a message bus. This daemon is responsible for keeping track of the applications that are connected and for properly routing messages from source to destination.  The D-Bus specification defines two well-known buses called the system bus and the session bus.  These buses are special in that they have well-defined semantics and associated services.

A message bus can start (launch) an application on behalf of another application.  An application that can be started in this way is called a service and must have a well-known name.  To find an application corresponding to a particular name, the bus daemon looks for service description files which are XML files that map names to applications.  Different message buses will look for these files in different places on a system. Once the application is launched, it registers a service name.  Service names use reverse domain syntax (similar to Java), e.g org.freedesktop.Notifications.

D-Bus is object-orientated and supports the notion of named object interfaces.  As in C++, a method is an operation on an instantiated object.  Methods may have optional typed parameters and may return typed values.  Signals are broadcasts from an object to any interested observer.  All methods, method returns, signals, errors, etc. are encoded as D-Bus messages consisting of a packet header with source and destination addresses, a type signature and the body containing the parameters (for signals and method calls) or the return values (for a method return).  The type signature describes the types of the data contained within the message.  D-Bus uses interfaces to provide a namespacing mechanism for methods.  An interface is a group of related methods and signals, identified by a name which is a series of dot-separated components starting with a reversed domain name.

An application can expose one or more objects.  An object path is used to specify the address of each object.  The syntax of an object path is similar to that of the full pathname of a file on a Unix platform.  For example, /org/freedesktop/Notification is the object path to an object provided by the org.freedesktop.Notification service.  Service names, interface names and object paths do not have to be related in terms of their names but in many cases they are with the with the object path typically being a slashed version of the dotted service name as in the above example.  Messages buses can also expose interfaces, i.e. org.freedesktop.DBus.

The D-Bus specification mandates two special D-Bus interfaces - org.freedesktop.DBus.Introspect and org.freedesktop.DBus.Properties.  For our first example, let us ask the system message bus to list all it's methods.
dbus-send --system --print-reply --reply-timeout=2000 \
--type=method_call --dest=org.freedesktop.DBus \
/org/freedesktop/DBus \
org.freedesktop.DBus.Introspectable.Introspect

method return sender=org.freedesktop.DBus -> dest=:1.51 reply_serial=2
string "<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node>
<interface name="org.freedesktop.DBus.Introspectable">
<method name="Introspect">
<arg name="data" direction="out" type="s"/>
</method>
</interface>
<interface name="org.freedesktop.DBus">
<method name="Hello">
<arg direction="out" type="s"/>
</method>
<method name="RequestName">
<arg direction="in" type="s"/>
<arg direction="in" type="u"/>
<arg direction="out" type="u"/>
</method>
<method name="ReleaseName">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="StartServiceByName">
<arg direction="in" type="s"/>
<arg direction="in" type="u"/>
<arg direction="out" type="u"/>
</method>
<method name="UpdateActivationEnvironment">
<arg direction="in" type="a{ss}"/>
</method>
<method name="NameHasOwner">
<arg direction="in" type="s"/>
<arg direction="out" type="b"/>
</method>
<method name="ListNames">
<arg direction="out" type="as"/>
</method>
<method name="ListActivatableNames">
<arg direction="out" type="as"/>
</method>
<method name="AddMatch">
<arg direction="in" type="s"/>
</method>
<method name="RemoveMatch">
<arg direction="in" type="s"/>
</method>
<method name="GetNameOwner">
<arg direction="in" type="s"/>
<arg direction="out" type="s"/>
</method>
<method name="ListQueuedOwners">
<arg direction="in" type="s"/>
<arg direction="out" type="as"/>
</method>
<method name="GetConnectionUnixUser">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="GetConnectionUnixProcessID">
<arg direction="in" type="s"/>
<arg direction="out" type="u"/>
</method>
<method name="GetAdtAuditSessionData">
<arg direction="in" type="s"/>
<arg direction="out" type="ay"/>
</method>
<method name="GetConnectionSELinuxSecurityContext">
<arg direction="in" type="s"/>
<arg direction="out" type="ay"/>
</method>
<method name="ReloadConfig">
</method>
<method name="GetId">
<arg direction="out" type="s"/>
</method>
<signal name="NameOwnerChanged">
<arg type="s"/>
<arg type="s"/>
<arg type="s"/>
</signal>
<signal name="NameLost">
<arg type="s"/>
</signal>
<signal name="NameAcquired">
<arg type="s"/>
</signal>
</interface>
</node>
"
The following command lists out the names of services which can be activated (launched).
dbus-send --system --print-reply --reply-timeout=2000 \
--type=method_call --dest=org.freedesktop.DBus \
/org/freedesktop/DBus \
org.freedesktop.DBus.ListActivableNames

method return sender=org.freedesktop.DBus -> dest=:1.68 reply_serial=2
array [
string "org.freedesktop.DBus"
string "org.fedoraproject.Config.Services"
string "org.gnome.CPUFreqSelector"
string "org.freedesktop.ConsoleKit"
string "org.freedesktop.PackageKitAptBackend"
string "org.freedesktop.PackageKit"
string "org.freedesktop.NetworkManagerSystemSettings"
string "org.freedesktop.PackageKitTestBackend"
string "org.gnome.ClockApplet.Mechanism"
string "org.freedesktop.PolicyKit"
string "fi.epitest.hostap.WPASupplicant"
string "org.gnome.GConf.Defaults"
string "org.gnome.SystemMonitor.Mechanism"
string "org.freedesktop.nm_dispatcher"
]
Instead of using dbus-send which requires a number of parameters, a simpler utility to use is dbus which is part of the Google dbus-cli package.  Here is some sample output.
bash-3.2$ ./dbus
org.freedesktop.DBus
org.freedesktop.Notifications
org.freedesktop.PowerManagement
com.redhat.imsettings.IMInfo
org.gnome.SessionManager
org.gnome.GConf
org.freedesktop.PackageKit
com.redhat.imsettings
org.gnome.SettingsDaemon
com.redhat.setroubleshoot
org.gnome.ScreenSaver
org.bluez.applet
com.redhat.imsettings.GConf
Another useful utility is qdbus.
# list all GNOME-related  buses 
$ qdbus | grep gnome
org.gnome.SessionManager
org.gnome.SettingsDaemon
org.gnome.ScreenSaver
org.gnome.GConf

# list ScreenSaver interfaces
bash-3.2$ qdbus org.gnome.ScreenSaver
/

# list methods for ScreenSaver interface
$ qdbus org.gnome.ScreenSaver /ScreenSaver
method QString org.freedesktop.DBus.Introspectable.Introspect()
signal void org.gnome.ScreenSaver.ActiveChanged(bool new_value)
signal void org.gnome.ScreenSaver.AuthenticationRequestBegin()
signal void org.gnome.ScreenSaver.AuthenticationRequestEnd()
method void org.gnome.ScreenSaver.Cycle()
method bool org.gnome.ScreenSaver.GetActive()
method uint org.gnome.ScreenSaver.GetActiveTime()
method QStringList org.gnome.ScreenSaver.GetInhibitors()
method bool org.gnome.ScreenSaver.GetSessionIdle()
method uint org.gnome.ScreenSaver.GetSessionIdleTime()
method uint org.gnome.ScreenSaver.Inhibit(QString application_name, QString reason)
method void org.gnome.ScreenSaver.Lock()
signal void org.gnome.ScreenSaver.SessionIdleChanged(bool new_value)
signal void org.gnome.ScreenSaver.SessionPowerManagementIdleChanged(bool new_value)
method void org.gnome.ScreenSaver.SetActive(bool value)
method void org.gnome.ScreenSaver.SimulateUserActivity()
method uint org.gnome.ScreenSaver.Throttle(QString application_name, QString reason)
method void org.gnome.ScreenSaver.UnInhibit(uint cookie)
method void org.gnome.ScreenSaver.UnThrottle(uint cookie)

# retrieve session idle time. Zero in this case as I have been working on this post.
$ qdbus org.gnome.ScreenSaver /ScreenSaver org.gnome.ScreenSaver.GetSessionIdleTime
0

# activate the screen saver - may or may not lock depending on settings
$ qdbus org.gnome.ScreenSaver /ScreenSaver org.gnome.ScreenSaver.SetActive True

# lock screen using the screensaver
$ qdbus org.gnome.ScreenSaver /ScreenSaver org.gnome.ScreenSaver.Lock

# query whether this computer can be suspended?
$ qdbus org.freedesktop.PowerManagement /org/freedesktop/PowerManagement org.freedesktop.PowerManagement.CanSuspend
true
You can also query devices via HAL as the following examples show.
$ dbus-send --system --print-reply --dest=org.freedesktop.Hal \
/org/freedesktop/Hal/devices/computer \
org.freedesktop.DBus.Introspectable.Introspect

method return sender=:1.0 -> dest=:1.106 reply_serial=2
string "<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node>
<interface name="org.freedesktop.DBus.Introspectable">
<method name="Introspect">
<arg name="data" direction="out" type="s"/>
<method>
<interface>
<interface name="org.freedesktop.Hal.Device">
........
<interface>
<interface name="org.freedesktop.Hal.Device.SystemPowerManagement">
........
<interface>
<interface name="org.freedesktop.Hal.Device.CPUFreq">
........
<method name="GetCPUFreqAvailableGovernors">
<arg name="return_code" direction="out" type="as"/>
<method>
<interface>
<node>
"

$ dbus-send --system --print-reply --dest=org.freedesktop.Hal \
/org/freedesktop/Hal/devices/computer \
org.freedesktop.Hal.Device.CPUFreq.GetCPUFreqAvailableGovernors

method return sender=:1.0 -> dest=:1.109 reply_serial=2
array [
string "ondemand"
string "userspace"
string "performance"
]

$ dbus-send --system --print-reply --dest=org.freedesktop.Hal \
/org/freedesktop/Hal/Manager \
org.freedesktop.Hal.Manager.GetAllDevices

method return sender=:1.0 -> dest=:1.115 reply_serial=2
array [
string "/org/freedesktop/Hal/devices/volume_part_1_size_403367936"
string "/org/freedesktop/Hal/devices/net_46_b2_32_e5_29_04"
string "/org/freedesktop/Hal/devices/computer_logicaldev_input_0"
string "/org/freedesktop/Hal/devices/usb_device_45e_734_noserial_if0_logicaldev_input"
string "/org/freedesktop/Hal/devices/usb_device_45e_734_noserial_if1_logicaldev_input"
string "/org/freedesktop/Hal/devices/computer_logicaldev_input"
string "/org/freedesktop/Hal/devices/volume_uuid_9e1859de_0cf0_4048_96f5_4ef32ac904f7"
string "/org/freedesktop/Hal/devices/computer"
string "/org/freedesktop/Hal/devices/volume_part2_size_205632000"
string "/org/freedesktop/Hal/devices/volume_uuid_a332a6d6_a034_4f19_b470_bacf7ffa81e2"
string "/org/freedesktop/Hal/devices/volume_uuid_RyytQp_f73U_LzdN_6JjC_08yN_Aw3e_p3fNF1"
string "/org/freedesktop/Hal/devices/storage_model_DVD_Writer_1070d"
string "/org/freedesktop/Hal/devices/platform_serial8250_serial_platform_1"
string "/org/freedesktop/Hal/devices/storage_serial_SATA_ST3500320AS_9QM49NMQ"
string "/org/freedesktop/Hal/devices/storage_serial_SATA_ST3500320AS_9QM4GKDK"
.........
]

$ dbus-send --system --print-reply --dest=org.freedesktop.Hal \
/org/freedesktop/Hal/devices/storage_model_DVD_Writer_1070d
org.freedesktop.Hal.Device.GetProperty string:'info.vendor'

method return sender=:1.0 -> dest=:1.120 reply_serial=2
string "HP"
While D-Bus was not really designed with shell scripting in mind, it is possible to use various utilities to access system and session data via D-Bus.  I hope you have not found this post too long-winded and that you will go away and experiment with D-Bus on your own computer.  Unfortunately there is a lot of out-of-date information, terminology and examples on the Internet relating to D-Bus.  Always refer back to the specification when in doubt.  All the examples included on this port were tested on Fedora 10.

P.S.  I plan to cover scripting D-Bus signals in a future post.
 

Scripting HAL

Recent releases of Fedora and other GNU/Linux distributions include a Hardware Abstraction Layer (HAL) which is used to support device plug-and-play capabilities.  In this post I will show you how your shell scripts can use HAL to retrieve device and system information.

The term HAL is overloaded as it used to refer both to a specification and the actual software which implements the specification.  From an application developers viewpoint, HAL is way to enumerate the capabilities and features of hardware attached to a system and receive notification when something about the hardware changes.

First, a very quick overview of HAL.  Each item of physical hardware in a computer is regarded as being an device object which identified by a Unique Device Identifier (UDI).  Associated with each device object is a variable set of well-defined typed key-value pairs (or metadata) called device properties which describe what each device object represents together with its properties.  Some device properties are derived from the actual physical hardware, some are merged from XML-formatted files, known as Device Information Files, and some are derived from the actual device configuration.  Mandatory device properties are defined in the HAL specification.



A HAL daemon is used to maintain and update the list of device objects forming a hardware configuration and is notified when devices are added or removed.  HAL also provides callbacks so that the operating system and applications can react to hardware configuration changes in order to maintain system policy.  An example of such a callback and policy is when you plug in a USB memory stick HAL automatically creates a mount point and mounts the device.

For a good backgrounder on HAL, see this article Making Hardware Just Work by Havoc Pennington.  For more detailed information, you should read the HAL Specification and the HAL project page at freedesktop.org.

A number of command line utilities are provided for accessing and manipulating device objects.



















hal-disable-pollingdisable polling on drives with removable media
hal-find-by-capabilityfind device objects by capability matching
hal-find-by-propertyfind device objects by property matching
hal-get-propertyget a property from a device object
hal-is-caller-locked-out  determine if caller is locked out
hal-is-caller-privilegeddetermine if caller is privileged
hal-locklock an interface
hal-set-propertyset a property on a device object
lshallist devices objects and properties

For the purposes of this post, we are only interested in a small number of the above utilities, i.e. those utilities which locate and return information about device objects.

Here is a simple use of the hal-find-by-property utility to retrieve a list of network interfaces.
$ hal-find-by-property --key linux.subsystem --string net
/org/freedesktop/Hal/devices/net_3a_67_56_92_fb_73
/org/freedesktop/Hal/devices/net_computer_loopback
/org/freedesktop/Hal/devices/net_00_1c_c0_6d_8a_f7
The HAL specification specifies a system namespace which contains information about a system and it's currently running kernel.  The following script retrieves certain information from the system namespace.
udi=$(hal-find-by-property --key info.product --string Computer)
printf "Board Manufacturer: $(hal-get-property --udi $udi --key system.board.vendor)\n"
printf " Model: $(hal-get-property --udi $udi --key system.board.product)\n"
printf " Version: $(hal-get-property --udi $udi --key system.board.version)\n"
printf " Serial No.: $(hal-get-property --udi $udi --key system.board.serial)\n"
printf " Release Date: $(hal-get-property --udi $udi --key system.firmware.release_date)\n"
printf " Bios Version: $(hal-get-property --udi $udi --key system.firmware.version)\n"
printf " Motherboard UUID: $(hal-get-property --udi $udi --key system.hardware.uuid)\n"
Here is the output from the computer which I used to write this post.
Board Manufacturer: Intel Corporation
Model: DX48BT2
Version: AAE26191-205
Serial No.: BQBQ8280004G
Release Date: 08/07/2008
Bios Version: BTX3810J.86A.1814.2008.0807.2334
Motherboard UUID: 2237F666-4F75-11DD-894F-0007E9747DB3
The next example retrieves certain information about all the storage devices on a system using the volume capability.  The volume namespace is for device objects that represent storage devices with a filesystem that are mountable.  Such device objects have the capability volume.
for udi in $(/usr/bin/hal-find-by-capability --capability volume)
do
mount=$(hal-get-property --udi $udi --key volume.mount_point)
device=$(hal-get-property --udi $udi --key block.device)
label=$(hal-get-property --udi $udi --key volume.label)
fstype=$(hal-get-property --udi $udi --key volume.fstype)
echo "$device $mount $fstype $label"
done
Here is sample output from this script.
/dev/sda1  /boot      ext3  /boot
/dev/sdb2 /abac ext3 /abac
/dev/sdb1 /home/fpm ext3 fpm
/dev/sda2 LVM2_member
/dev/sdc1 LVM2_member
/dev/sdc2 LVM2_member
You can also drill down and return information about specific devices such as an optical drive, i.e. device objects which have a capability of cdrom.
for i in $(hal-find-by-property --key storage.drive_type --string cdrom)
do
printf "Manufacturer: $(hal-get-property --udi $i --key storage.vendor)\n"
printf " Model: $(hal-get-property --udi $i --key block.device)\n"
printf " Bus: $(hal-get-property --udi $i --key storage.model)\n"
printf " Path: $(hal-get-property --udi $i --key storage.bus)\n"
done
Here is the output for the computer which I am using to write this post.  It contains a single DVD RW drive.
Manufacturer: HP
Model: /dev/sr0
Bus: DVD Writer 1070d
Path: pci
In many cases there are multiple ways of retrieving the information you want.  For example, here is another way of retrieving the same information, together with details of the volume label and mount point if there is a disk in the drive, using the storage.cdrom capability.
for udi in $(/usr/bin/hal-find-by-capability --capability storage.cdrom)
do
device=$(hal-get-property --udi $udi --key block.device)
vendor=$(hal-get-property --udi $udi --key storage.vendor)
model=$(hal-get-property --udi $udi --key storage.model)
if [[ $(hal-get-property --udi $udi --key storage.removable.media_available) = true ]]
then
parent_udi=$(hal-find-by-property --key block.storage_device --string $udi)
mount=$(hal-get-property --udi $parent_udi --key volume.mount_point)
label=$(hal-get-property --udi $parent_udi --key volume.label)
fi
printf "$vendor $model $device $mount $label\n"
done
Here is the output from this script when there is a DVD in the drive followed by when there is not a DVD in the drive.
$ ./findodrive
HP DVD Writer 1070d /dev/sr0 /media/Kota_Kinabalu1 Kota_Kinabalu1
$ ./findodrive
HP DVD Writer 1070d /dev/sr0
Here is a script to list details about all removable USB storage drives that are currently installed on your system.
#!/bin/ksh93
#
# list attached USB storage devices
#

for udi in $(/usr/bin/hal-find-by-capability --capability storage)
do
device=$(hal-get-property --udi $udi --key block.device)
vendor=$(hal-get-property --udi $udi --key storage.vendor)
model=$(hal-get-property --udi $udi --key storage.model)
if [[ $(hal-get-property --udi $udi --key storage.bus) = "usb" ]]
then
parent_udi=$(hal-find-by-property --key block.storage_device --string $udi)
mount=$(hal-get-property --udi $parent_udi --key volume.mount_point)
label=$(hal-get-property --udi $parent_udi --key volume.label)
media_size=$(hal-get-property --udi $udi --key storage.removable.media_size)
size=$(( ceil(media_size/(1000*1000*1000)) ))
printf "$vendor $model $device $mount $label "${size}GB" \n"
fi
done
And here is the output when two USB thumb drives are present.
./listusb   
Kingston DataTraveler 2.0 /dev/sdd /media/KINGSTON KINGSTON 1GB
Kingston DataTraveler 2.0 /dev/sdc /media/USB-4GB USB-4GB 4GB
$
Well that is about all for now.  You should have gained sufficient information from this post to go away and experiment on your own system. 

Remember, however, that the HAL specification and it's implementation in GNU/Linux distributions is still somewhat in a state of flux.  The above examples worked on Fedora 10 as of the date of this post.  If an example does not work on your system, I recommend that you first check the output from the lshal utility to see if the device object and associated device properties are instantiated on your system.  You can use udevadm --monitor to see what kernel events are being pushed out to user space.  HAL monitors these events.  You can also use lshal --monitor to see what events HAL emits to user space.
 

JavaScript E4X

In previous posts, I discussed the SpiderMonkey command line shell js and how to add support to it to enable full access (read, write, create, copy, delete, etc.) to the local filesystem via the File object and the NSPR library.

While rumaging around in the source code and documentation for js, I found that js partially supported the EX4 XML extension via a user configurable option.

This post looks at what it takes to load an XML document into js from your local filesystem, process it and write out the resulting document to your local filesystem using File objects and the E4X extension.

The ECMAScript for XML (E4X) (ECMA-357) specification adds native support for XML objects and XMLList objects to the JavaScript programming language.  This standard was first published in 2004 and was based on XML extensions provided in the BEA (now Oracle) Weblogic Workshop product.  These extensions were designed by Terry Lucas and John Schneider who led the ECMAScript for XML (E4X) initiative.

The basis idea behind E4X was that declarative languages such as XSL and XPATH are too complex for the average programmer to quickly learn and therefore a simpler way of accessing and manipulating XML documents was needed.  Personally I do not agree with that assertion.

As an aside, currently Schneider is founder and CEO at AgileDelta which developed the Efficient XML binary format specification which I plan to write about in a future post.  A W3C working group is currently developing the EXI specification which is based on the AgileDelta specification.

TO BE CONTINUED

JavaScript File Object

As you are probably aware JavaScript engines such as SpiderMonkey typically do not allow access to the local filesystem for reasons of security.  To enable developers to test the scripts from a command line, js includes the load() function which enables you to load one or more JavaScript scripts into the SpiderMonkey engine.  However this is not sufficient for our purposes as no means is provided to write to the filesystem.  Looking more closely at the source code, I noticed support for File objects.  This support is not enabled by default however.  It is not sufficient to simply recompile SpiderMonkey with this option enabled; you must also download and build the Netscape Portable Runtime (NSPR) library.  This library provides a platform-neutral API for system level and libc-like functions, and is used by a number of Mozilla projects and other third party software developers.  The current release is 4.7.3 and you can download it here.

There are some gotchas to building Spidermonkey with NSPR.  First of all, you need to successfully build NSPR.  The source code tarball for NSPR comes with the standard GNU autoconfigure tools.  If you are on a 64-bit system, you need to execute configure with the -enable-64bit option; otherwise the build will quickly fail.  You should then test the build by going to the test subdirectory, building the testsuite and executing it.  You also need to modify SpiderMonkey's Makefile.ref (I am assuming you are building SpiderMonkey 1.7 and not an earlier release) to include libnspr and the NSPR headers.  Two compile time defines are needed.  You can either define JS_HAS_FILE_OBJECT and JS_THREADSAFE in Makefile.ref or as command line arguments to make.  After than you, should be able to successfully build SpiderMonkey with native File object support.

Now that we have js build with support for File objects, what can we do with it.  Well, I guess we should start with the expected Hello World script.
js> File.output.writeln("Hello World");     
Hello World
true
js> File.output.writeln("Hello, world"); "OK"
Hello, world
OK
js> File.output.writeln("Hello, world"); ""
Hello, world

js>
Notice that true is outputted unless you append something else as shown above.  Here is another short example which demonstrates how to list the properties of the instance File object for the current directory.
js> dir = new File('.');
/home/fpm/js/.
js> for ( i in dir ) print(i);
length
parent
path
name
isDirectory
isFile
exists
canRead
canWrite
canAppend
canReplace
isOpen
type
mode
creationTime
lastModified
size
hasRandomAccess
hasAutoFlush
position
isNative
js>print(dir.path);
/home/fpm/js/.
js>print(dir.size)
4096
js>
The next example shows how to list some information about files in the current directory.
js> dir = new File('.');
/home/fpm/js/.
js> files = dir.list(); 'OK'
OK
js> for (i in files ) print (files[i].name + ' ' + files[i].size + ' ' + files[i].creationTime);
music.xml 1081 Tue Jan 06 2009 17:37:14 GMT-0500 (EST)
xml.js 259 Tue Dec 30 2008 18:23:22 GMT-0500 (EST)
xml1.js 699 Tue Jan 06 2009 23:33:26 GMT-0500 (EST)
2.xml 96 Tue Jan 06 2009 22:41:37 GMT-0500 (EST)
3.xml 127 Wed Jan 07 2009 00:02:18 GMT-0500 (EST)
multiply.js 249 Tue Dec 30 2008 17:49:02 GMT-0500 (EST)
helloworld.js 88 Tue Dec 30 2008 17:12:50 GMT-0500 (EST)
hw.js 124 Thu Jan 01 2009 00:24:38 GMT-0500 (EST)
xml2.js 502 Wed Jan 07 2009 00:02:17 GMT-0500 (EST)
regex.js 143 Tue Dec 30 2008 18:10:55 GMT-0500 (EST)
1.xml 15 Tue Jan 06 2009 20:35:27 GMT-0500 (EST)
js>
In the above example, list is an instance method of the File object.  Using other File object instance methods, you can read data from a file and write data to a file.

In the following example, the script reads lines in from file.in and write the lines out to another file file.out, prepending each line with the corresponding line number.
#!/bin/js

var filein = new File("file.in");
filein.open("read", "text");

var fileout = new File('file.out');
fileout.open("write,create", "text");

var n=1;
while (data = filein.readln())
fileout.writeln(n++ + ' ' + data);

filein.close();
fileout.close();
The File object provides two ways to access data inside a file: a text oriented access, based on characters, and a binary oriented access, based on bytes. In text mode, the maximum line length is 256 and the following encodings are supported: ASCII (text), UTF-8 and UCS-2.  The File object has a number of instance methods including read, readLn, readAll, write, writeLn and writeAll.  If you just want to copy the file in it's entirety, you can use the copyTo method.
var file = new File("file.in");
file.open("read", "text");

file.copyTo("file.out");

file.close();
Similar instance methods which work on files include remove to delete a file or a directory and removeTo to rename a file.

Now that I have shown you how to enable js to access the local filesystem, I would be amiss if I did not point out to you the following warning which is in the Javascript File Object Proposal.

Flaming disclaimer:   Do not play with the File object if
you are not prepared to have your hard drive erased,
smashed, and broken into little bits! It mostly works
right now, but no guarantees.
So far I have had no problem using the File object on my Fedora 10 64-bit platform but one never knows!  Proceed with caution.

JavaScript Shells

Recently I was working on a fairly complex JavaScript script relating to floating point conversions for a new Web page.  After a while I got tired on trying to debug the problem via a Web browser and decided to see if I could find a JavaScript shell, i.e. a standalone Javascript intrepreter just like Ruby's irb, Python's interactive prompt or the Korn shell, which could load and run JavaScript scripts from the command line without having to reload a Web page.

First, some background on the JavaScript langauge for those who are unfamilar with the details.  JavaScript is a complex full-featured weakly typed object- based functional programming language originally developed by Brendan Eich in 1995 while working on the Netscape Navigator browser.  It is most frequently used in client-side web applications but is also used to enable scripting access to embedded objects in other applications.

The langauge has been standardized in the ECMA-262 (ECMAScript) specification.  The first version of ECMAScript was published in June 1997, and was partially based on JavaScript v1.2.  The current version is Edition 3 (Dec 1999) and work is ongoing on the next edition.  Formally, Javascript is a dialect of ECMAScript whose langauge specification is controlled by the Mozilla Foundation.  There are other dialects including ActionScript which the scripting language used in Adobe Flash.  Javascript is still evolving as a language and several versions are in daily use.  The current version is JavaScript 1.8.

The JavaScript engine in Firefox is written in C.  It was orginally called Javascript Reference (JSRef) but nowadays is known as SpiderMonkey.  Other Mozilla products also use this engine and it is available to the public under a MPL/GPL/LGPL tri-license.  The current version, SpiderMonkey 1.7, conforms to JavaScript 1.8 which is a superset of ECMA-262 Edition 3. It consists of a library (or DLL) containing the JavaScript runtime (compiler, interpreter, decompiler, garbage collector, atom manager and standard classes) engine.  This codebase has no dependencies on the rest of the Mozilla codebase. The codebase also contains the routines for a simple user interface which can be linked to the runtime library in order to make a command line shell.

You can download the source code for SpiderMonkey 1.7 here.  Aternatively you can use wget,curl or ftp to download the tarball.  No build script is provided with this version of SpiderMonkey.  Here is how I downloaded, built and smoketest'ed the shell.
mkdir mozilla
cd mozilla
wget http://ftp.mozilla.org/pub/mozilla.org/js/js-1.7.0.tar.gz
tar xzf js-1.7.0.tar.gz
cd js/src
make -f Makefile.ref
If everything compiles correctly, you should then smoketest the JavaScript shell (js) by executing the following command:
./Linux_All_DBG.OBJ/js ./perfect.js 
If the 3 perfect numbers between 1 and 500 are printed and you are returned to your shell prompt without any error messages, all is well.  You should then copy the ./Linux_All_DBG.OBJ/js to /bin or /usr/local/bin to make it easier to use.

Unlike other programming languages, JavaScript does not have a concept of printing to STDOUT or reading from STDIN.  These functions, along with quit(), load() and a small number of other functions are provided within js.  They are not part of the Javascript runtime library.  The first thing you will notice is that everything is function-based.  To exit js, you do not type quit, instead you have to type quit().

Here is the standard Hello World example.
$ cat helloworld.js
//
// Hello World!
//

function helloWorld(name)
{
print("Hello World, " + name);
}
$ /bin/js
js> load('helloworld.js')
js> helloWorld('Finnbarr')
Hello World, Finnbarr
js> helloWorld('Patricia')
Hello World, Patricia
js> quit()
$
Here is the same script called directly from ksh93.  This is possible because of the shebang (!#) sytax on the first line of the script.
$ cat hw.js
#!/bin/js

//
// Hello World!
//

function helloWorld(name)
{
print("Hello World, " + name);
}

helloWorld('Finnbarr');
$ ./hw.js
Hello World, Finnbarr
$
The shell comes with a readline() function which enables you to ask a user to enter a value such a string or a number as the following example shows.
$ cat multiply.js
//
// multiply.js
//

function multiply()
{
print("Enter a number:");
var n1 = readline();
print("Enter another one:");
var n2 = readline();

print("You entered " + n1 + " and " + n2 + ". The result is " + n1*n2);
}

$ /bin/js
js> load('multiply.js')
js> multiply()
Enter a number:
12
Enter another one:
10
You entered 12 and 10. The result is 120
You have full access to all functionality that is defined in the Javascript 1.8 specification.
$ cat regex.js
//
// test regular expression
//

myRe = /d(b+)d/g;
myArray = myRe.exec("cdbbdbsbz");

print("The value of lastIndex is " + myRe.lastIndex);

$ /bin/js -f regex.js
The value of lastIndex is 5
$
You can also disassemble your JavaScript script using the dissrc() function.
js> dissrc(multiply)     

;------------------------- 7: print("Enter a number:");
00000: 7 name "print"
00003: 7 pushobj
00004: 7 string "Enter a number:"
00007: 7 call 1
00010: 7 pop
;------------------------- 8: var n1 = readline();
00011: 8 name "readline"
00014: 8 pushobj
00015: 8 call 0
00018: 8 setvar 0
00021: 8 pop
;------------------------- 9: print("Enter another one:");
00022: 9 name "print"
00025: 9 pushobj
00026: 9 string "Enter another one:"
00029: 9 call 1
00032: 9 pop
;------------------------- 10: var n2 = readline();
00033: 10 name "readline"
00036: 10 pushobj
00037: 10 call 0
00040: 10 setvar 1
00043: 10 pop
;------------------------- 11:
;------------------------- 12: print("You entered " + n1 + " and " + n2 + ". The result is " + n1*n2);
00044: 12 name "print"
00047: 12 pushobj
00048: 12 string "You entered "
00051: 12 getvar 0
00054: 12 add
00055: 12 string " and "
00058: 12 add
00059: 12 getvar 1
00062: 12 add
00063: 12 string ". The result is "
00066: 12 add
00067: 12 getvar 0
00070: 12 getvar 1
00073: 12 mul:0
00074: 12 add
00075: 12 call 1
00078: 12 pop
00079: 12 stop
js> quit()
$
A major limitation of js is that it cannot access files such as an XML documents, DOM objects nor upload or download files.  An alternative command line shell is available from Mozilla called xpcshell.  It is a XPConnect-enabled JavaScript command line shell where scripts running in it can access XPCOM functionality.  For those of you who are unfamilar with XPCOM, it is a cross platform component object model, similar to Microsoft's COM.  XPCOM is frequently used for unit testing.

Besides SpiderMonkey and TraceMonkey, there are a number of other standalone JavaScript shells available including wxjs and JSDB
.  There also is the Rhino engine, created primarily by Norris Boyd, which is a JavaScript implementation written in Java that is ECMA-262 Edition 3 compliant.

I would be amiss if I did not point out that there are alternatives to using a command line shell such as js.  My personal favorite is Firebug which is a Firefox extension that provides a more advanced interactive shell, an advanced DOM inspector, a JavaScript debugger, a profiling tool and various other useful tools.

Plans for SpiderMonkey 1.8 appear to be shelved.  The Firefox development team are currently in the process of replacing SpiderMonkey with TraceMonkey which is based on a technique developed at UC Irvine called trace trees, builds on code and ideas shared with the Tamarin Tracing project, and adds just-in-time (JIT) native code compilation.  The net result is a seriously significant speed increase both in the browser chrome and Web page content.  As with SpiderMonkey, you can download the source code from the TraceMonkey mercurial repository and build your own command line shell.

After languishing for a number of years, JavaScript is becoming an increasing important language for Web applications. It is one of those languages that every Web programmer needs to understand in depth.  Being able to run and debug JavaScript scripts from the command line greatly assists in that understanding.

Enjoy!