KSH93 Bit Manipulation

When programmers think about bitwise manipulation, they usually think about using C or C++ to solve their problem since both programming languages provide a rich set of features which make it easy to perform bitwise manipulation.

However it is possible to just as easily perform such operations using the ksh93 shell.  This post will explain what bitwise manipulation and number conversion facilities are available in ksh93 and provide some hopefully useful utilities and examples.

Many programmers are unaware that ksh93 has builtin support for different numeral radices (AKA arithmetic bases) as shown in the following example.
`\$ print \$(( 10 + 20 ))30\$ print \$(( 2#10 + 20 ))22\$ print \$(( 2#10 + 4#20 ))10\$`
where base 2 is denoted by 2#, base 4 by #4 and so on.  The general case syntax is [base#]n where base is a decimal number between 2 and 36 representing the arithmetic base and n is a number in that base.  You can mix and match bases within expressions.  As you would expect, the default base, i.e. if [base#] is omitted, is decimal(10).

Here is an example of how to convert decimal 255 to base 2, base 8 and base 16 respectively using the typeset command.
`\$ typeset -i2 a=255\$ echo \$a2#11111111\$ typeset -i8 a=255\$ echo \$a8#377\$ typeset -i16 a=255\$ echo \$a16#ff\$`
Note that ksh93 displays the base prefix before each number unless it is a decimal number, i.e. base 10.

The bitwise manipulation operators in ksh93 are AND, OR, XOR, SHIFT LEFT and SHIFT RIGHT.  There are no SHIFT ROTATE or COMPLEMENT operators.  The following is the list of supported bitwise operators in decreasing order of precedence which, by the way, is the same as in the C programming language.

 << >> Bitwise shift left, shift right & Bitwise AND ^ Bitwise XOR | Bitwise OR

The following shell script includes both logical and bitwise operators together with the expected output for each statement.  Notice the difference in output between a logical and bitwise operator.
`#!/bin/ksh93integer -i2 a=5integer -i2 b=6# expected output 1print \$(( a < b ))# expected output 0print \$(( a > b ))# expected output 3print \$(( a ^ b ))# expected output 4print \$(( a & b ))# expected output 1print \$(( a && b ))# expected output 7print \$(( a | b ))# expected output 1print \$(( a || b ))`
A useful feature of bitwise operators is that by using the shift left and shift right bitwise operators you can mimic multiplication or division by 2 or a power of 2.
`\$ x=4\$ print \$((x << 1))8\$ print \$(( x << 2 ))16\$ print \$(( x >> 1 ))2\$ print \$(( x >> 2 ))1`
Here is a fairly simple shell script which uses numeric base conversion and bitwise operators to calculate the network and broadcast addresses for a given IP address and subnet mask.
`#!/bin/ksh93typeset -i2 mask=255[[ \$# != 2 ]] && {   echo "Usage: \$0 ipaddress subnetmask"   exit 1}SaveIFS=\$IFSIFS=.typeset -a IParr=(\$1)typeset -a NMarr=(\$2)IFS=\$SaveIFStypeset -i2 ip1=\${IParr}typeset -i2 ip2=\${IParr}typeset -i2 ip3=\${IParr}typeset -i2 ip4=\${IParr}typeset -i2 nm1=\${NMarr}typeset -i2 nm2=\${NMarr}typeset -i2 nm3=\${NMarr}typeset -i2 nm4=\${NMarr}echoecho "       IP Address: \$1"echo "      Subnet Mask: \$2"echo "  Network Address: \$((ip1 & nm1)).\$((ip2 & nm2)).\$((ip3 & nm3)).\$((ip4 & nm4))"echo "Broadcast Address: \$((ip1 | (mask ^ nm1))).\$((ip2 | (mask ^ nm2))).\$((ip3 | (mask ^ nm3))).\$(( ip4 | (mask ^ nm4)))"echoexit 0`
Some sample output for this script.
`\$ ./calculate-address 10.150.12.1 255.255.255.0       IP Address: 10.150.12.1      Subnet Mask: 255.255.255.0  Network Address: 10.150.12.0Broadcast Address: 10.150.12.255\$ ./calculate-address 10.150.12.1 255.255.254.0       IP Address: 10.150.12.1      Subnet Mask: 255.255.254.0  Network Address: 10.150.12.0Broadcast Address: 10.150.13.255`
The next example calculates the network and broadcast addresses for a Classless Inter-Domain Routing (CIDR) compliant IP address.  Note the use of the shift right operator in calculating the network mask.
`#!/bin/ksh93[[ \$# != 1 ]] && {   echo "Usage: \$0 ipaddress/netmask"   exit 1}SaveIFS=\$IFSIFS="./"typeset -a IParr=(\$1)IFS=\$SaveIFStypeset -i2 ip1=\${IParr}typeset -i2 ip2=\${IParr}typeset -i2 ip3=\${IParr}typeset -i2 ip4=\${IParr}typeset -i2 cidr=\${IParr}typeset -i2 nm1=0 nm2=0 nm3=0 nm4=0typeset -i quad=\$(( cidr / 8 ))sigbits=\$(( cidr % 8 ))if (( sigbits != 0 )); then   slot=\$(( 256 - ( 256 >> \$sigbits ) ))fifor (( i=1; i < 5; i++ ))do   nameref nm=nm\${i}   if (( quad != 0 )); then      nm=255      (( --quad ))   elif (( quad == 0 )); then      nm=slot      break   fidonetypeset -i2 mask=255printprint "           IP Address: \$((ip1)).\$((ip2)).\$((ip3)).\$((ip4))"print "    CIDR Netmask Mask: \$((nm1)).\$((nm2)).\$((nm3)).\$((nm4))"print " CIDR Network (Route): \$((ip1 & nm1)).\$((ip2 & nm2)).\$((ip3 & nm3)).\$((ip4 & nm4))"print "    Broadcast Address: \$((ip1 | (mask ^ nm1))).\$((ip2 | (mask ^ nm2))).\$((ip3 | (mask ^ nm3))).\$((ip4 | (mask ^ nm4)))"printexit 0`
Sample output:
`\$ ./calculate-address "192.168.21.12/18"           IP Address: 192.168.21.12    CIDR Network Mask: 255.255.192.0 CIDR Network (Route): 192.168.0.0    Broadcast Address: 192.168.63.255`
While both the bash and zsh shells also provide a full set of bitwise operators, only ksh93 and zsh with their builtin support for numeric base conversions makes such bitwise operators truly useful in shell scripts.