KSH93 Custom Builtins II

This blog entry continues an earlier discussion of ksh93 custom builtins and includes a number of more advanced examples.

First of all, the development of custom builtins is easier when the libast header <shell.h> is included. It ensures that other necessary AST headers are included so that C prototypes are provided and calls to stdio routines are re-mapped to use the equivalent but safer sfio (AST Safe Fast Input Output) routines. The maintainers of ksh93 (David Korn and Glen Fowler) regard the standard stdio routines as having too many weaknesses to be used safely by ksh93.

Another useful feature is the lib_init() function. If your custom shared library contains lib_init(), this function is invoked with an argument value of 0 when the shared library is loaded. In addition, lib_init() can be used to load a custome builtin using sh_addbuiltin(). In this case there is no restriction on the name of a custom builtin function name, i.e. it does not have to start with "b_". Note that in the current version of ksh93, lib_init() can take a second argument, void *context, but this seems to do nothing.

Finally we will discuss memory management within custom builtins, how to include online documentation within a custom builtin and how to access internal shell data about variables and more.

Example 1   Use lib_init() to load a custom builtin.
#include <ast/shell.h>

int
fpm_goodbye(int argc, char *argv[], void *extra)
{
   if (argc != 2) {
      printf("Usage: goodbye arg\n");
      return(2);
   }

   printf("Goodbye %s\n", argv[1]);

   return(0);
}

int
b_hello(int argc, char *argv[], void *extra
{
   if (argc != 2) {
      printf("Usage: hello arg\n");
      return(2);
   }

   printf("Hello %s\n", argv[1]);

   return(0);
}

void
lib_init(int c, void *context)
{
   /* automatically load goodbye when library is loaded */
   sh_addbuiltin("goodbye", fpm_goodbye, 0);
}
Assuming you built a shared library called libhello.so which contains the above code, and placed this library in the shared library location, you can access the two custom builtins, hello and goodbye as follows:
$ builtin hello hello
$ hello Joe
Hello Joe
$ goodbye Joe
Goodbye Joe
$
To remove both custom builtins at the same time you must specify both on the command line as follows
$ builtin –d hello –d goodbye
$
Note that goodbye was not named internally as b_goodbye in this example but as fpm_goodbye since the “b_” function prefix naming requirement is relaxed when a custom builtin is loaded via lib_init() and sh_addbuiltin().

Example 2   Another way to load custom builtins.

If for some reason you do not wish to use the AST sfio routines, you should include <shcmd.h> and then call LIB_INIT(context) to initialize the shell context so that ksh93 knows that you are not using the sfio routines.
#include <ast/shcmd.h>
#include <stdio.h>

int
b_goodbye(int argc, char *argv[], void *extra)
{
   if (argc != 2) {
      printf("Usage: goodbye arg\n");
      return(2);
   }

   printf("Goodbye %s\n", argv[1]);

   return(0);
}

int
b_hello(int argc, char *argv[], void *extra)
{
   if (argc != 2) {
      printf("Usage: hello arg\n");
      return(2);
   }

   printf("Hello %s\n", argv[1]);

   return(0);
}

void
lib_init(int c, void *context)
{
   LIB_INIT(context);

   sh_addbuiltin("hello", b_hello, 0);
   sh_addbuiltin("goodbye", b_goodbye, 0);
}
Assuming that you build a shared library called libhello.so, both custom builtins are automatically loaded when the shared library is loaded, i.e.
$ builtin –f libhello.so

Example 3    Self-documentating custom builtins

One of the advantages of ksh93 over other shells is the fact that online help is provided for most commands using -?, --man or --help. Self-documenting code is supported in custom builtins but is very poorly documented. All custom builtins can generate their own manual page in several formats. While this documentation takes up space in the shared library and loaded image, the benefits probably outweight the cost.
#include <ast/shell.h>

#define SH_DICT "libgoodbye"

static const char usage_goodbye[] =
   "[-?\n@(#)$Id: goodbye 2008-04-06 $\n]"
   "[-author?Finnbarr P. Murphy <fpmATun-ixDOTcom>]"
   "[-licence?http://www.opensource.org/licenses/cpl1.0.txt]"
   "[+NAME?goodbye - output goodbye message]"
   "[+DESCRIPTION?\bgoodbye\b outputs either a short or extended"
   "message to stdout.]"
   "[x:xtended?Output extended message]"
   "\n"
   "\nname\n"
   "\n"
   "[+EXIT STATUS?] {"
      "[+0?Success.]"
      "[+>0?An error occurred.]"
   "}"
   "[+SEE ALSO?\bprint\b(1), \becho\b(1)]"
;

int
b_goodbye (int argc, char *argv[], void *extra)
{
   register int n, extend=0;

   while (n = optget(argv, usage_goodbye)) switch(n) {
      case 'x':
         extend=1;
         break;
      case ':':
         error(2, "%s", opt_info.arg);
         break;
      case '?':
         errormsg(SH_DICT,
         ERROR_usage(2), "%s", opt_info.arg);
         break;
      }

      argc -= opt_info.index;
      argv += opt_info.index;

      if (argc != 1)
         errormsg(SH_DICT,
         ERROR_usage(2), "%s", optusage((char *)0));

      if (extend)
         sfprintf(sfstdout, "Goodbye for now %s\n", *argv);
      else
         sfprintf(sfstdout, "Goodbye %s\n", *argv);

      return(0);
}

void
lib_init(int c, void *context)
{
   sh_addbuiltin("goodbye", b_goodbye, 0);
}

Example 4   Memory management within custom builtins

This example shows how to encrypt a character string using a numeric key. It uses a relatively simple XOR operation to encrypt the string and is commonly known as the Vernam cipher or XOR encryption.

To prevent memory leaks in your custom builtins you should avoid using malloc() and calloc() and similar routines. The preferred way to obtain, free and manage memory space when required in a custom builtin is to use the libast stk(3) routines. Memory leaks can arise when a custom builtin does not free all of its allocated memory upon return or is interrupted by a signal such as SIGINT before it can do so.

This custom builtin uses the stkcopy() routine to get the required space. This memory is guaranteed to be freed by libast when the custom builtin exits, i.e. no additional code such as free() is required to free up the allocated memory.
#include <shell.h>
#include <stk.h>

#define SH_DICT "strcrypt"

static const char usage_strcrypt[] =
   "[-?\n@(#)$Id: strcrypt 2008-05-04 $\n]"
   "[-author?Finnbarr P. Murphy <fpmAThotmailDOTcom>]"
   "[-licence?http://www.opensource.org/licenses/cpl1.0.txt]"
   "[+NAME?strcrypt - encrypt string using numeric key]"
   "[+DESCRIPTION?\bencrypt\b a string using a numeric key.
        Note uses XOR to do encryption. Only works when
        numeric key used. Use same numeric key to decrypt.]"
   "[+OPTIONS?none.]"
   "\n"
   "\nstring key\n"
   "\n"
   "[+EXIT STATUS?] {"
        "[+0?Success.]"
        "[+>0?An error occurred.]"
   "}"
   "[+SEE ALSO?\bcrypt\b(2)]"
;

int
b_strcrypt(int argc, char *argv[], void *extra)
{
   int i, sl, pl, c;
   char *v, *s, *p;

   while (i = optget(argv, usage_strcrypt)) switch(i) {
      case ':':
         error(2, "%s",opt_info.arg);
         break;
      case '?':
         errormsg(SH_DICT, ERROR_usage(2),
            "%s", opt_info.arg);
         break;
    }
   argc -= opt_info.index;
   argv += opt_info.index;

   if (argc != 2)
      errormsg(SH_DICT, ERROR_usage(2), "%s",
         optusage((char *)0));

   s = argv[0];
   sl = strlen(s);
   p = argv[1];
   pl = strlen(p);

   /* copy string onto stack so it can be modified */
   if (!(v = (char *)stkcopy(stkstd, s))
      error(3, "stkcopy failed");

   for (i = 0; i < sl; i++) {
      c = s[i] ^ toupper(p[i%pl]);
      if (c != 0)
         v[i] = (char)c;
   }

   sfprintf(sfstdout,"%s", v);

   return(0);
}
With this custom builtin, a string of text (argument 1) is encrypted using the supplied numeric key (argument 2). To decrypt the encrypted string, simply invoke the custom builtin again using the encrypted string and same numeric key.
$ strcrypt “hello” 1478
YQ[T^
$ strcrypt “YQ[T^“ 1478
hello
$
Note that libast also contains a set of routines whose names start with stak and which provide similar functionality. However the stak(3) routines are marked deprecated and should not be used in custom builtins.

Example 5   Using name-value routines to access ksh93 internals

The purpose of this custom builtin is to set the value of a specified variable (argument 1) to the size of the specified file (argument 2).
$ statsize myfilesize /usr/bin/ksh
$ print ${myfilesize}
1157295
$
This example uses the libast nval(3) name-value routines to access the internals of ksh93 and set up a variable with the specifed name (i.e. myfilesize) and value of 1157295 as returned by stat(2) for the file /usr/bin/ksh. See the nval(3) documentation for more detailed information.
#include <shell.h>
#include <nval.h>

#define SH_DICT "statsize"

static const char usage_statsize[] =
   [-?\n@(#)$Id: stat 2008-05-03 $\n]"
   [-author?Finnbarr P. Murphy <fpmAThotmailDOTcom>]"
   [-licence?http://www.opensource.org/licenses/cpl1.0.txt]"
   [+NAME?statsize - assign size of file to variable]"
   [+DESCRIPTION?\bstat\b assigns the size of the specified file"
      "(in bytes) to the specified variable.]"
   "[+OPTIONS?none.]"
   "\n"
   "[+EXIT STATUS?] {"
      "[+0?Success.]"
      "[+>0?An error occurred.]"
   "[+SEE ALSO?\bstat\b(2)]"
;

int
b_statsize(int argc, char *argv[], void *extra)
{
   Namval_t *nvp = (Namval_t *)NULL;
   Shell_t *shp = (Shell_t *)NULL;
   struct stat st;
   long d;
   register int n;

   while (n = optget(argv, usage_statsize)) switch(n) {
      case ':':
        error(2, "%s", opt_info.arg);
        break;
      case '?':
        errormsg(SH_DICT, ERROR_usage(2), "%s",
           opt_info.arg);
        break;
   }
   argc -= opt_info.index;
   argv += opt_info.index;

   if (argc != 2)
      errormsg(SH_DICT, ERROR_usage(2), "%s",
         optusage((char*)0));

   /* get current shell context */
   shp = sh_getinterp();

   /* retrieve information about file */
   stat(argv[1], &st);
   /* assign size of file to long */
   d = (long) st.st_size;

   /* access the variables tree and add specified variable */
   nvp = nv_open(argv[0], shp->var_tree,
          NV_NOARRAY|NV_VARNAME|NV_NOASSIGN);
   if (!nv_isnull(nvp))
      nv_unset(nvp);
   nv_putval(nvp, (char *)&d, NV_INTEGER|NV_RDONLY);
   nv_close(nvp);

   return(0);
}
This custom builtin can easily be extended to provide much more information about a file using command line options. I will leave it as an exercise for you the reader.

Example 6   Print out detailed information about a specified shell variable.
$ showvar HOME  
Value: /home/fpm, Flags: 12288 NV_EXPORT NV_IMPORT
$ integer i=123
$ showvar i
Value: 123, Flags: 10 NV_UINT64 NV_UTOL
See the libast header <nval.h> for detailed information about the different flags which can be associated with each variable. Note that some flags are overloaded so that they mean different things according to how they are OR’ed with other flags.
#include <shell.h>
#include <nval.h>

#define SH_DICT "showvar"

static const char usage_showvar[] =
"[-?\n@(#)$Id: showvar 2008-05-04 $\n]"
"[-author?Finnbarr P. Murphy <fpmAThotmailDOTcom>]"
"[-licence?http://www.opensource.org/licenses/cpl1.0.txt]"
"[+NAME?showvar - display variable details]"
"[+DESCRIPTION?\bshowvar\b displays details about the
specified variable.]"
"[+OPTIONS?none.]"
"\n"
"\nvariable_name\n"
"\n"
"[+EXIT STATUS?] {"
"[+0?Success.]"
"[+>0?An error occurred.]"
"}"
"[+SEE ALSO?\bstat\b(2)]"
;

struct Flag {
int flag;
char *name;
};

/* Note: not a complete list of all possible flags */
struct Flag Flags[] = {
NV_ARRAY, "NV_ARRAY",
NV_BINARY, "NV_BINARY",
NV_EXPORT, "NV_EXPORT",
NV_HOST, "NV_HOST",
NV_IMPORT, "NV_IMPORT",
NV_LJUST, "NV_LJUST",
NV_LTOU, "NV_LTOU",
NV_RAW, "NV_RAW",
NV_RDONLY, "NV_RDONLY",
NV_REF, "NV_REF",
NV_RJUST, "NV_RJUST",
NV_TABLE, "NV_TABLE",
NV_TAGGED, "NV_TAGGED",
NV_UTOL, "NV_UTOL",
NV_ZFILL, "NV_ZFILL",
0,(char *)NULL
};

struct Flag IntFlags[] = {
NV_LTOU|NV_UTOL|NV_INTEGER, "NV_UINT64",
NV_LTOU|NV_RJUST|NV_INTEGER, "NV_UINT16",
NV_RJUST|NV_ZFILL|NV_INTEGER, "NV_FLOAT",
NV_UTOL|NV_ZFILL|NV_INTEGER, "NV_LDOUBLE",
NV_RJUST|NV_INTEGER, "NV_INT16",
NV_LTOU|NV_INTEGER, "NV_UINT32",
NV_UTOL|NV_INTEGER, "NV_INT64",
NV_RJUST, "NV_SHORT",
NV_UTOL, "NV_LONG",
NV_LTOU, "NV_UNSIGN",
NV_ZFILL, "NV_DOUBLE",
NV_LJUST, "NV_EXPNOTE",
NV_INTEGER, "NV_INT32(NV_INTEGER)",
0,(char *)NULL
};

int
b_showvar(int argc, char *argv[], void *extra)
{
Shell_t *shp = (Shell_t *)NULL;
Namval_t *nvp = (Namval_t *)NULL;
char *ptr = (char *)NULL;
int i;

while (i = optget(argv, usage_showvar)) switch(i) {
case ':':
error(2, "%s", opt_info.arg);
break;
case '?':
errormsg(SH_DICT, ERROR_usage(2), "%s", opt_info.arg);
break;
}
argc -= opt_info.index;
argv += opt_info.index;

if (argc != 1)
errormsg(SH_DICT, ERROR_usage(2), "%s", optusage((char*)0));

/* get current shell context */
shp = sh_getinterp();

if ((nvp = nv_search(*argv, shp->var_tree, 0)) == NULL) {
errormsg(SH_DICT, ERROR_exit(1),
"%s: variable not found", *argv);
return(1);
}

if ((ptr = nv_getval(nvp)) == NULL) {
errormsg(SH_DICT, ERROR_exit(3),
"%s: variable is NULL", *argv);
return(1);
}

sfprintf(sfstdout,
"Value: %s, Flags: %d", ptr, (int)nvp->nvflag);
if ((int)nvp->nvflag & NV_INTEGER) {
for (i=0; IntFlags[i].name != NULL; i++) {
if ((int)nvp->nvflag & IntFlags[i].flag) {
sfprintf(sfstdout, " %s", IntFlags[i].name);
break;
}
}
}
for (i=0; Flags[i].name != NULL; i++) {
if ((int)nvp->nvflag & Flags[i].flag)
sfprintf(sfstdout, " %s", Flags[i].name);
}

sfprintf(sfstdout,"\n");
nv_close(nvp);

return(0);
}
Well, that is about all that you really need to know about ksh93 custom builtins in order to start developing your own custom builtins. For your information, these examples were tested on versions ksh93s and ksh93t. Good luck and feel free to ask questions. I will try and answer any relevant questions.

0 comments:

Post a Comment