Porting WaitForSingleObject to Linux - Part 2

In my last post I discussed the use of WaitForSingleObject in relation to mutexes and possible ways to implement equivalent functionality when porting such code to GNU/Linux.  In this post I will describe the use of this API with event objects in Microsoft Windows and suggest possible ways of posting such code to GNU/Linux or Unix.

First, some background on event objects.  An event object is just another type of Windows kernel dispatcher object.  From a coding prespective, an event object is a synchronization object which encapsulates one or more kernel dispatcher objects and whose synchronization semantics are accessable via WaitForSingleObject and its cousins.  At any given time a synchronization object is either nonsignaled or signaled, i.e. the object can only be in one of two possible states.

All of the WaitFor family of APIs including WaitForSingleobject wait on an object handle or handles until some specified criteria is met.  The two basic criteria for all these APIs are the signaled state of the object on whose handle it is waiting and a time-out value.  Thus a thread which calls this API waits till the specified object enters the signaled state or the specified time-out has expired.  Little or no CPU time is used when such a thread is in the wait state.

In the case of events, a CreateEvent or OpenEvent returns a handle to an event object.  When an event is in the signaled state it means that that the event has the capacity to release one or more threads waiting for this particular event to be signaled.  When an event is in the nonsignaled state it will not release any waiting thread.  Initially the state of an event is nonsignaled.  An event object's state is set explicitly to signaled by SetEvent or PulseEvent.  Event objects are also used in overlapped operations such as reading from a socket, in which case the event object state is set to signaled by the kernel rather than by an application.

Events also come in two reset types.  If an event is a manual-reset event, then all WaitForSingleObjects return that wait for that event if so configured.  In other words a manual-reset event can trigger action by one or more WaitForSingleObject or its cousins.  A manual-reset event object's state must be reset explicitly to nonsignaled by ResetEvent.

For an auto-reset event object, WaitForSingleObject and it's relations reset the state of the event object to nonsignaled before returning.  While an auto-reset event is guaranteed to set the event to nonsignaled and release a single thread that is waiting on the event to occu, if more than one thread is waiting for this particular event to occur then which particular thread is released is random.

Consider the following skeleton Microsoft Windows application.

HANDLE hEvent;
HANDLE hMutex;
HANLE  hThread;
....
BOOLEAN  Run = TRUE;
....

main()
{

    hMutex = CreateMutex(0, FALSE, NULL);
    hEvent = CreateEvent( NULL, FALSE, FALSE, NULL);
    .....
    hThread = CreateThread( NULL, 0 (LPTHREAD_START_ROUTINE)SendThread, 0, 0, 0);
    .....

    /* produce and post messages */
    .....

    /* clean up and exit */
    CloseHandle(hEvent);
    CloseHandle(hMutex);
}


DWORD SendThread (LPVOID Parm)
{
    do {
          /* wait for the event */
          WaitForSingleObject( hEvent, INFINITE );

          /* send the message */
          ....
          ....
          /* lock message queue and delete message */
          WaitForSingleObject( hMutex, INFINITE );
          .....
          .....
          ReleaseMutex( hMutex );

    } while ( Run };

}

PostMessage( PMESSAGE Message )
{
    /* lock message queue and add message */
    WaitForSingleObject( hMutex, INFINITE );
    .....
    .....
    ReleaseMutex( hMutex );

    /* signal SendThread that there is a message in the queue */
    SetEvent( hEvent );
}

Assume that the purpose of this application, for whatever reason, is to broadcast UDP messages.  A separate thread of execution, i.e SendThread, is created to handle the actual broadcasting.  It simple takes whatever messages are in the send message queue amd broadcasts them.  If there are no messages in the message queue, the thread waits (yields) until it is signalled that there is one or more messages waiting to be broadcast.

Here is the same application code after it was ported to GNU/Linux using POSIX theads.

pthread_cond_t   hEvent;
pthread_mutex_t  hMutex;
pthread_t        hThread;
....
BOOLEAN  Run = TRUE;
....

main()
{
    pthread_mutexattr_t  mattr;
    pthread_attr_t       attr;

    /* initialize condition variable */
    pthread_cond_init( &hEvent, NULL);

    /* initialize mutex as a recursive type */
    pthread_mutexattr_init( &mattr );
    pthread_mutexattr_settype( &mattr, PTHREAD_MUTEX_RECURSIVE );
    pthread_mutex_init( &hMutex, &mattr );

    /* create thread as detached and process scope */
    pthread_attr_init( &attr);
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
    pthread_attr_setscope(&attr, PTHREAD_SCOPE_PROCESS);
    pthread_create( &hThread, &attr, SendThread, 0);

    .....
    /* produce and post messages */
    .....

    /* clean up and exit */
    pthread_attr_destroy( &attr );
    pthread_cond_destroy( &hEvent );
    pthread_mutex_destroy( &hMutex );
}

void *
SendThread ( void *Parm )
{
    do {
          /* wait for the event */
          pthread_mutex_lock( &hMutex );
          pthread_cond_wait( &hEvent, &hMutex );
          pthread_mutex_unlock( &hMutex );

          /* send the message */
          ....
          ....

          /* lock message queue and delete message */
          pthread_mutex_lock( &hMutex );
          .....
          .....
          pthread_mutex_unlock( &hMutex );

    } while ( Run };

}

PostMessage( PMESSAGE Message )
{
    /* lock message queue and add message */
    pthread_mutex_lock( &hMutex );
    .....
    .....
    pthread_cond_signal( &hEvent );    /* signal SendThread */
    pthread_mutex_unlock( &hMutex );

}

*** UNDER CONSTRUCTION ***
 

Porting WaitForSingleObject to Linux - Part 1

Recently I was involved in porting a 32-bit application which was initially written for Microsoft Windows NT to GNU/Linux.  This application contained a large number of calls to NtWaitForSingleObject and a smaller number of calls to NtWaitForMultipleObject. 

Now anybody who has had to port code containing more than a few instances of these particular Win32 APIs, or their close cousins WaitForSingleObjectEx, MsgWaitForMultipleObjects, MsgWaitForMultipleObjectsEx, etc. to Unix or GNU/Linux is probably already shuddering with the recollection of long arduous days and nights of trial and error coding to try and correctly mimic the semantics and functionality of these particular Microsoft Windows specific APIs, but for the reader who has not yet had to attempt to port such an application, this post and my next post may help you save your sanity (and possibly your hair!) sometime in the future.

By the way, both of these APIs are marked deprecated in MSDN by Microsoft but still work as expected in Windows NT and Windows XP.  I am not sure about Windows Vista or Windows 7 as I have not tested them on these operating systems.  The two deprecated APIs have been replaced by the equivalant APIs WaitForSingleObject and WaitForMultipleObject respectively.  For the remainder of this post I shall just discuss the replacement APIs but most of what I say will be valid for either the deprecated or the replacement API.

On first examination WaitForSingleObject seems fairly benign.  The description in MSDN states that "This function returns when the specified object is in the signaled state or when the time-out interval elapses".  Sounds like a fairly simple and innocuous API, right?  Maybe something similar to the POSIX.1 API pthread_cond_timedwait.  Well, you are dead wrong and this post and the following will explain why.

WaitForSingleObject and its cousins can wait for a signal from any or all of the following "objects": change notification, console input, event, job, memory resource notification, mutex, process, semaphore, thread and waitable timer and in limited circumstances on files and file I/O.  When appropriately signaled, a thread is unblocked and continues.  No published standardized API in the GNU/Linux or Unix world comes come to handling this range of objects in a single API.

This is probably the one single area where a Win32 API is better designed than the GNU/Linux or Unix API set.  In GNU/Linux and Unix there are specific APIs to wait for different kinds of events: select waits for I/O events such as sockets, semop waits for semaphores, wait waits for child processes, etc.

Unfortunately the semantics of WaitForSingleObject and its cousins varies depending on the different object types.  For example, it destroys a thread, but only unlocks a mutex.  For this reason understanding Win32 source code from the viewpoint of porting to GNU/Linux can be much more difficult.

Our first example, i.e. locking a mutex (or mutants as they are sometimes called by Windows programmers), is probably the simplest use of WaitForSingleObject and is relatively easy to port to GNU/Linux.  Consider the following Windows code which creates 2 threads, and a mutex to control access to an integer variable x

INTEGER x;
HANDLE  hMutex;
...

void function1() 
{
   ...
   WaitForSingleObject(hMutex,INFINATE);
   x= 5;
   ReleaseMutex(hMutex);
   ...
}

void funtion2() 
{
   ...
   WaitForSingleObject(hMutex,INFINATE);
   x = 12;
   ReleaseMutex(hMutex);
   ...
}

int WINAPI WinMain(...) 
{
   hMutex = CreateMutex(NULL,FALSE,NULL);
   ....
   HANDLE hThread1 = CreateThread(..., (LPTHREAD_START_ROUTINE)function1, ...);
   ....
   HANDLE hThread2 = CreateThread(..., (LPTHREAD_START_ROUTINE)function2, ...);
   ....
}

Here is the same code ported to GNU/Linux.

int x;
pthread_mutex_t  hMutex;
...

void *function1( void *Context) 
{
   ...
   pthread_mutex_lock(&hMutex);
   x= 5;
   pthread_mutex_unlock(&hMutex);
   ...
}

void *funtion2( void *Context) 
{
   ...
   pthread_mutex_lock(&hMutex);
   x = 12;
   pthread_mutex_unlock(&hMutex);
   ...
}

int WINAPI WinMain(...) 
{
   pthread_mutexattr_t mAttr;
   pthread_t hThread1;
   pthread_t hThread2;

   pthread:mutexattr_init(&mAttr);
   pthread_mutexattr_settype(&mAttr, PTHREAD_MUTEX_RECURSIVE);
   pthread_mutex_init(&hMutex, &mAttr);
   ....
   pthread_create(&hThread1, NULL, function1, 0);
   ....
   pthread_create(&hThread2, NULL, function2, 0);
   ....

}

I am going to assume that you are familar with POSIX threads and thus the above code snippet should be fairly self-explanatory.  When dealing with a mutex, WaitForSingleObject with no timeout is functionally equivalant to pthread_mutex_lock.  The only thing of note is that you should probably use a recursive mutex since a Window thread can relock a mutex without first unlocking it.

If the second parameter in WaitForSingleObject is 0 and not INFINITE, you should probably use pthread_mutex_trylock instead of pthread_mutex_lock since WaitForSingleObject does not enter a wait state if the mutex is not, to use Microsoft's term, signaled; it always returns immediately.

If the second parameter in WaitForSingleObject is neither 0 or INFINITE, you should probably use pthread_mutex_timedlock instead of pthread_mutex_lock. Under Windows the time-out interval is in milliseconds.  Under GNU/Linux you must pass a populated timespec structure to pthread_mutex_timedlock.  Another wrinkle is that the interval clock and thus the clock resolution may or may not be based on the CLOCK_REALTIME clock depending on whether the Timers option is supported or not.  Therefore care needs to be taken to ensure that the time interval is correctly converted.

The second wrinkle is that pthread_mutex_timedlock and pthread_mutex_trylock do not work with recursive mutexes which are locked by the same thread as shown by the following example.

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <time.h>
#include <pthread.h>

int
main(int  argc,
     char *argv)
{
    pthread_mutex_t hMutex1;
    pthread_mutex_t hMutex2 = PTHREAD_MUTEX_INITIALIZER;
    pthread_mutexattr_t mAttr;
    struct timespec abstime;
    int rc;

    pthread_mutexattr_init(&mAttr);
    pthread_mutexattr_settype(&mAttr, PTHREAD_MUTEX_RECURSIVE_NP);
    pthread_mutex_init(&hMutex1, &mAttr);

    /* lock the mutexes */
    pthread_mutex_lock(&hMutex1);
    pthread_mutex_lock(&hMutex2);

    /* set absolute time to now + 5 seconds */
    clock_gettime(CLOCK_REALTIME, &abstime);
    abstime.tv_sec += 5;

    /* try to relock the first mutex */
    if ((rc = pthread_mutex_timedlock(&hMutex1, &abstime)) > 0 ) {
        fprintf(stderr, "ERROR first pthread_mutex_timedlock. %s\n", strerror(rc));
    }

    /* set absolute time to now + 5 seconds */
    clock_gettime(CLOCK_REALTIME, &abstime);
    abstime.tv_sec += 5;

    /* try to relock the second mutex */
    if ((rc = pthread_mutex_timedlock(&hMutex2, &abstime)) > 0 ) {
        fprintf(stderr, "ERROR second pthread_mutex_timedlock. %s\n", strerror(rc));
    }

    exit(EXIT_SUCCESS);
}

The only way around this problem is to either use non-recursive mutexes or ensure that threads do not attempt to relock recursive mutexes.  In my experience Windows programmers tend to be rather loose and free in this respect so careful code inspection is usually required.

Well that is all for now about the use of WaitForSingleObject with mutexes.  See my next post for ideas on how to port code which uses this API to handle events.