Wednesday, May 29, 2013

Siebel eScript and your file system

Everybody knows (including you, right?) that the usage of eScript should be limited for customizations in Siebel in all cases that is not possible (or too hard) to use Siebel Workflows and vanilla Business Services.

Considered that, eScript surely has it's space: in some cases, applying business logic with eScript is far easier than using a Workflow. eScript also has more generic features than Workflows, like working with Siebel Server file system.

Clib provides several functions to work with file systems, most of them looking like ANSI C functions. I won't write here the same information that is already available at the corresponding Siebel Bookshelf (Siebel eScript Language Reference) but I will give some tips about what I had learned the hard way.

Test the directory before using it

Before trying to use a directory it is always a good idea to check if the eScript will have access to it (specially if the Siebel version that you have doesn't have the Clib.strerror working correctly). One way to test a directory is using the functions Clib.chdir and Clib.getcwd combined, as shown the testDir function below:

function testDir(dir)
{
    try
    {
        //tests if the directory is valid
        var previousDir = Clib.getcwd();
        var testResult = -1;
      
        testResult = Clib.chdir( dir );
      
        if ( testResult == -1 )
        {
            TheApplication().RaiseErrorText('The directory ' + dir + ' does not exists or cannot be accessed.');
        }
    }
    catch(e)
    {
        TheApplication().RaiseErrorText('Error in function testDir: ' + e.toString());
    }
}


This will help to find out missing file system permissions (or missing the directory itself) before trying to use the directory.

Of course, using Clib.strerror should be your first option if the function is returning the error message from the OS as expected!

Check if the file was really opened

Looks like a silly thing, but here and then I find some programs that still do not check the file pointer after trying to open a file with Clib.fopen. It is very simple to test it, as you can check below:

    fp = Clib.fopen(myFile,"au");

    if ( fp == null )
    {
        TheApplication().RaiseErrorText('An error ocurrered when trying to open ' + myFile);
    }


With three lines of code you will avoid having to debug code due file system permissions, for example. Why that? Because those errors are not considered exceptions by Clib, and failure will happen only if you try to use the file pointer...

By the way, the functions Clib.fputs e Clib.fclose follows the same idea, returning null in something goes wrong. If you're felling specially paranoiac you can include tests for those functions as well.

Use the buffer Luke!

The read and write functions of Clib use buffering by default: that means they will keep data in memory before doing a I/O operation. This helps with I/O performance of your program since read/writing to the disk is slower than in RAM memory (I'm not even considering disk cache). If you write code like this in the same scope:

    fp = Clib.fopen(myFile,"au");

    if ( fp == null )
    {
        TheApplication().RaiseErrorText('An error ocurrered when trying to open ' + myFile);
    }
   
    Clib.fputs(myData,fp);
    Clib.fclose(fp);

    //later your repeat the same process

you're simply wasting buffer usage.

Once a college of mine argued that using Clib for logging messages would be "reinventing the wheel" since there are already functions for that like TheApplication().Trace. The problem with Trace is that you don't have any control about the buffer: the function would (probably) use I/O immediately to the disk, in other words:
  1. Open the file.
  2. Write to the file.
  3. Close the file.
Closing a file does an implicit flush of buffer.

With Clib you could have better control about buffer usage and if you need to write large sets of data to a file it will help with the I/O performance.

To accomplish that, just create a global file pointer variable (in the Declarations area) and open the file just once with Clib.fopen. After that, just go on using Clib.fputs and it's cousins.

How to guarantee that the file will be properly closed? This is even more important if the Siebel Server is using a file system with mandatory locking like NTFS: the file will be kept as locked until the process that opened it close the file or cease to exist.

To avoid this kind of issue, just use the finally block to close the file:

# Declarations area
    var fp;
   
function foobar {

    try {

    fp = Clib.fopen(myFile,"au");

    if ( fp == null )
    {
        TheApplication().RaiseErrorText('An error ocurrered when trying to open ' + myFile);
    }
   
    Clib.fputs(myData,fp);
   
    } catch(e) {
   
        //do something
   
    } finally {
   
        Clib.fclose(fp);
   
    }
   
}


Doing this will guarantee that the process that is executing the Business Service will release the file lock and the file will be closed, having all data that was not yet written to the file to be flushed from the buffer.

And finally, if you need to give more security to not loosing data in the case of a hard failure, at each execution of  Clib.fputs execute a Clib.fflush, without closing the file.

Copying files

In the article "How Can You Copy Files from One Location to Another Using Siebel Scripting?" (ID 477948.1) available at Oracle support webpage there is a description of some solutions to copy files with eScript since Clib does not have those features (one thing that I still trying to understand why not).

One of the solutions is to use Clib.rename: if you ever used any UNIX shell you will notice it looks like the mv command.

The other alternatives in the article (Clib.system and SElib) would be too "expensive" in terms of computer resources to use in my opinion (while I must say I never did a formal test about this).

If you need to copy lots and lots of files, my suggestion is to create a small program (in any language that you prefer) and then execute it with Clib.system: this way you will create just one additional OS process to do all the necessary work.

Listing files in a directory

This tip only works for Siebel setups in a Microsoft Windows OS.

The Oracle technical support website has several articles teaching to use SElib and the kernel32.dll to look for filenames patterns in some directory. Unfortunately those articles just teach you that you should use kerne32.dll for the job, but not how to use it. I had already saw code that keeps calling FindFirstFileA recursively when the function FindNextFileA is just there waiting to be used. Reading the kernel32.dll documentation is a painful process if you're not a C++ programmer (and I'm not) but it is there ((http://msdn.microsoft.com/en-us/library/windows/desktop/aa364418%28v=vs.85%29.aspx) to be checked to avoid doing silly things with the DLL.

Another issue is to deal with NUL characters: a predefined amount of memory is allocated to keep the filename that was found and in the case missing something more interesting the functions FindFirstFileA and FindNextFileA fill the missing alocated space with NUL characters if the filename is as large as the buffer. This is quite normal to be expected if you're programming with C or C++, but try to concatenate strings (or do anything else like using split) with eScript and the result will not be something that you're expecting.

The function below is a possible solution for the problem (probably not the most elegant one): it will look for a NUL character in a string and return the number of "printable" characters read until it.


function removeNul(string)
{
    try
    {
        var iEndPos = string.length;
        var lastPrintChar = 0;

        for (var i = 0; i < iEndPos; i++) {
 
            //if char is printable, go to next until test fails
            if ( Clib.isprint( string.charAt(i) ) )
            {
           
                lastPrintChar++;
           
            }
            else {
           
                break;
           
            }
       
        }
       
        return string.substring(0, lastPrintChar);
   
    }
    catch(e) {
   
        TheApplication().RaiseErrorText('An error ocurred with removeNul function: ' + e.errText);
   
    }

}


Finally, below I put a complete example of listing files considering the points that I commented, just to illustrate the concepts. Pay attention that the results of found files are stored in a array so the garbage collector of eScript will have a chance to release as soon as possible the allocated memory for the objects and kernel32.dll.

    try {

        var blob = new blobDescriptor();
        var fileData;
        var hFind;
       
        blob.dwFileAttributes   = SWORD32;
        blob.ftCreationTime     = SWORD32;
        blob.ftCreationTime_    = SWORD32;
        blob.ftLastAccessTime   = SWORD32;
        blob.ftLastAccessTime_  = SWORD32;
        blob.ftLastWriteTime    = SWORD32;
        blob.ftLastWriteTime_   = SWORD32;
        blob.nFileSizeHigh      = SWORD32;
        blob.nFileSizeLow       = SWORD32;
        blob.dwReserved0        = SWORD32;
        blob.dwReserved1        = SWORD32;
        blob.cFileName          = 1000000;
        blob.cAlternateFileName = 1000000;
       
        var nextIndex = filesToProcess.length;
       
        // brand new start of filesToProcess
        if (typeof nextIndex == 'undefined') {
       
            nextIndex = 0;
            filesToProcess[0] = '';
       
        }
       
        hFind = SElib.dynamicLink("Kernel32.DLL", "FindFirstFileA", STDCALL, filenameToMatch, blob, fileData);

        //found a file with the request pattern
        if (hFind != -1) {

            filesToProcess[nextIndex] = removeNul(fileData.cFileName.toString());

            while (SElib.dynamicLink("Kernel32.DLL", "FindNextFileA", STDCALL, hFind, blob, fileData) != 0)
            {
           
             nextIndex++;
            
             filesToProcess[nextIndex] = removeNul(fileData.cFileName.toString());            
           
            }
               
            return true;
        }
    }
    catch(e) {
   
        throw(e);
       
    }
    finally {
   
        nextIndex = null;
        fileData = null;
   
    }
}


Thinking a little bit more, working with a UNIX-like OS would be possible if you have a library with the same functions of kernel32.dll, but this would be highly dependent of the OS being used (as the same thing happens to kernel32.dll). So, I wonder, why doesn't Siebel have such an library? Doing something with ANSI C should be enough portable.

Read and write in Unicode

One of the file modes available for Clib.fopen is reading and writing in Unicode by using the "u" character specified.

Working with this mode is a better guarantee that you will be able to export or import data in a portable way between systems, specially if the Siebel data base is configured to use Unicode. Working with Unicode will avoid a lot of headaches with accents and other characters.

But be sure to remember this: about the time I'm writing this article, Siebel regular expressions does not support Unicode.