Thursday, September 29, 2016

Painless (native) load balancing your Siebel Enterprise

So, how do you setup the native load balancer for your AOM components in a Siebel Enterprise?

You can do it manually, of course. That's the way people is doing for years already. Just go through the documentation, plan and execute all the search-and-edit steps (although you actually don't have actually to think too much after the planning phase since those tasks are quite repetitive).

Or, you can jump the traditional last steps by:
  1. Installing Siebel::Lbconfig (available in your nearest CPAN mirror)
  2. Reading how to create a INI file.
  3. Execute in your shell:
lbconfig -e </my/directory/to/eapps*.cfg> -p <SCBroker port number> -c <my INI file>

The lbconfig program will connect to your Siebel Enterprise (using the information provided in the INI file) and create a lbconfig.txt in your current directory, with all required configuration to use the native Siebel Load Balancer. And, in the directory defined with the -e parameter, you're have the all the eapps*.cfg files copied and updated with the definitions from lbconfig.txt file (they will have a ".new" extension added).

Before you get too excited, I cheated a bit about the whole process because you're going to have install first the Siebel::Srvrmgr Perl distribution (also available in your nearest CPAN mirror). But you already have it, haven't you? :-)

Just kidding, be sure to use the last release of Siebel::Srvrmgr (0.27), some important updates were made over there, specially to allow Siebel::Lbconfig do it's stuff:
  • Support the Siebel components' start mode column from the "list comp" command output.
  • Support the Siebel Server Id from the "list server" command output.
Since this is the first release of Siebel::Lbconfig, I wouldn't be much surprised if there is a bug or two, even though there are unit tests in place (actually this is my first run with TDD and Perl). Be careful and please let me know if you find something wrong.

Sunday, May 29, 2016

Comparing Siebel component parameters

Sometimes I get some ideas for a new post on this blog but usually I don't have the time to write it down to completion. To not loose completely the opportunity, usually I start a post with only some basic notes and leave it for future review.

That's exactly how this post started. :-)

I was asked some time ago to provide performance information about EAI components deployed in a specific Siebel Enterprise. A very basic step-by-step receipt to recover those details are listed below:
  • Find which servers have the EAI components deployed
  • Get those parameters for each server
  • Generate a (hopefully) nice to read report of it.
If I want just to compare values, Siebel already has the cfgmerge tool for that. Here is a nice reference about it: http://siebeltechworld.blogspot.com/2013/04/component-parameter-export-and.html.

Since I'm a fan of srvrmgr program (in detriment of using IE + Siebel Web Client + CRTL C + CRTL V) and was on a Linux environment, that is how I did it (after connecting to the Siebel Enterprise to check which servers had the component):

srvrmgr> list comp EAI%

SV_NAME|CC_ALIAS|CC_NAME|CT_ALIAS|CG_ALIAS|CC_RUNMODE|CP_DISP_RUN_STATE|CP_STARTMODE|CP_NUM_RUN_TASKS|CP_MAX_TASKS|CP_ACTV_MTS_PROCS|CP_MAX_MTS_PROCS|CP_START_TIME|CP_END_TIME|CC_INCARN_NO|CC_DESC_TEXT|

someserver01|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |20          |1                |1               |2014-10-23 01:37:37|           |            |            |
someserver02|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |100         |1                |5               |2014-10-23 01:15:09|           |            |            |
someserver03|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |100         |3                |5               |2014-10-23 01:24:14|           |            |            |

I could also used the Server Group feature (if implemented) for that but that was not the case. And then I added following commands to a text file named "params.txt":

list param MaxTasks for component EAIObjMgr_enu
list advanced param MaxSharedDbConns for component EAIObjMgr_enu
list advanced param MinSharedDbConns for component EAIObjMgr_enu
list param MaxMTServers for component EAIObjMgr_enu
list param MinMTServers for component EAIObjMgr_enu
list param BusObjCacheSize for component EAIObjMgr_enu

And applying a bit of Bash scripting:

$ for i in $(seq 1 3); do srvrmgr -e ENTERPRISE -g $SIEBEL_GATEWAY -u sadmin -p PASS -s "someserver0${i}" -i $HOME/params.txt -o $HOME/params.out -b; cat $HOME/params.out >> "$HOME/params_result_${i}.txt"; done; rm $HOME/params.out

And done! Right? No... some more editing will be required here to get a nice table with the information per server. Boring.

When I wrote Siebel::Srvrmgr I thought it would be a better idea to make something generic to be reused later. That's a nice example of that, enter Siebel::Params::Checker.

Built upon Siebel::Srvrmgr, the Siebel::Params::Checker distribution provides the command line program scpc.pl (not a very creative name, I know) that does exactly the steps I gave above plus generating a nice to read (hopefully) HTML report.


And you should get something like that after the program finishes (you're on your own to open the HTML file):


Enjoy it! Release 0.001 is already available on CPAN, 0.002 should follow in the next few days.

Monday, April 11, 2016

eeePC 701 and Debian GNU Linux

After some attempts and adjustments, I decided to publish my notes about configuring my eeePC 701 with Debian GNU Linux "Jessie".

Initially, my idea was to setup Jessie to use the netbook as a smoker machine for CPAN. It turned out that those tips (which I gathered around and made some experiments myself) are generic enough for anybody trying to configure this particular eeePC model.

"Fix" the fans

The eeePC fans are somehow problematic, they won't work properly automatically after you install Jessie. But fixing it is basically two commands on the shell, so no big issue at all:

apt-get install lm-sensors fancontrol
pwmconfig

Without fancontrol and configuration, the fans will be working all the time. Well, it could be worst, take a look at my previous post eeePC 701 and OpenBSD.

Problems with keyboard

My keyboard options were not taken correctly during setup, probably because I made some mistake or because I replaced the default keyboard on my eeePC for a Brazilian ABNT 2. Nevertheless, the commands below will take care of it:

apt-get install console-setup
dpkg-reconfigure keyboard-configuration
service keyboard-setup restart

Compile yourself a new kernel

It is useful to create a smaller kernel and save memory and disk space by removing unnecessary features for the hardware resources lacking eeePC 701 . Also be sure set the processor as “core duo” and disable all resources for virtualization.

This process will take a long time, but it will payoff later.

First, install the following packages: fakeroot, kernel-package and linux-source-<version> (where version is whatever version you're interested in).

Then copy /usr/src/linux-source to a pendrive and compile the kernel from there. "Why a pendrive?" you might be asking yourself. Well, eeePC 701 has 4GB of SSD space and this will not help you to compile a kernel due the size of the source and temporary files. Of course, any other media will work too as long as Jessie has the proper kernel modules to mount and use it.

Then set the already discussed kernel options and compile it. I'll not give you all the details about kernel compilation on Jessie, there are a lot of tutorials available for that.

Finally, don't forget to remove the installed packages to recover disk space after installing and testing the new kernel.

Configure improvements for SSD

That's a very important configuration for eeePC 701, not only to improve speed but also to extend the SSD life. Here are the steps:
  1. Set I/O scheduler to noop (configure the systemd to start the rc.local file) with a echo 'noop > /sys/block/sda/queue/schedule' >> /etc/rc.local
  2. Use ext4 as the file system. It already has good features for SSD by default.
  3. Add “noatime,discard,commit=30” to the respective /etc/fstab entry.

Configure to ignore lid closing to avoid suspend mode

This is somehow optional. But since I was configuring the eeePC to leave it running unattended, it seems an obvious option. The alternative was removing dust from the screen and keyboard everyday...

Edit the file /etc/systemd/logind.conf to include the following two lines:

HandleLidSwitch=ignore
HandleLidSwitchDocked=ignore

Install local DNS cache

That's some kind of optimization I like to do myself for any Linux box, but since you're already at the shell, do yourself a favor and install this little guy to help you speed up DNS queries:

apt-get install dnsmasq

Conclusion

That's it! I found out that Debian "Jessie" was a good option to install on eeePC 701. It was lightweight (boot time is really fast compared to other distributions I tested) and setup was good enough. I didn't test configuring a Window Manager to it, but I believe that LXDE and Xfce are good options for the small screen of the netbook.

Thursday, April 7, 2016

Cross platform sleep with eScript

Actually this post should be split in two, but I mixing them here... anyway, you can adapt it to your specific needs.

Now and then you might find yourself in the situation that you're doing some processing with eScript and got some kind of contention that forbids the code to continue processing... could be a resource missing (delayed generation) or other conditions.

In those cases (specially if you're doing some kind of long batch processing and don't want to abort everything), you wish that the eScript related OS process just wait a bit.

eScript has this deficiency of not providing a function for that. You don't want also to put your code in a loop doing something just to expend time because it will use CPU time for that. You want it to just sit down for a while and check later if the condition regarding the contention changed.

For that, you could use SElib.dinamicLink method. This allows to load a library and execute a function from it. It is not efficient by any mean (to load an external library) but you will (hopefully) put your process to a sleep state without expending much CPU while doing it.

The library you're going to invoke depends highly on the OS you running the Siebel Server. For MS Windows, it should be "kernel32.dll". For Linux, "libc.so" will do it for you.

If you have a mixed composition of Siebel Server on different OS's, creating a reusable function gets more complicated. There are several workarounds to solve this, but I'm specially inclined in using environment variables to check for that. Those variables are easy to use and can be checked from eScript by using the Clib.getenv method.

Here is an example:

function sleep(time) {

    try {
        if ( Clib.getenv("OS") == "Windows_NT" ) {
            var miliTime = time * 1000;
            SElib.dynamicLink("kernel32.dll", "Sleep", STDCALL, militime);
        } else {
            SElib.dynamicLink("libc.so", "sleep", time);
        }
    catch(e)
    {
        TheApplication().RaiseErrorText("Failed to invoke sleep: " + e.toString() + e.errText());
        return false;
    }
    
    return true;

}

At least my MS Windows 7 box here has this "OS" environment variable by default. Linux doesn't have one, but you could create it as well.

And then you can this function whenever you need to put your code to sleep for a while, probably inside a loop.

You could also use Clib to invoke an external program that will sleep for you, but this can not be as efficient as using the OS library function for that. You should test if you find SElib to be too slow for your case.

Wednesday, March 30, 2016

How to update Oracle JRE on Ubuntu

Finally I'm glad to be able to have my Ubuntu PPA's available on Launchpad.

I'm a Ubuntu user for desktop: I use it for my personal stuff as well for development.

In Brazil it is quite common that home banking makes use of signed Java applets for authentication process. I'll not discuss the security merits of such decision (specially considering that Oracle's wants to discontinue the availability of Java applets at all in the future), but for all those cases it is required to have the Sun/Oracle JVM. Free implementations of JRE won't do.

Meanwhile I was a happy user from WebUpd8 team PPA repository for a long time, I noticed that recently they were not updating it.

So I decided to go ahead and cook something myself to fix that. Actually, I was looking for something like the "jusched.exe" program available for Windows. Never understood quite well why Linux cannot have. JRE is also available in RPM packages but I was never a fan of RPM based distributions for desktop.

Anyway, the byproduct of this idea is ready to use and available on Github as well on it's own repository at https://launchpad.net/~glasswalk3r-yahoo/+archive/ubuntu/ubuntu-justched.

Not anything fancy, but it will download the latest JRE available for your Ubuntu and generated a DEB package for it automatically. In the future I might add a GUI for it to enable the user to install the package directly from it, but right now the setup process is manually (that is, you double click the DEB package and let Ubuntu take care of it for you).

Update: although the PPA was created for Ubuntu LTS 14.04, it will work as well on 16.04 (latest release).

Thursday, December 3, 2015

Consuming Siebel web services with Perl

The first time that I tried to consume a Siebel web service with Perl wasn't a pleasant experience.

Going back to 2009, I saw a opportunity to flex my programming muscles and do it to get contact information. It was not only a opportunity to put my old book about web services with Perl to use, but also to greatly reduce operations time in the company I was working at that time.

Let's be fair: that book was really outdated. Back that time the book was published WSDL was available, but things were much easier since SOAP was RPC-based, not document based. Guess which implementation those web services in Siebel were based? Document.

I exhausted all my options with SOAP::Lite. I tried even editing manually the WSDL exported from Siebel, gave a shot with SOAP::WSDL (better, but not good enough) and after hours debugging I decided to have a chat with a fellow .Net programmer that worked in the same company and see if he was able to consume the WSDL from Siebel and do something with the results. To my shame he was and not just accomplished the job but did it in 15 minutes with a working POC.

I already had found references about XML::Compile at that moment, but the documentation from it was poor... and given the amount of time already spent, I went with my college .Net code.

Let's now go back to present. This year I decided to give another try with SOAP in Perl and the choice was with XML::Compile. After reading it's documentation (that was improved compared to the past, but still has a lot of space for doing better), starting from XML::Compile::WSDL and after a while I was able to pull up a working piece of code:

    use XML::Compile::WSDL11;
    use XML::Compile::SOAP11;
    use XML::Compile::Transport::SOAPHTTP;

    my $wsdlfile = File::Spec->catfile( 't', 'SWIContactServices.WSDL' );
    my %request = (
        ListOfSwicontactio => {
            Contact =>
                { Id => '0-1', FirstName => 'Siebel', LastName => 'Administrator' }
        }
    );

    my $wsdl = XML::Compile::WSDL11->new($wsdlfile);


    my $call = $wsdl->compileClient(
        operation      => 'SWIContactServicesQueryByExample',
        transport_hook => \&do_auth
    );

    my ( $answer, $trace ) = $call->(%request);

    if ( my $e = $@->wasFatal ) {

        $e->throw;

    } else {

        # do something with the response

    }

The "SWI Contact Services" is a vanilla Siebel inbound web service, very simple indeed. In this case, I'm just providing to it some information regarding SADMIN and recover the contact details in the response payload. Not very exciting, but SADMIN contact will always be in database.

Siebel has it's own authentication process instead of using some of the standards available out there. Unless the inbound web service in question uses that degenerate "user and password in the URL" method, you probably will want to use Siebel session management. That's exactly what the sub reference do_auth does in this sample code: such sub is well documented in XML::Compile::WSDL Pod, which includes the manipulation of the SOAP envelope containing the SOAP header, which is exactly what Siebel expects you to do to use session management.

Session management in Siebel has several advantages, including improved performance. By receiving a authentication token, several steps of authentication are skipped until that token is not valid anymore.

With that in mind, I think that doing this over and over would be quite boring. It was time to cook something and deliver it to CPAN: enter Siebel::SOAP::Auth.

By using a tiny object built with Moo now you can not only provide the Siebel authentication but also handle it entirely automatically. Here is the same code with it:

    use XML::Compile::WSDL11;
    use XML::Compile::SOAP11;
    use XML::Compile::Transport::SOAPHTTP;
    use Siebel::SOAP::Auth;

    my $wsdlfile = File::Spec->catfile( 't', 'SWIContactServices.WSDL' );
    my %request = (
        ListOfSwicontactio => {
            Contact =>
                { Id => '0-1', FirstName => 'Siebel', LastName => 'Administrator' }
        }
    );

    my $wsdl = XML::Compile::WSDL11->new($wsdlfile);
    my $auth = Siebel::SOAP::Auth->new(
        {
            user          => 'sadmin',
            password      => 'XXXXXXX',
            token_timeout => MAGIC_NUMBER,
        }
    );

    my $call = $wsdl->compileClient(
        operation      => 'SWIContactServicesQueryByExample',
        transport_hook => sub { 
            my ( $request, $trace, $transporter ) = @_;
            # request was modified
            my $new_request = $auth->add_auth_header($request);
            return $trace->{user_agent}->request($new_request);
        }
    );

    my ( $answer, $trace ) = $call->(%request);

    if ( my $e = $@->wasFatal ) {

        $e->throw;

    } else {

        # do something with the answer

    }

    $auth->find_token( $answer );

The MAGIC_NUMBER above is constant that defines the number of seconds the token will timeout.

The secret is to use the instance inside a sub reference, pass to it the $request (which will have the SOAP envelope to be manipulated) and return from the request made, exactly the same you would need to do in the do_auth sub. You need then to provide the new token  receive to the $auth object find_token method.

The good news is that a instance of Siebel::SOAP::Auth, being maintained and used during all the life time you need a web service from Siebel will take care of renewing the token, and hopefully, even avoid a additional round trip to the server in the case the token expires before it requests a new one. In the case it misses the opportunity, you need to be careful and check the exception throw by $@->wasFatal and do something with it (probably just repeat the request).

I'm still waiting for some bad news (not because I didn't find one it is not there). Please let me know if you find any!

Tuesday, November 10, 2015

eeePC 701 and OpenBSD

EeePC is somehow a quite old netbook nowadays, but there are a lot of folks around that still have it and keeping put it to good use.

I myself have a tiny 701 model. Back some years ago I used it basically for e-mail and Internet browsing, but it's limited disk space (4GB SSD) and single core processor make it useless for the current requirements for that (specially the websites that still uses the bitch called Adobe Flash). And, let's agree, any smartphone can beat it nowadays in terms of processor and memory.

Anyway, I was looking for an excuse to learn OpenBSD a bit. I always was curious about it but never gave me a change to learn it.

The "excuse" I was looking for was setting up a CPAN Reporter Smoke machine with OpenBSD, and my second thought (first was a VM in Virtualbox) was installing it on my EeePC.

I tried OpenBSD 5.7 (a few weeks before the release of 5.8) and setup run smoothly. The OpenBSD installer offered me a quite pleasant setup processes, with almost no questions and easy partitioning (very different of my first experience with it at 2000, when it offered disk space in sectors and let me do the math myself!). Keyboard configuration (something that is usually a pain to configure since I use ABNT2) was pretty simple too.

OpenBSD also does a good job about dealing with the limited disk space available on my EeePC, since it's basic install being really basic. I just removed the X-Window options from the file sets since the CPAN Reporter Smoker doesn't need one (and the screen size is really small) and fired away the installer.

Unfortunately, those were the good points about this "marriage": there are two big issues about installing OpenBSD on EeePC 701:
  1. File system (FFS).
  2. No fan control.
The default OpenBSD file system (FFS) is really slow compared to options available to Linux. Even after setting up the partitions to use noatime and soft updates my EeePC took a long time (most of it with I/O) to prepare the CPAN indexes. That's something that I could have fixed with mfs as a workaround, but the 512MB of memory of EeePC does not allow it. And, to be fair, I'm still working on providing statistics for each tested module by CPAN::Reporter::Smoker, some more things might doing the tests execution slower and I'm not aware of it.

Also, FFS does not implement TRIM (at least that's what my research found), so basically the support for SSD disks on OpenBSD is pretty weak.

The EeePC fan is another problem too. The thing is that the hardware configuration of it leaves the netbook get to high temperatures (and I'm not overclocking the processor) without turning the fans properly (yes, it really sucks). Since OpenBSD 5.7 kernel is not capable to detect the fans control, there is nothing left to do about it. I searched for some patches/user land software but none seems to be "production ready".

Conclusion

So, my conclusion about the experiment is: although OpenBSD has a good install for the limited hardware of EeePC 701, the limitations I explained above (specially number 2) tells me that is not even that safe to let the EeePC running for long periods with it, something that a CPAN Reporter Smoker requires. And, just to leave a comparison, I was able to easily install Debian Jessie on the EeePC with fan-control and lm-sensors packages, run a quickly setup and voila: the fans worked like a charm.