Saturday, December 3, 2016

Poor's man test notification

Somewhere 2014, my manager asked me to write down test cases for the application the team was developing. He didn't care the programming language I was going to use as long as I could implement it quickly.

Those tests should run regularly, and an e-mail should be sent in the event things go south.

Being used to write tests with TAP (Test::More), I went with Perl and Linux crontab for the job. Besides the tests itself (not really interesting for this post anyway), I end up with a generic script to run over the tests, analyze the results and send an e-mail with a nice HTML report attached. Here is the script:

use warnings;
use strict;
use TAP::Harness 3.30;
use TAP::Formatter::HTML 0.11;
use Email::Stuffer 0.009;
use Email::Sender::Transport::SMTP 1.300010;
use MIME::Base64 3.14;
use Authen::SASL 2.16;
use Config::Tiny 2.20;
use lib './lib';
my @tests = glob('Tests/*.t');


foreach my $test (@tests) {
   my $output_filename = 'results.html';
   my $fmt = TAP::Formatter::HTML->new();
   $fmt->output_file($output_filename);
   $fmt->verbosity(-2);
   my $harness = TAP::Harness->new(
       {
           formatter => $fmt,
           merge     => 1,
           lib       => ['lib']
       }
   );
   my $aggregator = $harness->runtests($test);


   if ( $aggregator->has_problems ) {
       send_email( $test, $output_filename );
   }
   else {
       remove_report($output_filename);
   }


}


sub remove_report {
   my $report_filename = shift;
   unlink $report_filename
     or warn "Could not remove unused report file $report_filename: $!";
}


sub send_email {
   my ($test_name, $attachment) = @_;
   my $cfg = Config::Tiny->read('xyz.ini');
   my $body = <<BLOCK;


Greetings,


Sadly a test ($test_name) executed against XYZ REST API failed.
Please check the attachment for details.


BLOCK


   my $transport = Email::Sender::Transport::SMTP->new(
       {
           host          => $cfg->{SMTP}->{'smtp.host'},
           ssl           => 1,
           sasl_username => $cfg->{SMTP}->{'smtp.username'},
           sasl_password => $cfg->{SMTP}->{'smtp.password'},
       }
   );


   Email::Stuffer->from( $cfg->{SMTP}->{'smtp.default.from.email'} )
     ->to('afreitas@foobar.com')->text_body($body)->transport($transport)
     ->subject('XYZ REST API failed')->attach_file($attachment)->send;
   remove_report($attachment);
}

Being generic enough, I could just point the script to different folders containing the tests and hopefully got nothing because all tests passed. In fact, it was so simple I wondered why something like this was not available on CPAN.

Well, not anymore... Test::Health is available for download already, so you can install it and run the script health_check.pl to run your tests. Since this script was created initially to health check the application, the name makes sense... but it seems to be generic enough to be reused in different situations that I didn't foresee when I wrote it.

The only thing that is missing from my original script is the SMTP authentication... hopefully I should implement subclasses to make it in the future (but contributions in that sense are welcome!).

Happy testing!


Tuesday, October 4, 2016

OpenBSD CPAN smoker: mission accomplished

I finally reached my personal set milestone regarding configuring and running a CPAN smoker on OpenBSD OS and grabbed me a honorable third place in number of submissions:



That was quite a ride! Despite my short experience with OpenBSD, I naively though it wouldn't be that hard... well, it was hard enough to generate some debate on CPAN Testers discussion list because  looks like nobody is doing much test on OpenBSD (or at least is not submitting results about it) nowadays.

All things considered, I decided to go a little further, so I grabbed all my lessons learned and created a document about the process. This work result is available at http://wiki.cpantesters.org/wiki/SmokerOnOpenBSD.


Thursday, September 29, 2016

Painless (native) load balancing your Siebel Enterprise

So, how do you setup the native load balancer for your AOM components in a Siebel Enterprise?

You can do it manually, of course. That's the way people is doing for years already. Just go through the documentation, plan and execute all the search-and-edit steps (although you actually don't have actually to think too much after the planning phase since those tasks are quite repetitive).

Or, you can jump the traditional last steps by:
  1. Installing Siebel::Lbconfig (available in your nearest CPAN mirror)
  2. Reading how to create a INI file.
  3. Execute in your shell:
lbconfig -e </my/directory/to/eapps*.cfg> -p <SCBroker port number> -c <my INI file>

The lbconfig program will connect to your Siebel Enterprise (using the information provided in the INI file) and create a lbconfig.txt in your current directory, with all required configuration to use the native Siebel Load Balancer. And, in the directory defined with the -e parameter, you're have the all the eapps*.cfg files copied and updated with the definitions from lbconfig.txt file (they will have a ".new" extension added).

Before you get too excited, I cheated a bit about the whole process because you're going to have install first the Siebel::Srvrmgr Perl distribution (also available in your nearest CPAN mirror). But you already have it, haven't you? :-)

Just kidding, be sure to use the last release of Siebel::Srvrmgr (0.27), some important updates were made over there, specially to allow Siebel::Lbconfig do it's stuff:
  • Support the Siebel components' start mode column from the "list comp" command output.
  • Support the Siebel Server Id from the "list server" command output.
Since this is the first release of Siebel::Lbconfig, I wouldn't be much surprised if there is a bug or two, even though there are unit tests in place (actually this is my first run with TDD and Perl). Be careful and please let me know if you find something wrong.

Sunday, May 29, 2016

Comparing Siebel component parameters

Sometimes I get some ideas for a new post on this blog but usually I don't have the time to write it down to completion. To not loose completely the opportunity, usually I start a post with only some basic notes and leave it for future review.

That's exactly how this post started. :-)

I was asked some time ago to provide performance information about EAI components deployed in a specific Siebel Enterprise. A very basic step-by-step receipt to recover those details are listed below:
  • Find which servers have the EAI components deployed
  • Get those parameters for each server
  • Generate a (hopefully) nice to read report of it.
If I want just to compare values, Siebel already has the cfgmerge tool for that. Here is a nice reference about it: http://siebeltechworld.blogspot.com/2013/04/component-parameter-export-and.html.

Since I'm a fan of srvrmgr program (in detriment of using IE + Siebel Web Client + CRTL C + CRTL V) and was on a Linux environment, that is how I did it (after connecting to the Siebel Enterprise to check which servers had the component):

srvrmgr> list comp EAI%

SV_NAME|CC_ALIAS|CC_NAME|CT_ALIAS|CG_ALIAS|CC_RUNMODE|CP_DISP_RUN_STATE|CP_STARTMODE|CP_NUM_RUN_TASKS|CP_MAX_TASKS|CP_ACTV_MTS_PROCS|CP_MAX_MTS_PROCS|CP_START_TIME|CP_END_TIME|CC_INCARN_NO|CC_DESC_TEXT|

someserver01|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |20          |1                |1               |2014-10-23 01:37:37|           |            |            |
someserver02|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |100         |1                |5               |2014-10-23 01:15:09|           |            |            |
someserver03|EAIObjMgr_enu|EAI Object Manager (ESN)|EAIObjMgr|EAI     |Interactive|Activated         |Auto        |0               |100         |3                |5               |2014-10-23 01:24:14|           |            |            |

I could also used the Server Group feature (if implemented) for that but that was not the case. And then I added following commands to a text file named "params.txt":

list param MaxTasks for component EAIObjMgr_enu
list advanced param MaxSharedDbConns for component EAIObjMgr_enu
list advanced param MinSharedDbConns for component EAIObjMgr_enu
list param MaxMTServers for component EAIObjMgr_enu
list param MinMTServers for component EAIObjMgr_enu
list param BusObjCacheSize for component EAIObjMgr_enu

And applying a bit of Bash scripting:

$ for i in $(seq 1 3); do srvrmgr -e ENTERPRISE -g $SIEBEL_GATEWAY -u sadmin -p PASS -s "someserver0${i}" -i $HOME/params.txt -o $HOME/params.out -b; cat $HOME/params.out >> "$HOME/params_result_${i}.txt"; done; rm $HOME/params.out

And done! Right? No... some more editing will be required here to get a nice table with the information per server. Boring.

When I wrote Siebel::Srvrmgr I thought it would be a better idea to make something generic to be reused later. That's a nice example of that, enter Siebel::Params::Checker.

Built upon Siebel::Srvrmgr, the Siebel::Params::Checker distribution provides the command line program scpc.pl (not a very creative name, I know) that does exactly the steps I gave above plus generating a nice to read (hopefully) HTML report.


And you should get something like that after the program finishes (you're on your own to open the HTML file):


Enjoy it! Release 0.001 is already available on CPAN, 0.002 should follow in the next few days.

Monday, April 11, 2016

eeePC 701 and Debian GNU Linux

After some attempts and adjustments, I decided to publish my notes about configuring my eeePC 701 with Debian GNU Linux "Jessie".

Initially, my idea was to setup Jessie to use the netbook as a smoker machine for CPAN. It turned out that those tips (which I gathered around and made some experiments myself) are generic enough for anybody trying to configure this particular eeePC model.

"Fix" the fans

The eeePC fans are somehow problematic, they won't work properly automatically after you install Jessie. But fixing it is basically two commands on the shell, so no big issue at all:

apt-get install lm-sensors fancontrol
pwmconfig

Without fancontrol and configuration, the fans will be working all the time. Well, it could be worst, take a look at my previous post eeePC 701 and OpenBSD.

Problems with keyboard

My keyboard options were not taken correctly during setup, probably because I made some mistake or because I replaced the default keyboard on my eeePC for a Brazilian ABNT 2. Nevertheless, the commands below will take care of it:

apt-get install console-setup
dpkg-reconfigure keyboard-configuration
service keyboard-setup restart

Compile yourself a new kernel

It is useful to create a smaller kernel and save memory and disk space by removing unnecessary features for the hardware resources lacking eeePC 701 . Also be sure set the processor as “core duo” and disable all resources for virtualization.

This process will take a long time, but it will payoff later.

First, install the following packages: fakeroot, kernel-package and linux-source-<version> (where version is whatever version you're interested in).

Then copy /usr/src/linux-source to a pendrive and compile the kernel from there. "Why a pendrive?" you might be asking yourself. Well, eeePC 701 has 4GB of SSD space and this will not help you to compile a kernel due the size of the source and temporary files. Of course, any other media will work too as long as Jessie has the proper kernel modules to mount and use it.

Then set the already discussed kernel options and compile it. I'll not give you all the details about kernel compilation on Jessie, there are a lot of tutorials available for that.

Finally, don't forget to remove the installed packages to recover disk space after installing and testing the new kernel.

Configure improvements for SSD

That's a very important configuration for eeePC 701, not only to improve speed but also to extend the SSD life. Here are the steps:
  1. Set I/O scheduler to noop (configure the systemd to start the rc.local file) with a echo 'noop > /sys/block/sda/queue/schedule' >> /etc/rc.local
  2. Use ext4 as the file system. It already has good features for SSD by default.
  3. Add “noatime,discard,commit=30” to the respective /etc/fstab entry.

Configure to ignore lid closing to avoid suspend mode

This is somehow optional. But since I was configuring the eeePC to leave it running unattended, it seems an obvious option. The alternative was removing dust from the screen and keyboard everyday...

Edit the file /etc/systemd/logind.conf to include the following two lines:

HandleLidSwitch=ignore
HandleLidSwitchDocked=ignore

Install local DNS cache

That's some kind of optimization I like to do myself for any Linux box, but since you're already at the shell, do yourself a favor and install this little guy to help you speed up DNS queries:

apt-get install dnsmasq

Conclusion

That's it! I found out that Debian "Jessie" was a good option to install on eeePC 701. It was lightweight (boot time is really fast compared to other distributions I tested) and setup was good enough. I didn't test configuring a Window Manager to it, but I believe that LXDE and Xfce are good options for the small screen of the netbook.

Thursday, April 7, 2016

Cross platform sleep with eScript

Actually this post should be split in two, but I mixing them here... anyway, you can adapt it to your specific needs.

Now and then you might find yourself in the situation that you're doing some processing with eScript and got some kind of contention that forbids the code to continue processing... could be a resource missing (delayed generation) or other conditions.

In those cases (specially if you're doing some kind of long batch processing and don't want to abort everything), you wish that the eScript related OS process just wait a bit.

eScript has this deficiency of not providing a function for that. You don't want also to put your code in a loop doing something just to expend time because it will use CPU time for that. You want it to just sit down for a while and check later if the condition regarding the contention changed.

For that, you could use SElib.dinamicLink method. This allows to load a library and execute a function from it. It is not efficient by any mean (to load an external library) but you will (hopefully) put your process to a sleep state without expending much CPU while doing it.

The library you're going to invoke depends highly on the OS you running the Siebel Server. For MS Windows, it should be "kernel32.dll". For Linux, "libc.so" will do it for you.

If you have a mixed composition of Siebel Server on different OS's, creating a reusable function gets more complicated. There are several workarounds to solve this, but I'm specially inclined in using environment variables to check for that. Those variables are easy to use and can be checked from eScript by using the Clib.getenv method.

Here is an example:

function sleep(time) {

    try {
        if ( Clib.getenv("OS") == "Windows_NT" ) {
            var miliTime = time * 1000;
            SElib.dynamicLink("kernel32.dll", "Sleep", STDCALL, militime);
        } else {
            SElib.dynamicLink("libc.so", "sleep", time);
        }
    catch(e)
    {
        TheApplication().RaiseErrorText("Failed to invoke sleep: " + e.toString() + e.errText());
        return false;
    }
    
    return true;

}

At least my MS Windows 7 box here has this "OS" environment variable by default. Linux doesn't have one, but you could create it as well.

And then you can this function whenever you need to put your code to sleep for a while, probably inside a loop.

You could also use Clib to invoke an external program that will sleep for you, but this can not be as efficient as using the OS library function for that. You should test if you find SElib to be too slow for your case.

Wednesday, March 30, 2016

How to update Oracle JRE on Ubuntu

Finally I'm glad to be able to have my Ubuntu PPA's available on Launchpad.

I'm a Ubuntu user for desktop: I use it for my personal stuff as well for development.

In Brazil it is quite common that home banking makes use of signed Java applets for authentication process. I'll not discuss the security merits of such decision (specially considering that Oracle's wants to discontinue the availability of Java applets at all in the future), but for all those cases it is required to have the Sun/Oracle JVM. Free implementations of JRE won't do.

Meanwhile I was a happy user from WebUpd8 team PPA repository for a long time, I noticed that recently they were not updating it.

So I decided to go ahead and cook something myself to fix that. Actually, I was looking for something like the "jusched.exe" program available for Windows. Never understood quite well why Linux cannot have. JRE is also available in RPM packages but I was never a fan of RPM based distributions for desktop.

Anyway, the byproduct of this idea is ready to use and available on Github as well on it's own repository at https://launchpad.net/~glasswalk3r-yahoo/+archive/ubuntu/ubuntu-justched.

Not anything fancy, but it will download the latest JRE available for your Ubuntu and generated a DEB package for it automatically. In the future I might add a GUI for it to enable the user to install the package directly from it, but right now the setup process is manually (that is, you double click the DEB package and let Ubuntu take care of it for you).

Update: although the PPA was created for Ubuntu LTS 14.04, it will work as well on 16.04 (latest release).