Wednesday, November 26, 2014

Better checking your Java version

A couple of days ago I received a task to check installed applications in a bunch or servers. One of the things to check was the Java VM installed into those and see which versions was there.

It looks a simple task, but it is not: some applications requires different versions of Java, so there is none "one JVM to rule them all".

It should a be simple thing to do it automatically, but actually Java doesn't help as much I as would like. This is the output from Java in my MS Windows box:

C:\>java -version
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) Client VM (build 24.71-b01, mixed mode, sharing)

C:\>


The version is there, right? Well, it is, but can you tell me if this JVM is 64 or 32 bits version?

The truth is, if the JVM is 64 bits it should let me know... but for 32 I'm just guessing. That's OK when I'm doing it manually, but guessing from script is not so.

After some searching over the Internet I got this ridiculous simple code, actually I could say this could be the "Hello World" example of Java, and here it goes:

public class JavaArch {
 

    public static void main(String[] args) {
                          System.out.println(System.getProperty("sun.arch.data.model"));


    }

}


Compile that code with javac and ship it with your script and you're good to go. No guessing required, just the bare information.

C:\>javac JavaArch.java

C:\>java JavaArch
32

C:\>


Can't be easier than that.

See more information about here.

If you need even more properties available for the environment from the JVM point of view, go checking http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#getProperties%28%29.

Here is an example from my environment:

C:\>java JavaArch
java.runtime.name = Java(TM) SE Runtime Environment
sun.boot.library.path = C:\Program Files (x86)\Java\jre7\bin
java.vm.version = 24.71-b01
user.country.format = BR
java.vm.vendor = Oracle Corporation
java.vendor.url = http://java.oracle.com/
path.separator = ;
java.vm.name = Java HotSpot(TM) Client VM
file.encoding.pkg = sun.io
user.script =
user.country = US
sun.java.launcher = SUN_STANDARD
sun.os.patch.level = Service Pack 1
java.vm.specification.name = Java Virtual Machine Specification
user.dir = C:\Users\foobar\Documents\temp
java.runtime.version = 1.7.0_71-b14
java.awt.graphicsenv = sun.awt.Win32GraphicsEnvironment
java.endorsed.dirs = C:\Program Files (x86)\Java\jre7\lib\endorsed
os.arch = x86
java.io.tmpdir = C:\Users\foobar\AppData\Local\Temp\
line.separator =

java.vm.specification.vendor = Oracle Corporation
user.variant =
os.name = Windows 7
sun.jnu.encoding = Cp1252
java.library.path = C:\Program Files (x86)\Java\jre7\bin;C:\windows\Sun\Java\bin;C:\windows\system32;C:\windows;C:\oracle\product\11.2.0\client_1\bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Lenov
o\Fingerprint Manager Pro\;C:\strawberry\c\bin;C:\strawberry\perl\site\bin;C:\strawberry\perl\bin;C:\oracle\product\11.2.0\client_1\bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Lenovo\Fingerprint
Manager Pro\;C:\strawberry\c\bin;C:\strawberry\perl\site\bin;C:\strawberry\perl\bin;C:\Program Files (x86)\GnuWin32\bin;C:\Program Files (x86)\Java\jre7\bin;C:\Program Files (x86)\PuTTY;C:\Users\foobar\Downloads\sqldeveloper\jdk\bin;.
java.specification.name = Java Platform API Specification
java.class.version = 51.0
sun.management.compiler = HotSpot Client Compiler
os.version = 6.1
user.home = C:\Users\foobar
user.timezone =
java.awt.printerjob = sun.awt.windows.WPrinterJob
file.encoding = Cp1252
java.specification.version = 1.7
user.name = foobar
java.class.path = .
java.vm.specification.version = 1.7
sun.arch.data.model = 32
java.home = C:\Program Files (x86)\Java\jre7
sun.java.command = JavaArch
java.specification.vendor = Oracle Corporation
user.language = en
user.language.format = pt
awt.toolkit = sun.awt.windows.WToolkit
java.vm.info = mixed mode, sharing
java.version = 1.7.0_71
java.ext.dirs = C:\Program Files (x86)\Java\jre7\lib\ext;C:\windows\Sun\Java\lib\ext
sun.boot.class.path = C:\Program Files (x86)\Java\jre7\lib\resources.jar;C:\Program Files (x86)\Java\jre7\lib\rt.jar;C:\Program Files (x86)\Java\jre7\lib\sunrsasign.jar;C:\Program Files (x86)\Java\jre7\lib\jsse.jar;C:\Program Files (x86)\Java\jre7\lib\jce.jar;C:\Program Files (x86)\Java\jre7\lib\charsets.jar;C:\Program Files (x86)\Java\jre7\lib\jfr.jar;C:\Program Files (x86)\Java\jre7\classes
java.vendor = Oracle Corporation
file.separator = \
java.vendor.url.bug = http://bugreport.sun.com/bugreport/
sun.cpu.endian = little
sun.io.unicode.encoding = UnicodeLittle
sun.desktop = windows
sun.cpu.isalist = pentium_pro+mmx pentium_pro pentium+mmx pentium i486 i386 i86


And the updated code to retrieve that:

import java.util.*;

public class JavaArch {

    public static void main(String[] args) {
   
        Properties props = System.getProperties();
        Enumeration names = props.propertyNames();
       
        while (names.hasMoreElements()) {
       
            String current = names.nextElement().toString();
       
            System.out.println(current + " = " + System.getProperty(current));
       
        }
   
    }

}

Monday, July 21, 2014

Some notes about srvrmgr usage

The command line program srvrmgr is not that popular in Siebel, but any reasonably seasoned sadmin will soon discover how useful it is considering the GUI limitations.

One of the most obvious reasons to use is when the AOM is out-of-service for any reason and you don't have a Siebel Client Dedicated configured. Other usage is for tasks automation.

This posts is about some tips that I found here and there about using srvrmgr.

Note: this post was updated at October 6, 2015 to include additional content.

Hide your password

Let's consider for a moment the password usage with the program: you may have notice that doesn't matter if you use the "-p" option or not, the password is always echoed to the standard output!

There are different forms to solve this, but the better one is to disable the echo when typing the password. Several programming languages may help you in this. With Perl, for example, this is very straightforward in a Microsoft Windows environment:

use Term::ReadKey qw(ReadMode);
use Win32::Process;

my $server     = '';
my $gateway    = '';
my $enterprise = '';
my $user       = '';
my $srvrmgr    = 'sea752\\client\\BIN\\srvrmgr.exe';

system('chcp 1252');

print 'Type your password to connect to Siebel Server: ';
ReadMode(2);
my $password = ;
ReadMode(0);

chomp($password);

print "\n\n";

my $process;

Win32::Process::Create( $process, $srvrmgr,
    "srvrmgr /e $enterprise /g $gateway /u $user /p $password",
    0, NORMAL_PRIORITY_CLASS, '.' )
  or die errorReport();

$process->Wait(10);

sub errorReport {

    print Win32::FormatMessage( Win32::GetLastError() );

}

Of course, any modern programming language will help you with that.

If you have a MS Windows environment and is particularly interested in better hiding the password, take a look at the Server Manager 2 program that is part of the Siebel GNU Tools project.

Configuring the output

Character encoding

First, let's talk about configuring character encoding. Let's suppose that your system is configured in PTB and you run a simple list comp command in the srvrmgr prompt:

srvrmgr> list comp

CC_ALIAS           CC_NAME                               CG_ALIAS      CC_RUNMODE   CP_DISP_RUN_STATE
-----------------  ------------------------------------  ------------  -----------  -----------------
ClientAdmin        Client Administration                 System        Background   Ativado
CommConfigMgr      Communications Configuration Manager  CommMgmt      Batch        Ativado
CommOutboundMgr    Communications Outbound Manager       CommMgmt      Batch        Ativado
CommSessionMgr     Communications Session Manager        CommMgmt      Batch        Ativado
DbXtract           Database Extract                      Remote        Batch        Ativado
EAIObjMgr_enu      EAI Object Manager (ENU)              EAI           Interactive  Ativado
MailMgr            Email Manager                         CommMgmt      Background   Ativado
EIM                Enterprise Integration Mgr            EAI           Batch        Ativado
FSMSrvr            File System Manager                   System        Batch        Ativado
GenNewDb           Generate New Database                 Remote        Batch        Ativado
GenTrig            Generate Triggers                     Workflow      Batch        Ativado
PageMgr            Page Manager                          CommMgmt      Background   Ativado
PDbXtract          Parallel Database Extract             Remote        Batch        Em execuþÒo
RepAgent           Replication Agent                     Remote        Background   Ativado
ServerMgr          Server Manager                        System        Interactive  Em execuþÒo
SRBroker           Server Request Broker                 System        Interactive  Em execuþÒo
SRProc             Server Request Processor              System        Interactive  Em execuþÒo
SynchMgr           Synchronization Manager               Remote        Interactive  Ativado
TxnMerge           Transaction Merger                    Remote        Background   Em execuþÒo
TxnProc            Transaction Processor                 Remote        Background   Em execuþÒo
TxnRoute           Transaction Router                    Remote        Background   Em execuþÒo
UpgKitBldr         Upgrade Kit Builder                   SiebAnywhere  Batch        Ativado
WorkActn           Workflow Action Agent                 Workflow      Background   Ativado
WorkMon            Workflow Monitor Agent                Workflow      Background   Ativado
WfProcBatchMgr     Workflow Process Batch Manager        Workflow      Batch        Ativado
WfProcMgr          Workflow Process Manager              Workflow      Batch        Ativado
ePharmaObjMgr_enu  ePharma Object Manager (ENU)          LifeSciences  Interactive  Ativado
ePharmaObjMgr_esn  ePharma Object Manager (ESN)          LifeSciences  Interactive  Ativado
ePharmaObjMgr_ptb  ePharma Object Manager (PTB)          LifeSciences  Interactive  Ativado  
  
Pretty standard output, except by the weird characters. Output from srvrmgr is usually Unicode (UTF-8) but your OS terminal must be set to use it.

If your are in a Microsoft Windows environment, you will have to change your current code page. Your default configuration may vary, but to get all the characters corrected printed (assuming the output is in UTF-8) you will need to set it to 65001:

I:\> chcp 65001
Active code page: 65001

That will do. In Linux, most probably your terminal is already configured to UTF-8. If not, you may want to take a look at exporting the LC_ALL variable to your language. For Brazilian Portuguese, that would be the desired value:

export LC_ALL=pt_BR.UTF-8

That may vary depending in your Linux distribution. If you are on UNIX, you may need to consult the OS documentation for that.
It is also good to remember that the spool command may also follow your database encoding. At least when I used databases in UTF-8, the output of srvrmgr followed the same.

Manipulating the commands output

It is possible to change the way that srvrmgr will show the output from commands. For example, the default output of list comp is this:

srvrmgr> list comp
SV_NAME     CC_ALIAS                       CC_NAME                                   CT_ALIAS  CG_ALIAS      CC_RUNMODE   CP_DISP_RUN_STATE  CP_NUM_RUN_  CP_MAX_TASK  CP_ACTV_MTS  CP_MAX_MTS_ CP_START_TIME        CP_END_TIME  CP_STATUS  CC_INCARN_NO  CC_DESC_TEXT
----------  -----------------------------  ----------------------------------------  --------  ------------  -----------  -----------------  -----------  -----------  -----------  -----------  -------------------  -----------  ---------  ------------  ------------
siebsrvdev  BusIntBatchMgr                 Business Integration Batch Manager                  EAI           Batch        Online             0            20           1            1            2012-03-01 14:59:11
siebsrvdev  BusIntMgr                      Business Integration Manager                        EAI           Batch        Online             0            20           1            1            2012-03-01 14:59:11
siebsrvdev  ClientAdmin                    Client Administration                               System        Background   Online             0            1                                      2012-03-01 14:59:11
        
The output is a bit difficult to read and I did it intentionally: this is exactly how the data is displayed in my terminal. If you use this frequently, it may be a pain.
This can be changed with the command configure list. Let's see it's on line help:

srvrmgr> help configure list
  configure list  
       [show  [ AS ]]
       [,     [ AS ]] ....

The command allows you to configure the columns that will be shown, the size of it and even creating a different alias for them.

You can also check the current configuration of list comp:

srvrmgr> configure list comp
    SV_NAME (31):  Server name
    CC_ALIAS (31):  Component alias
    CC_NAME (76):  Component name
    CT_ALIAS (31):  Component type alias
    CG_ALIAS (31):  Component GRoup Alias
    CC_RUNMODE (31):  Component run mode (enum)
    CP_DISP_RUN_STATE (61):  Component display run state
    CP_NUM_RUN_TASKS (11):  Current number of running tasks
    CP_MAX_TASKS (11):  Maximum tasks configured
    CP_ACTV_MTS_PROCS (11):  Active MTS control processes
    CP_MAX_MTS_PROCS (11):  Maximum MTS control processes
    CP_START_TIME (21):  Component start time
    CP_END_TIME (21):  Component end time
    CP_STATUS (251):  Component-reported status
    CC_INCARN_NO (23):  Incarnation Number
    CC_DESC_TEXT (251):  Component description

With the show option you be able to see which columns you want and in the desired sequence:

srvrmgr> configure list comp show SV_NAME(31),CC_ALIAS(21),CC_NAME(76)
srvrmgr> configure list comp
        SV_NAME (31):  Server name
        CC_ALIAS (21):  Component alias
        CC_NAME (76):  Component name
srvrmgr> list comp
SV_NAME     CC_ALIAS              CC_NAME
----------  --------------------  ----------------------------------------
siebsrvdev  BusIntBatchMgr        Business Integration Batch Manager
siebsrvdev  BusIntMgr             Business Integration Manager
siebsrvdev  ClientAdmin           Client Administration
siebsrvdev  CommConfigMgr         Communications Configuration Manager
siebsrvdev  CommInboundMgr        Communications Inbound Manager
...
siebsrvdev  eSitesClinicalObjMgr  eSitesClinical Object Manager (PTB)
52 rows returned.

Now it is easier on the eyes! But what if you need all the columns to be displayed?
There is a trick for that and as far as I know, it's undocumented. I present you the set command. With it you can change or (show current) configuration. Let's see the options available:

The parameter we want here is ColumnWidth. If you enable it will be possible to define the width of the columns output using the configure list command:

srvrmgr> set ColumnWidth true
srvrmgr> set
username            sadmin*
service             (null)
enterprise          (null)
app server          (null)
compression         FALSE*
encryption          FALSE*
protocol            TCP/IP*
port                (null)*
cluster             (null)*
name server         (null)*
log filter          (null)
verbose             FALSE
batch               FALSE
header              TRUE
footer              TRUE
ColumnWidth         1
delimiter


All done! If you paid attention, you saw that the configure list command uses the column name followed by a parenthesis with a integer between: this is the width in characters of the column. With the command below you can change the columns width to have longer (or narrower) output. Beware that you may truncate the output or even the columns names if the width is not larger enough:

srvrmgr> configure list comp show
SV_NAME(31),CC_ALIAS(21),CC_NAME(76),CT_ALIAS(31),CG_ALIAS(31),CC_RUNMODE(31),CP_DISP_RUN_STATE(61),CP_NUM_RUN_TASKS(16),CP_MAX_TASKS(12),CP_ACTV_MTS_PROCS(17),CP_MAX_MTS_PROCS(16),CP_START_TIME(21),CP_END_TIME(21),CP_STATUS(251),CC_INCARN_NO(23),CC_DESC_TEXT(251)

Time saving tips

Avoid some typing

After configuring your optimal output, you may wonder that is a bit boring to repeat this every time. That's right, it would be boring.
Here are some commands to save your time and patience:
  • alias
  • save preferences
  • load preferences
  • read
alias is for creating aliases for your commands. You type:

alias [name] [command]

And that's it.

save and load preferences will keep your current environment configuration to a text file and load those preferences, respectively. The default location for this file will always be the bin directory where the srvrmgr program is available and the name of the file will always be ".Siebel_svrmgr.pref".
Beware that any set you used won't be save to the preferences file. Of course, being a text file is just a matter to open it and repeat the commands there.
Then you may be in trouble if the srvrmgr is shared between different users. You may want to have your own copy of preferences. That's where read commands helps:

read c:\temp\create_custom_compdefs.txt

Use the wild card

You can filter further the output from any command by using the wild card "%" (without the parenthesis) for some of the values that are part of the output. Depending on the command, this is be applicable to different columns but not all of them. For example, if you want to list components that have the alias starting with "wf":
list comp wf%
The wild card only matches what starts or ends with the given pattern. You cannot use both at the same time.

Configure and use Server Group feature

Since Siebel version 8 you can use the feature of Server Group with srvrmgr. This feature allow you to create... servers groups. This will allow you to connect to a defined group and send a single command to all of them at once. When you want to connect to the server group, include the /z command line option to the server, followed by the server group name you had defined previously.
To create a server group, use the command below to each one of the server that you want to be part of the group:
  
change attribute groupname=[server group name] for server [siebel server name]

This example is assuming that you had connected to the Enterprise instead of specifying the Siebel server name.
To remove a Siebel Server from a Server Group use:

change attribute groupname=" " for server [siebel server name]

For any reason that escapes my imagination, it is necessary to have a space between the double quotes.

Performance tips

And here are some tips related to performance when using srvrmgr:
  • srvrmgr program instantiates new processes of Server Manager component, which is not a multi-thread component, that means that each connection creates a new process, which is a expensive task for any OS.
  • use the "/s" options whenever possible to retrieve data: instead of connecting to the enterprise and submitting a command to all servers, connect individually to each server and retrieve the data you want: this will avoid serializing the output and creating additional processes in the other servers instead of a single one.
  • if you have a lot of commands to send to the Server Manager component, don't use several srvrmgr executions with the "/c" option. Instead, write all the commands to a text file (one command per line) and use the "/i". This will use a single connection to run all the commands. You most probably will want to the check the "/o" and "/b" options too.

Batch mode tips

srvrmgr  can also work in batch mode. That means you can connect only once and get out only after all the commands were executed. There are several situations when this feature is interesting. One is executing a large amount of commands that you might need to repeat several times. Another is executing some commands in a scheduled task. These are some tips for batch mode:
  • use the /b option whenever possible. This options will required that all commands executed before the current command were executed successfully. Any command that raised an error will cause the program to abort execution. This will help you to generate more errors or putting the system into a invalid state.
  • use the sleep command to put a time between a command and other. Useful if you starting a server or are submitting a lot of new tasks.
  • specify a value for the parameter TaskTag when starting a new task. This text will appear in the list tasks command if you include the TK_TASKTAG column with the configure output command or the show parameter.
  • you can omit the header when connecting with the Siebel Server by using the -q command line option of srvrmgr (also undocumented).

Caveats

There are some caveats using srvrmgr. I'll give some workarounds when possible.

Your settings with output width is not working

If you are fetching information from several servers, you will notice that the output may be larger than the width you set. This seems to happen when one of the servers have larger output for a column than the other servers. If you get output from a single servers you will be able to control that.

Another possibility is to set the delimiter variable to a single character to use as delimiter. Once you do this you will be able to fetch correctly the values from one column to another. In fact, you can totally ignore column width output unless some columns are having their name truncated due being larger than their respective output (for example, columns with Boolean values).

To set the delimiter, you can use the set delimiter command inside the srvrmgr prompt or use the command line option -k for it. Both expects as parameter a single character to be used as the delimiter.

Advanced parameters output configuration is not working

When setting list parameters output, all works. For any reason, the same is not applied to the list advanced parameters command.

You can't set the output of list advance parameters directly with configure. Also, you can't rely on the list parameters to get the same formated output.

This looks like a bug that I saw only with Siebel 8.1. So far, the workaround is not setting the list parameters output anyhow before running list advanced parameters: if you do it, the respective columns that shows when the parameters starts to take effect will be truncated.

set command is not saved to preferences files

If you need to keep set commands to the preferences files, know that save preferences will not help you. Instead, go there and write the command manually. The load preferences will do the expected thing.

The non-documented commands

Despite the fact that you can always use the on-line help command, there are some commands that are not documented there.

list advanced parameters

To see advanced parameters for any one of the available items of list parameters command.

list hidden parameters

To see hidden parameters for for any one of the available items of list parameters command.

List threads id's

When analyzing Siebel component crashes, having the associated threads id's will help you identifying the root cause of the errors.

srvrmgr> list tasks show CC_ALIAS, TK_TASKID, TK_TID, TK_PID

This will generate a list of tasks and include the component alias (CC_ALIAS), the task id (TK_TASKID), the OS thread id (TK_TID), and OS process id (TK_PID).

Note the TK_TID and TK_PID values for the appropriate task id that you have flushed the FDR buffer for. This will help you find the appropriate FDR file (the last part of the file name should map to the TK_PID value), and after decoding the file, will help you identify the entries relevant to the task id you are interested in. Each entry should have a ThreadID value equal to the TK_TID value. This is especially important when considering that a single process may have many threads.

Listing servers id's

The command list server show SBLSRVR_NAME, SV_SRVRID allows you to show the server Id created when the server was installed in the Siebel Enterprise. This information is required when configuring the native load balance of AOM's. The configure command for list server will not show the column SV_SRVRID either.

list procs

This command will show the process associated with the Siebel components with some additional data.

srvrmgr:foobar_srv01> configure list procs
        SV_NAME (31):  Server name
        CC_ALIAS (31):  Component alias
        TK_PID (11):  Task process id
        TK_SISPROC (11):  Task sisproc number
        TK_NUM_NORMAL_TASKS (11):  Number of normal tasks in this process
        TK_NUM_SUBTASK_TASKS (11):  Number of subtask tasks in this process
        TK_NUM_HIDDEN_TASKS (11):  Number of hidden tasks in this process
        PROC_VM_FREE_PAGES (11):  Process virtual memory free pages
        PROC_VM_USED_PAGES (11):  Process virtual memory used pages
        PROC_PH_USED_PAGES (11):  Process physical memory used pages
        TK_IS_PROC_ENABLED (31):  Is the process enabled for tasks
        TK_DISP_RUNSTATE (31):  Process run state
        TK_SOCKETS (11):  Sockets Received

Some sample data:

srvrmgr:foobar_srv01> list procs

SV_NAME       CC_ALIAS                   TK_PID  TK_SISPROC  TK_NUM_NORMAL_TASKS  TK_NUM_SUBTASK_TASKS  TK_NUM_HIDDEN_TASKS  PROC_VM_FREE_PAGES  PROC_VM_USED_PAGES  PROC_PH_USED_PAGES  TK_IS_PROC_ENABLED  TK_DISP_RUNSTATE  TK_SOCKETS
------------  -------------------------  ------  ----------  -------------------  --------------------  -------------------  ------------------  ------------------  ------------------  ------------------  ----------------  ----------
foobar_srv01  CommInboundRcvr            5504    29          0                    0                     35                   947650              100925              40205               True                Running           0
foobar_srv01  ServerMgr                  2153    119         1                    0                     2                    1020680             27895               7538                True                Running           0
foobar_srv01  EAIObjMgr_enu              5394    24          0                    0                     7                    914435              134140              74499               True                Running           0
foobar_srv01  EAIObjMgr_enu              5371    23          0                    0                     7                    930429              118146              60284               True                Running           0
foobar_srv01  EAIObjMgr_enu              5353    22          0                    0                     7                    916442              132133              73079               True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5558    34          0                    0                     7                    749560              299015              183322              True                Running           0
foobar_srv01  ServerMgr                  2910    120         1                    0                     2                    1020804             27771               7523                True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5702    39          0                    0                     7                    714186              334389              216924              True                Running           0
foobar_srv01  SiebSrvr                   5024    1           2                    0                     2                    1039160             9415                6633                True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5571    32          1                    0                     13                   712222              336353              217281              True                Running           0
foobar_srv01  WfRecvMgr                  5184    13          0                    0                     26                   954054              94521               38468               True                Running           0
foobar_srv01  ServerMgr                  15100   121         1                    0                     2                    1020315             28260               7299                True                Running           0
foobar_srv01  WfProcMgr                  5203    15          0                    0                     27                   887391              161184              99693               True                Running           0
foobar_srv01  AdminNotify                5094    7           0                    0                     17                   1035909             12666               2607                True                Running           0
foobar_srv01  SvrTaskPersist             5056    8           1                    0                     3                    961872              86703               40503               True                Running           0
foobar_srv01  SRProc                     5093    6           2                    0                     6                    995037              53538               30798               True                Running           0
foobar_srv01  CommConfigMgr              5487    28          0                    0                     26                   954182              94393               38165               True                Running           0
foobar_srv01  eSalesObjMgr_enu           5433    25          0                    0                     7                    1018825             29750               6389                True                Running           0
foobar_srv01  EAIObjMgr_enu              5313    20          0                    0                     7                    866626              181949              124903              True                Running           0
foobar_srv01  eSalesCMEObjMgr_enu        5724    40          0                    0                     7                    1018834             29741               6451                True                Running           0
foobar_srv01  SCBroker                   5048    4           1                    0                     2                    1039526             9049                2375                True                Running           0
foobar_srv01  ServerMgr                  14885   100         1                    0                     2                    1020803             27772               7788                True                Running           0
foobar_srv01  SRBroker                   5039    2           43                   0                     10                   1007397             41178               8950                True                Running           0
foobar_srv01  EAIObjMgr_enu              5291    19          0                    0                     7                    904051              144524              84758               True                Running           0
foobar_srv01  SCCObjMgr_enu              5227    16          0                    0                     7                    1018808             29767               6381                True                Running           0
foobar_srv01  WfProcBatchMgr             5162    11          0                    0                     26                   954304              94271               38448               True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5656    38          1                    0                     9                    730965              317610              200334              True                Running           0
foobar_srv01  CommInboundProcessor       5530    30          0                    0                     56                   947578              100997              37553               True                Running           0
foobar_srv01  SCBroker                   5040    3           1                    0                     2                    1039423             9152                2376                True                Running           0
foobar_srv01  ServerMgr                  16328   101         1                    0                     2                    1020586             27989               7906                True                Running           0
foobar_srv01  ServerMgr                  7735    45          1                    0                     2                    1020032             28543               8112                True                Running           0
foobar_srv01  FSMSrvr                    5053    5           0                    0                     26                   1033215             15360               4564                True                Running           0
foobar_srv01  CommOutboundMgr            5516    31          0                    0                     56                   943817              104758              38734               True                Running           0
foobar_srv01  AsgnSrvr                   5224    14          0                    0                     26                   964218              84357               57768               True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5670    37          1                    0                     10                   692726              355849              238679              True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  16116   61          0                    0                     7                    745602              302973              193532              True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5619    35          0                    0                     7                    742789              305786              187294              True                Running           0
foobar_srv01  ServerMgr                  458     118         1                    0                     2                    1020686             27889               7537                True                Running           0
foobar_srv01  eProdCfgObjMgr_enu         5462    27          0                    0                     8                    957354              91221               38498               True                Running           0
foobar_srv01  XMLPReportServer           5245    18          0                    0                     26                   956796              91779               37351               True                Running           0
foobar_srv01  eCustomerObjMgr_enu        5446    26          0                    0                     7                    1018852             29723               6423                True                Running           0
foobar_srv01  eChannelObjMgr_enu         5256    17          0                    0                     7                    1018732             29843               6356                True                Running           0
foobar_srv01  EAIObjMgr_enu              5332    21          0                    0                     7                    907893              140682              83965               True                Running           0
foobar_srv01  eCommunicationsObjMgr_enu  5640    36          2                    0                     11                   738753              309822              191577              True                Running           0

As you can see, the presence of PID, component alias and current number of executing tasks are quite useful to have in a single command output. Unfortunately, there is no documentation about the other details.

The command can be used with the additional options for server and for comp as well. For example, to monitor when a server has completely started up, you can either run list procs for server <servername> command and verify all processes are in running state.

Tuesday, March 18, 2014

Load test Siebel 8.1 with Jmeter

The problem

Last year I was asked to help with an issue regarding Siebel File System in the production environment of Siebel 8.1. The environment was installed within a Red Hat Linux, and the Siebel File System was showing issues of locking when the system started receiving a higher volume of requisitions than the normal.
In short, after help of Red Hat administrators, we had identified that the read/write of Siebel Preferences Files (*.spf) were generating a great number of locking that were not being released, provoking the related Siebel Component to stop responding new requests or even crashing. During some crises, it was even necessary to restart the Siebel Server to get things working again.
During initial setup of all environments, initial configuration went with NFS as choice for the Siebel File System and soon it showed poor performance. Due time constraints, it was chosen to go with the Red Hat GFS file system, which initially shown better performance. Of course, since Oracle didn't support GFS about that time, it was not clear that this was a bad design decision or not.

The proposal

It was clear too that we should go back with NFS since it is supported by Siebel on Linux. The problem was, of course, it was having poor performance: in fact the default configuration was so poor that is was not acceptable.
The group decided to give another try to NFS, but we would need to be sure that NFS wouldn't show the same problem. We need a way to test and guarantee that and a load test/stress test was the way to go. Now the problem was to simulate usage of Siebel Preferences Files in the Siebel File System.

Enter Jmeter

Since the company didn't have any tool to simulate web clients, I suggested to go with Jmeter, and the group accepted the proposal.
I started using Jmeter's proxy feature to see how the HTTP requests were send and received from a regular Siebel Web Client in High Interactive mode. We assumed that only doing a log in/log out of the application would be enough to force the Siebel Component to read the spf file, but I was also aware that the result could be messy considering that the High Interactive client is a complicated mix of HTTP, HTML, ActiveX and Java applets.
As you can see below, I was not that wrong:


And that was for only requesting the HTML form, entering with user and password and submitting it. After that, a log out request was made. At total, 20 requests for a simple log in the application.
The major issue was understanding the parameters and their meaning. Some of then were pretty cryptographic:


Since I could not guess their meaning and what should I use as a value for each one of them, creating this test required a series of try-and-error sessions, including monitoring the Siebel Server to see if the log in/log out was actually working as expected.
Once this was successfully created, we also decided that only reading the spf files was not enough: we should force it to be changed. I turned on the proxy feature of Jmeter and tried again.
An easy thing to do was to go to a list applet and swapping positions of two list columns in the Siebel Web Client. Implementing that in Jmeter was even more painful, because the amount of parameters were higher and number of requests too. In total, it was 30 request just to swapping the columns of position after log in and log out of the application! That required even more try-and-error tests, including checking for modification in the related spf and in the application itself.

Lessons learned

Well, Jmeter is a great tool for automating load test/stress test of web applications (and other things too), but for Siebel I would not recommended it.
When I saw that it would be even harder to implement as I initially thought, I opened a ticket for Oracle support asking for documentation about those parameters. Unfortunately, the answer was that those parameters were not documented for end user utilization.
After some researching, I found this article from Alex Amat that shows clearly why is so hard to simulate a Siebel High Interactive client by simply looking at the HTTP request exchanged. The secret for that seems to be the correlation library (represented by the ssdtcorr.dll in Windows), which enables you to simulate the HI client without doing yourself a brain damage. While this also helps with the matter of not having documented the usage of each HTTP parameter it's open to debate and I honestly don't have a opinion about that because I haven't tested the application either (Oracle has a product called Oracle Application Testing Suite which hopefully makes such tasks much easier).
Maybe when using Jmeter with Open UI will be easier to write such tests, but again, this is something to checkin the future but I would not bet on that.

And what about the tests results?

Fortunately, the tests went OK. After reproducing the issue on GFS, the team started setting up the NFSv4 in test environment and a lot of settings were tried on the Red Hat side. The OS specialists were able to find good settings and performance with NFSv4 was not an issue again. Of course, the Jmeter's tests were there to confirm that the same locking issue would not happen with the new Siebel File System.

Saturday, February 8, 2014

REST::Client and invalid TLS certificates

A couple of days ago I had the need to consume data from a REST application to complete a test automation job that I was creating.
While I could do it by simple using curl in a shell, the response was also in JSON, which I would need to process anyway, so I decided to do it with a Perl script using REST::Client and JSON::XS.
Since I was working in a development environment, the TLS certificates were self-signed and thus, not valid.
Looking for an option with REST::Client Pod I was disappointed to find nothing. That should be an easy enough thing to do.
REST::Client is built over LWP::UserAgent, so after some researching I found an option with it that does the job:

my $client = REST::Client->new();
$client->getUseragent()->ssl_opts( verify_hostname => 0 ); 


Simple enough. Also a shame to find nothing about it in Google (well, at least until now :-) ).