Thursday, September 28, 2017

Getting Ubuntu, Python 2.7, Jedi and Vim playing nice to each other

This took me some hours to figure out: how to finally set Jedi with Vim to have a good editor for Python 2.7 code.

Ubuntu 16.04 by default install Vim compiled against Python 3.5 interpreter, which is not immediate obvious but becomes a pain in the ass after you discover that even after installing Jedi and python-jedi, you're still getting the error message:

Please install Jedi if you want to use jedi-vim.

When opening a file with Python code.

To get things working for your Python 2.7 code, you might need to uninstall your current Vim related packages and install the following:
  • vim-addon-manager
  • vim-common
  • vim-gnome-py2
  • vim-gtk3-py2
  • vim-gui-common
  • vim-nox-py2
  • vim-python-jedi
  • vim-runtime
You can check your current configuration with:

$ dpkg-query -l 'vim*'

Thursday, July 6, 2017

Moving away from Dist::Zilla @Basic

Today a got an issue registered at my distribution Linux::Info and decided to take a quickly look at it since it was quite a while since the last release.

The issue itself was not that interesting (looks like the distribution Pod could receive some more attention, which indeed I added), but also gave me a chance to try and add a cpanfile to it.

Some months ago I was notified that my distributions were not providing a cpanfile. I wasn't even aware what was it about and what I was missing.

The thing is that a cpanfile is just another way to declare the distribution dependencies. Not really interesting from this point of view.

The good part of it is to allow the developer to install the distribution without having to install Dist::Zilla (and it's quite large number of dependencies) before even being able to build the distribution (well, assuming that the distribution uses Dist::Zilla).

That means you can skip some minutes and download bandwitdh, ignoring whatever Dist::Zilla depends on. Which also means you can just git clone a Perl project and install the requisites automatically with cpanm with two steps:

  1. git clone
  2. cpanm --installdeps .
That's it! Well, almost, you might need to install cpanm first. And be sure to include the dot in the step 2, it matters.

If your distribution uses Dist::Zilla already, you can add the cpanfile automatically by just adding [CpanFile] to the dist.ini file. The plug-in does the rest for you. Almost.

In order to have this cool stuff available from your code repository, you will need to have the cpanfile available over there... that means you need to build the distribution and copy the file to the repository, where it is going to be source controlled as everything else.

To cover that, you can use Dist::Zilla::Plugin::CopyFilesFromBuild, another plug-in to make your life easier.

But if you have the [@Basic] declared at the dist.ini (as I did) to load the most basic Dist::Zilla plug-ins you need, you will have a nasty surprise:

[DZ] attempt to add cpanfile multiple times; added by: GatherDir (Dist::Zilla::Plugin::GatherDir line 100); CPANFile (Dist::Zilla::Plugin::CPANFile line 65)

And the distribution generation is aborted. I was able to even find a bug registered about that, but it is not really a bug after all: it happens because the GatherDir, automatically registered by [@Basic], is not configured to ignore the cpanfile over there.

I tried to to that, but instead of getting rid of the error, I got a lot more of them, all of them related to GatherDir. Don't know why, but I was able to fix by removing [@Basic] and putting only the plug-ins I was indeed using and the over automatically included by the bundle and adding the following two lines to configured GatherDir:

[GatherDir]
exclude_filename = cpanfile

So, fixing the errors were a nice side effect of review what I indeed need in my distribution in terms of plug-ins.

Bottom line: Dist::Zilla::PluginBundle::Basic is great to start from scratch, but keep in mind that once your dist.ini is not that "basic" anymore, you will need to start cutting-off plug-ins!

Wednesday, April 19, 2017

Stop pestering me Yahoo!

I'm a user from Yahoo! from a long time ago already... something between 12 to 15 years.

For the most of the time, I can tell you I was a happy user... Yahoo! web interface was modern and I used it too with POP3 (later moved to IMAP of course) and SMTP with different e-mail clients.

From the last few years, well... we all know things are not good with Yahoo! anymore... Google took from it a good amount of the payed advertisement with it's (much more powerful) search engine. But from the past few months, the e-mail system is becoming a pain in the ass.

First, the SPAM filter doesn't work well. And I can't disabled it. I tried to refine it with some training but it seems it will be forever stupid because it keeps putting e-mails from mailing list that I'm subscribed for years in the SPAM folder.

Second, Yahoo! thinks my e-mail client is not secure anymore. Now and then, when I got check my e-mail on Thunderbird, I got a error message stating that the authentication process failed and asks me to try a different password. There is no password change, it is Yahoo! pestering me to stop using a e-mail client (yes, I tested with Claws too).

I didn't change my password, so let's go back to the web interface and see what is going on.


OK, it asks me my same old user and password. And this "nice" Captcha. And here is the reason for not being able to use Thunderbird:



For non Brazilian Portuguese speakers, it says:
"Alceu... disable the less secure access to your Inbox"
"Disable the applications that are not from Yahoo and that use methods of entering less secure to access the Yahoo Mail. If you don't do that, your account will be vulnerable"
Well... that's just bullshit. I use the same authentication method (Yahoo! doesn't asks me for a Captcha unless I got blocked first on my e-mail client) with Thunderbird, and also the connection uses TLS the same way that my browser uses.

Let's also add that I access it from my home network, that is behind a router/firewall, using a cabled connection (not WiFi) and that my PC uses a firewall as well.

What is considered secure as well? Their mobile application... that I should install on a Android, which I have little to no control (unless I got root on it) of how it works and the software that is installed and have to use WiFi or the shitty 4G available on Brazil.

Yeah, it doesn't look very secure too me so... STOP PESTERING ME Yahoo!

How to disable all logging from a Siebel component

You can change parameter to TRUE from the UI:

Here is default output from srvrmgr program:

srvrmgr:ssia811> list hidden param BypassHandler for comp EAIObjMgr_enu

PA_ALIAS PA_VALUE PA_DATATYPE PA_SCOPE PA_SUBSYSTEM PA_SETLEVEL PA_DISP_SETLEVEL PA_EFF_NEXT_TASK PA_EFF_CMP_RSTRT PA_EFF_SRVR_RSTRT PA_REQ_COMP_RCFG PA_NAME

------------- -------- ----------- --------- ------------- ------------- ---------------- ---------------- ---------------- ----------------- ---------------- ----------
-------------------
BypassHandler False Boolean Subsystem Event Logging Default value Default value N N N N Do not send events to logging

1 row returned.

Here is the output after I changed from UI:

srvrmgr:ssia811> list hidden param BypassHandler for comp EAIObjMgr_enu

PA_ALIAS PA_VALUE PA_DATATYPE PA_SCOPE PA_SUBSYSTEM PA_SETLEVEL PA_DISP_SETLEVEL PA_EFF_NEXT_TASK PA_EFF_CMP_RSTRT PA_EFF_SRVR_RSTRT PA_REQ_COMP_RCFG PA_NAME

------------- -------- ----------- --------- ------------- ----------- ------------------- ---------------- ---------------- ----------------- ---------------- ---------
--------------------
BypassHandler True Boolean Subsystem Event Logging Comp level Component level set N N N N Do not send events to logging
1 row returned.

srvrmgr:ssia811>

FAQ


  • This is will disable logging completely? Or the component log file will be created empty?

Setting BypassHandler to true will disable logging on component level there should be no logs.
  • This parameter can be enabled to other components? What are the restrictions?

You can enable same parameter for another components – no restrictions.
  • Is the output from srvrmgr "list hidden param" a bug?

It is not a bug, work this way as designed.

Sunday, March 26, 2017

Playing around with Vagrant

I finally finished preparing a Virtualbox image for OpenBSD to execute the CPAN::Reporter::Smoker, all provisioned by Vagrant.

Most of the provisioning was implemented with shell script (Bash) and honestly took me much more time I initially thought it would take.

Although I had almost all configuration steps already documented, the development cycle was long, specially because I had to wait for perl compilation as well all the required Perl modules. Disabling tests when provisioning did reduce the time (for the trade off regarding stability) , but that is still a long way to go.

Having to create a local mirror (CPAN::Mini) of CPAN inside the VM was not fun either: after finishing my first Vagrant box, I realized that shipping the VM with the mirror pre-initialized was not a good idea: the box ended with 3.2GB.

Worst... the mirror on the box might get considerable outdated pretty quickly.

Even though I was able to reduce the total size of the mirror in about 1Gb, it is still a lot, even to download. To my surprise, Vagrant doesn't seem to be able to resume downloads at all, which surprised me. This post confirms that too. It turns out that downloading boxes with wget on internet connections that sucks (like mine) is a better idea... At least I can download the box and still keep watching Netflix by using --limit-rate with it.

Lessons learned, I think I'll just cut out the pre-initialized CPAN mirror. At least this should give people a opportunity to maybe reuse their own CPAN on their local networks, although I do think that having a independent VM running the mirror is not a bad idea... disk is cheap anyway these days.

Next step should take Collectd installed on the OpenBSD VM... and try to figure out the best set of CPUs to be configured on it.

Saturday, December 3, 2016

Poor's man test notification

Somewhere 2014, my manager asked me to write down test cases for the application the team was developing. He didn't care the programming language I was going to use as long as I could implement it quickly.

Those tests should run regularly, and an e-mail should be sent in the event things go south.

Being used to write tests with TAP (Test::More), I went with Perl and Linux crontab for the job. Besides the tests itself (not really interesting for this post anyway), I end up with a generic script to run over the tests, analyze the results and send an e-mail with a nice HTML report attached. Here is the script:

use warnings;
use strict;
use TAP::Harness 3.30;
use TAP::Formatter::HTML 0.11;
use Email::Stuffer 0.009;
use Email::Sender::Transport::SMTP 1.300010;
use MIME::Base64 3.14;
use Authen::SASL 2.16;
use Config::Tiny 2.20;
use lib './lib';
my @tests = glob('Tests/*.t');


foreach my $test (@tests) {
   my $output_filename = 'results.html';
   my $fmt = TAP::Formatter::HTML->new();
   $fmt->output_file($output_filename);
   $fmt->verbosity(-2);
   my $harness = TAP::Harness->new(
       {
           formatter => $fmt,
           merge     => 1,
           lib       => ['lib']
       }
   );
   my $aggregator = $harness->runtests($test);


   if ( $aggregator->has_problems ) {
       send_email( $test, $output_filename );
   }
   else {
       remove_report($output_filename);
   }


}


sub remove_report {
   my $report_filename = shift;
   unlink $report_filename
     or warn "Could not remove unused report file $report_filename: $!";
}


sub send_email {
   my ($test_name, $attachment) = @_;
   my $cfg = Config::Tiny->read('xyz.ini');
   my $body = <<BLOCK;


Greetings,


Sadly a test ($test_name) executed against XYZ REST API failed.
Please check the attachment for details.


BLOCK


   my $transport = Email::Sender::Transport::SMTP->new(
       {
           host          => $cfg->{SMTP}->{'smtp.host'},
           ssl           => 1,
           sasl_username => $cfg->{SMTP}->{'smtp.username'},
           sasl_password => $cfg->{SMTP}->{'smtp.password'},
       }
   );


   Email::Stuffer->from( $cfg->{SMTP}->{'smtp.default.from.email'} )
     ->to('afreitas@foobar.com')->text_body($body)->transport($transport)
     ->subject('XYZ REST API failed')->attach_file($attachment)->send;
   remove_report($attachment);
}

Being generic enough, I could just point the script to different folders containing the tests and hopefully got nothing because all tests passed. In fact, it was so simple I wondered why something like this was not available on CPAN.

Well, not anymore... Test::Health is available for download already, so you can install it and run the script health_check.pl to run your tests. Since this script was created initially to health check the application, the name makes sense... but it seems to be generic enough to be reused in different situations that I didn't foresee when I wrote it.

The only thing that is missing from my original script is the SMTP authentication... hopefully I should implement subclasses to make it in the future (but contributions in that sense are welcome!).

Happy testing!


Tuesday, October 4, 2016

OpenBSD CPAN smoker: mission accomplished

I finally reached my personal set milestone regarding configuring and running a CPAN smoker on OpenBSD OS and grabbed me a honorable third place in number of submissions:



That was quite a ride! Despite my short experience with OpenBSD, I naively though it wouldn't be that hard... well, it was hard enough to generate some debate on CPAN Testers discussion list because  looks like nobody is doing much test on OpenBSD (or at least is not submitting results about it) nowadays.

All things considered, I decided to go a little further, so I grabbed all my lessons learned and created a document about the process. This work result is available at http://wiki.cpantesters.org/wiki/SmokerOnOpenBSD.