Showing posts with label Dancer2. Show all posts
Showing posts with label Dancer2. Show all posts

Saturday, 4 July 2015

How to become a Strehler developer

Strehler is a strange package to develope. It's easy to install from repository and use, but while you are just writing it you have to arrange many tricks to be confortable with the work. For example, Strehler needs a Dancer2 site to run, something the respository doesn't contain.

In this little tutorial I'll try to explain how to setup an environment to become a Strehler developer, because Strehler needs you, as any of the packages published on CPAN. It's a very basic tutorial, useful for beginners too.

I hope it will help.

The repository

You probably already have a sandbox created in one of the many ways available for perl development. I'm still a fan of the good'ol local::lib but it's good to use also, for example, perlbrew. Your sandbox has to be active during Strehler development and you have to put in it Dancer2
cpanm --notest Dancer2
But NOT Strehler. After installing Dancer2 most of the library useful to work with Strehler will be in place, but probably you'll need something more, libraries specific to Strehler work. Just install them in your sandbox the same way using cpanm when running the code point out their absence..

You'll surely need:

All this packages are in the requirements of Strehler package and would come automatically installing Strehler for CPAN. Installing them manually is the cleanest way to obtain them, but you can try some trick, like installing Strehler and then deleting it from repository, keeping all its dependencies.

The code

Just pull down all the source from the git repository in a directory you like
git pull git@github.com:cym0n/strehler.git 
Downloading the repository you'll have a directory named $GITURL/src/lib and one named $GITURL/src/script. These two directories will come useful soon.

The environment

Now that you have Strehler code it's easy to modify it and manage changes using git. Less obvious how to use this code and see it working. We just need to customize a little the environment. Open a file named strehler-devel and put in it
alias perl="perl -I$GITURL/src/lib"
alias strehler="perl -I$GITURL/src/lib $GITURL/src/script/strehler"
When the file is ok source it.
source strehler-devel

Testing site

All Strehler power is now available, coming directly from the git repository on your machine. Now you can easly create a testing site with Strehler onboard using strehler script itself.
strehler demo 
Just run it as
perl bin/app.pl 
to see Strehler and (eventually) your modifications in action.


Monday, 8 December 2014

EX power

So this weekend Strehler received a major release and two minor bugfix releases and it's now at 1.4.2.

When I completed Strehler backend I was so excited about how easy was to install it and start writing contents for a site. Then, I started working on a real frontend and I found doing that a very boring work, with a lot of copy-and-paste and clumsy writing.

Creating Strehler I was obsessed by the opportunity to make it highly configurable because in my life I always stop loving packages like it when I found what they decided by themself, without asking my opinion. Most of Strehler has a parameter that control it, side effect of this is that you have to write a lot to make the whole thing having the right place.

High level refactoring of this exists. What about taking all the boring code I wrote for fulfill simple tasks and "hide" it behind Dancer2 keyword? The EX plugin was born.

The EX plugin is a collection of keywords for standard cases. In particular they can be used to serve a content called using its slug, to display a list of contents of a category or just to manage a page with contents that you write once and then you throw away (like homepage boxes).

EX plugin keywords can still be a little complex to understand, because they need obviously nested hashes to work well, but at least you have just to understand the meaning of their parameters and not perl code to use them and documentation can help you doing this.

It's also a point of my todo page so i'm really proud of it!

I'm less proud about the two minor releases. The first was done because I wantend more options and configurable things in EX plugin plus a smarter way to write code. The second was done because I did the first too fast :-)

Saturday, 15 November 2014

A matter of time

Strehler 1.3.0 is released. This means that all the problems with Dancer2 0.15x are solved. More than that, requirement about Dancer2 moved up from Dancer2 0.11 to Dancer 0.15002.

In the previous episodes we saw that Dancer2 App, from version 0.15, try to find their configuration file in a directory near the one they are installed, ignoring where is the script that uses them. This was very bad for Strehler becase Strehler apps like Strehler::Admin are installed using CPAN so they are in the local repository, a place where no configuration file can be stored.

Only available solution (for now) is overriding thi behaviour setting environment variable DANCER_CONFDIR to the path where our config.yml actually is. Can we do it inside a script so that we can use this solution in any context, like CPAN tests? The answer is yes, if you know the right things about perl.

First of all, any perl script can access environment variables through the %ENV hash,  a read/write variable always available. So, first step should be to write in your code:

$ENV{DANCER_CONFDIR} = /the/directory/I/want

Now you can think that, including my app, all will work, so you're thinking about something like this:

$ENV{DANCER_CONFDIR} = /the/directory/I/want
use Strehler::Admin

Don't run too fast, this code WILL NOT work.

There's a lot of interesting concepts about perl interpreter behind this problem. Concepts, I hope, one day you'll be curious about. For now it's enough to say that "use" directive will run before DANCER_CONFDIR assignment even though you write it in the end of your file.

 Modules must be loaded before any other piece of code because consistency of any piece of code can be verified only with all the module loaded. Perl interpreter does it without asking nothing, ignoring that you necessary need to do something before.

Can you change this behaviour? Yes, two ways to do it:

  • BEGIN directive will run all the code inside its block before any other code, including use.
BEGIN
{
   $ENV{DANCER_CONFDIR} = /the/directory/I/want
}
use Strehler::Admin;

  • require directive instead of use load the module only when someone ask for it,with no precedence on anything.

$ENV{DANCER_CONFDIR} = /the/directory/I/want
require Strehler::Admin;

You can see an example of this here.

Here are described many ways to configure DANCER_CONFDIR.

Why don't I want to consider this as the definitive solution for the problem? Because I think it's still a dirty way to do things because force you to use an environment variable and disable the original App behaviour that allow each of them to have a separated config file, something that could come useful in some situations.

Sunday, 9 November 2014

Plack::Test and Dancer Session

PSGI is very easy to understand: a black box, something goes in, something goes out. What goes in: a request made of a path and headers. What goes out: a response with a content and more headers.

Plack::Test follows this structure. What you have to test is the black box, you inject the request in it and analyze the response you receive back.

Important thing to understand is that Plack::Test is something very different from an emulation of a browser like LWP::UserAgent. Most important difference is, obviously, in session management, because that's something that should survive between calls and in Plack::test every call is a test by itself, with no memory of what happens around it.

What do you have to do  if you want to test something like a login using Plack::Test? Well, the solution is very easy because the concept of session is AN ILLUSION throughout all the web. It's not just Plack::Test, the HTTP protocol itself is session-less. The "session" you see is just a set of tricks that clients and servers do under the hood.

As we said: a request go, a response returns, nothing more, so the session is something that must be hidden there. In particular, session is controlled throught headers, in particular headers that control browser cookies.

There're a lot of documentation about how cookies and response headers have to interact and probably studying it could be very interesting, but HTTP::Cookies module contains all the knowledge you need so learn to use it will be enough.

Let's do a login and get the response from it:

$r = $cb->( POST '/admin/login', [ user => 'admin', password => 'admin' ] );

Response object $r contain many headers, one of that ask the client to write in a cookie the identifier of dancer session. We can use HTTP::Cookies to extract this information.

my $jar = HTTP::Cookies->new;
$jar->extract_cookies($r);

Now we want that, in the next call, the server find us logged in.

When a browser does a request to a site, it passes in the request all the informations related to the cookies it has. This way it can say to the server what is its dances session identifier. Using that, the server can retrieve any information about the caller, like its status as logged user.

Let's cook a call for admin homepage.

my $req = HTTP::Request->new(GET => $site . '/admin');

WARNING: Remember to ALWAYS create request with complete URL, starting from http, otherwise HTTP::Cookies method add_cookie_header will not work!

Now we want that this call carry information about the cookies of the previous call, where logged in user session was configured. We just have to get information out from the cookie jar.

$jar->add_cookie_header($req);

Now this req will been seen by the server as the logged user from the previous call did it.

Well, if you configured in a clever way the server too.

Remember? There's no memory between requests client-side AS server-side. Client memory is in the cookies and HTTP::Cookies helps us to manage it. Where is server memory?

Standard session management in Dancer leave session information in RAM. On a development server this is enough, but a development server is something that stay alive between different calls so you can consider information stored in RAM persistent. Plack::Test instantiate the black box fresh new at every call so information about sessions is always trashed at the end of it.

Harddisk is more reliable if you want to store persisten information so switch session management to YAML when doing this kind of test.

Here is a little example of what we said.



Tuesday, 4 November 2014

The darkest hour

I've been away from this place for long time, I know, and I also know that also Strehler has been left with no updates for many months. There're technical questions about all this silence and I want to explain them here.

This summer Dancer2 reached release 0.150000, quite an improvement! A lot of things changed with this release of the framework, most of them related to the concept of App.

Major architectural changes are something I know I have to deal with, talking about Strehler's developement. Dancer2 code is not stable, it's usually considered alpha and I know that chasing its releases is one of the most difficult but important tasks for me as mantainer of a module based on it.

Problem is 0.150000 Dancer2 was too much. I described the issue introduced by this release here, Briefly, an App installed in your local repository gives a lot of trouble when it's time to read configuration files, because App uses its own path to find site resources, ignoring the path of the script that manage site startup.

For now, in my opinion, there's no way to solve the question fixing Strehler. Problem is inside Dancer2 architecture so I hope that the Dancer2 development team will do something. Waiting for it, this is what I can say if you want to use Strehler:


  • Strehler latest release is perfectly working with any Dancer2 0.14x release. So, if you don't update Dancer2 packages too much you have nothig to fear.
  • You can make Strehler works with Dancer2 0.15. You just have to ignore all the tests and run your site setting the environmental variable DANCER_CONFDIR as the directory where your config file is. It's quite simple and I know I could consider it the definitive fallback, but it's not clean and still I don't know how to use it to run the test suit, so I take this as a temporary workaround.
Well, I don't have much more to say, this is the situation at today. I hope it will change soon and I think it will, but patience, for a bit, has to be your new best friend.

Last announcement: Strehler has now a site. Click here for it. Important articles from this blog will be reported there, as a place to aggregate every communication about the development.

Friday, 2 May 2014

All the releases come with a bug

So I released Strehler 1.1.7. Now the filter is improved and you can select a category to obtain all the elements in it and in its children.

The real reason I did it, however, is that 1.1.6 has a terrible bug that break the strehler initdb command making impossible a complete Strehler installation as described in the documentation. Strehler installation process is something that MUST go well plain and fast or I judge the entire package useless, because I created it to make a CMS deployment easy.

But... now I found a little problem on strehler statics when it comes to use it to update your statics after an update of the package. Less vital than the solved bug on initdb, but still annoying, probably e new release will be done soon.
At the moment, if you update Strehler package from a version to another (for example from 1.1.6 to 1.1.7) to make all works delete the public/strehler directory in your project and then run strehler statics again. Al the resources will be regenerated in the right version.

Also the Dancer2 0.14000 release gave me some work. As described in the release notes, Runner-Server was untangled. I like the explanation about why this was done, my problem is that all my tests were written using Test::TCP relying on the Dancer2 Standalone Server for the server part. With Dancer2 0.140000 al this part went broken.
So I managed to change this line of code:

Dancer2->runner->server->port($port);
with this if clause

          if( $Dancer2::VERSION < 0.14 )
         {        Dancer2->runner->server->port($port);    }
         else
         {        Dancer2->runner->{'port'} = $port; }

just to keep retrocompatibility.

Thursday, 1 May 2014

Strehler: state of the art

I know that i'm not a frequent writer so a lot of interesting things about me doesn't appear on this place because i'm lazy. So I decided to write just a post to summing up how Strehler is going on after the 1.0.0 release.

Well, a lot of things changed and got better since that day. The first, fundamental step, with 1.1.0 was landing on CPAN with a real test suite and some supporting script. Now you can install Strehler using just cpan -i Strehler and getting an up and running CMS with few commands. In many ways the 1.1.0 release of Strehler is the real first release, because I know that the long, boring and messy installation procedure of 1.0.0 is something NOONE would use, out of me, obviously, but I'm the father of Strehler and my judge is irrilevant.

Strehler 1.1.0 was my first package released on CPAN so I learned a lot releasing it. I learned about PAUSE, about distzilla and about how many thing to your code after you upload it on the perl repository. I did a lot of things wrong because mistakes are the way nature teach you things, so in the week after 1.1.0 release I released FIVE bugfix packeges bringing rapidly Strehler to 1.1.5 version. Well, it's a good thing, high numbers make your software truly professional.

With a quite stable version of my software on the public repository, I started work about little features not yet implemented, like function to order admin list and searchbox. All these things went in the 1.1.6 version and i'm quite proud of it. Yes, a lot of work could be done on the search box because it is implemented in the easiest and plainest way I found, with no indexing or optimizations. It's good because it's accessible just from the backend of your site, but it's probably not a good implementation if you want it also on the frontend.  Probably in the future I'll do something, pluggable if I can, about it.

I'll probably release soon a 1.1.7 version. I wanted to wait after some implementations, but I found a big bug in the strehler script that make impossible to create a strehler database on your custom site, making all the installation process useless. I fixed it and I want to share it with the world as soon as possible.

After that, a lot of adventures are waiting for us!


Wednesday, 26 February 2014

Habemus Strehler!

Well, i worked a lot about it behind the scene, so i'm now a little excited to talk about Strehler, my most complex Perl work.

Link to github first: https://github.com/cym0n/strehler

As the README says, Strehler is a light-weight, nerdy, smart CMS in perl based on Perl Dancer2 framework. The story of his creation is very simple: I had to create a website (well, more than one) and after I decided to use Dancer2 I also decided to write the backend interface for it once for all, developing a tool easy to install on any type of webapp you need and with a lot of customizations available out of the box.

Strehler implements just the standard CRUD needed for a standard site. It can create articles, upload images, organize things  with tags and categories. I said more about it in the github wiki.

In my opinion the software reached enough stability to deserve version 1.0.0 tag. My hope now is for people to download it, try it and tell me where I'm wrong thinking this. At this point of development Strehler needs to be seen by fresh eyes, this is the reason i'm announcing that with so much emphasis.

You can try Strehler, you can use it, you can just find something you don't like and tell me. If you like collaborative work or if you think that Strehler can be what you need with some modifications, just work on it. Code is open and the git repository is there to be used. If you need some advice about how to improve the tool, try the TODO page. It's there for me as for everyone ho like to develope using Perl.

Strehler is ready! Cheers!

Saturday, 22 February 2014

Entering the time zone

It' a lot of time I don't write here, i'm working on something that will make me write a lot (I hope) but i'm keeping silence until it is finished. However, today I had to understand timezones more that how much I did before so I want to throw all the knowledge I acquired in this blog, to make it remeber it instead of me.

Problem of the day is common, but not simple. You have a database with a timestamp field, you want to print it on the screen. It seems easy, a timestamp is just a date and an hour, what could go wrong? Easy, the hour you print IS NOT the hour you expected.

We are a lot of people who lives in a lot of places. Sun can't shine on all these places at the same time. But everyone wants to wake up in the morning and go to bed in the evening so: timezones are born.

We assume that you have a MySQL database, that you will read the timestamp column using DBIx::Class and then that a Dancer app will make it visibile on the screen. Every step has its timezone issue.

Timezone Steps

MySQL

MySQL knows its timezone and every time it gives you a timestamp the timestamp is considered in that timezone.
MySQL timezone is the default_time_zone option in my.cnf. If no timezone is provided, the server will use the one of the machine where it it hosted.

DBIx::Class

DBIx::Class has a lot of tools to manage timestamps. Using InflateColumn::DateTime is nearly a standard. Problem is DBIx::Class can't ask MySQL what is its timezone so you have to configure it when you write the model. Your timestamp field will be:

  "timestamp",
  {
      data_type => "timestamp",
      datetime_undef_if_invalid => 1,
      default_value => \"current_timestamp",
      timezone => "Europe/Rome",
      is_nullable => 0,
  }

Dancer

Dancer is not really involved in this question. Every time you want to print a timestamp you'll go through this code. But this could be used when Dancer has to give you an output on the screen. Here you can decide once more what is the timezone to use. 

Few lines:
$ts->set_time_zone("Europe/Rome");


It's a trap!

What could go wrong? 
If timezone is not configured in the DBIx::Class you will have, as output, the hour in the timezone of MySQL, but with a "floating" timezone. This means that conversion will be... well... erratic.
If timezone configured in DBIx::Class is different from the timezone in the MySQL, MySQL timezone will be ignored although the time you receive is made from it. DBIx::Class timezone will be attached to timestamp instead. No automatic conversion will be triggered and you'll have to work just with wrong data.

How to deal with this

Problem I see with this situation is that MySQL is part of the environment while the rest is part of your code. You write the code, but the environment can change. I think that a good idea is always configure the timezone when you output the data ("Dancer" level) so you have a good point to change it if it's needed, without touching the structure below. Than be always sure of your MySQL configuration when you write DBIx::Classes.

Saturday, 14 December 2013

Ballroom Blitz (2)

Silence in the library

When you write a perl program you don't need just the perl interpeter, but also a lot of libraries that someone else wrote. It's easy to fetch them while you're in your developement environment, but when you go to production you have to manage a lot of issues about this problem.

First of all, there's the scenario where you don't have root access for the system where you are so libraries installation appear to be an impossible task. 
A worse scenario is the one where you have root access and you install the libraries directly in the system.
The worst case scenario is the one where you're installing things in the system, the system die and someone behind you start hitting you with a club. No, the problem is not the one with the club, the problem is you. The club is the solution.

Never do installations at system level. System level is just for... the system, the libraries installed that are the libraries need by your OS, not the libraries you need for your site. In many years of developement and deployment  people found many ways to do things better.

The article I linked last time, for this task, uses perlbrew, one of the best practices for that. Perlbrew manages not only libraries, but also perl version, so it can also be used to ensure that the bizarre incompatibilty between 5.16.0 and the cutting edge 5.18.0 is not an issue because you will configure on your machine just the version that works.

But timtowtdi so in this article I'll speak about probably the oldest practice about managing libraries, the oldest but still useful, clean and sharp: local::lib.

local::lib is a library that you have to install at system level:

(as root) cpan -i local::lib

 after that you'll have the power of playing with perl libraries just moving directories. Remember the directory where we decided to place our site, /var/www/com/example/my/?
Create under this directory a directory env and then a file named perl_sandbox with these lines:
libdir=/var/www/com/example/my/env
export PS1='\[\e[1;35m\]\u@\h:\w\$\[\e[0m\] '
eval $(perl -I$libdir -Mlocal::lib=$libdir)
Now source the file:

source perl_sandbox

This way your shell will become pink (the export PS1 line) and all the operations you'll do about perl libraries will have effect just on the env directory. However, with the perl_sandbox active, every perl command you'll use will see the libraries in the env directory.

Just a last tip: remember to install cpanm

cpan -i App::cpanminus

and use that for installations, with --notest option. Life is too short to let cpan to check all the repository every time you need an environment.


Up to the plack

Plack is a server used to run PSGI applications. PSGI is a interface that we can use to communicate with the webserver that will bring our site on the World Wide Web, so Plack is something we really need to make things work.

Talking about this issue, working with Dancer makes thing really easy because the standard bin/app.pl script that run your webapp can be used as is as a PSGI startup file, as the Dancer2 Deployment page says.

bin/app.pl script must run under a perl web server supporting PSGI. My favourite for this is starman (yes, I like David Bowie, but it's not just for that).
Copy&Paste from Dancer2 Deployment page how to run plackup and make your site works, however, is not enough for something that will need maintenance in the future.

The last piece of the puzzle is Server::Starter, a perl module that provide you the start_server script that will change you plackup server in a daemon (Argh! WItchcraft!), making easy for you stop/start procedures and monitoring.
After installing that module and after creating a /tmp directory under our site directory, you can try THIS copy&paste:

WORKINGDIR=/site/directory
PROJECT=/where/bin/directory/is
source perl_sandbox #as seen in the previous paragraph 
          cd $PROJECT
start_server \
--pid-file=$WORKINGDIR/tmp/starman.pid \
--status-file=$WORKINGDIR/tmp/starman.status \
-- \
plackup -E ec2 -s Starman --workers=2 -l $WORKINGDIR/tmp/plack.sock -a bin/app.pl & 
as your start_server script.
And this as last command:

start_server \
--pid-file=$WORKINGDIR/tmp/starman.pid \
--status-file=$WORKINGDIR/tmp/starman.status \
--restart

For restart.

For the timtowtdi section, take a look at Daemon::Control for an alternative implementation of the daemon. For a PSGI webserver general purpouse, one of the most used technologies of these days is uWSGI.

Gentlemen, start your NGINX

Last step is the easiest one. NGINX is just a proxy from the internet to your plackup structure. NGINX and Plackup will communicate using the plack socket that we configured in the start script: /my/site/dir/tmp/plack.sock. Obviously we're talking about an environment where Plack and uWSGI are on the same machine, otherwise you'll have to sail for the TCP/IP, as everything on the network.
Let's see nginx configuration:


upstream siteplack {
  server unix:/your/site/directory/tmp/plack.sock;
}


server {
  server_name www.yoursite.com;

  access_log /your/site/directory/logs/www.site.it_access.log;
  error_log /your/site/directory/logs/www.site.it_error.log;

  root /your/site/directory/project/src/public;

  location / {
    try_files $uri $uri @proxy;
    access_log off;
    expires max;
  }

  location @proxy {
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://siteplack;
  }
}

Few things to say. The public directory of the Dancer2 project should be server directly, because DancerApp has nothing to do with static things like css and javascripts. All the others things are just to copy and paste.

Restarting NGINX, praying all your gods, if all you did was did well your site will rise.
If not probably it will be my fault writing something wrong in here, but I'll be too far from you to be blamed. Just solve your problems and tell me how to fix the article. Comments are there for this reason.

Saturday, 16 November 2013

Ballroom blitz (1)

I read this interesting article about Dancer2 deployment. I have a lot to learn about Dancer2 deployment so I thought seriously about apply this practice to my sites. However, working on it, I didn't feel so confortable with the techniques applied there, probably because of my habist more than other reasons.
So, at the end, I came to a different solution and I want to describe it here, to clarify it in my head and to demostrate, once more, that timtowtdi.
I will skip the part about the provider subscription, I want to focus just on the technical issues and I don't want to talk about a specific environment. Let us think just about you having a server, anywhere you want, and the Power of Root is in your hands.

Who's the man?


When you're young and stupid you can think that root is enough, because root is powerful, shiny, charming and has a silver armour with gold drawings on it. Uncle Ben died to teach you that with great power comes great responsibility and you're still so stupid to use root and just root, also for pron watching!

Then you grow up and you know that there're three type of users:

  • root: the boss. Period. You don't annoy the boss with stupid deployment issues.
  • human users: they have to log in on your machine and do dirty things. They need a little, walled garden (some guards on the wall, too)
  • application users: they're on the machine just to work. They're strictly linked to the processes they run.
So, you need to start your app deployment and you have to choose a user that will do it. For this issue, after years chasing not-so-writable directories and evoking the sacred chmod sevensevenseven, I chose the easy way: do all using the nginx application user (nginx or www-data, usually). It's not perfect, it's not the cleanest way, it's not the most secure, but it's tidy enough and give you a lot of space to manage things.
From now on, consider yourself logged as nginx. It's home directory will be /var/www.

Where's the man?

Just a small paragraph about directories. If you don't have a strict discipline your system will become a mess. It's its destiny, it's nature. Entropy is part of every job in the universe.
My favourite idea about how to organize different sites on the same machine is to use reverse domain as subdirectories.
So, mylittlepony.evilgenius.com will go under /var/www/com/evilgenius/mylittlepony and fluffy.bunny.org will be /var/www/org/bunny/fluffy.
This way it will be easy to find the site you're looking for and directory names will never be too long. The policy for domain names is also one of the strictest of the computer science, so you'll never have to manage UTF-256 characters or things like that.


Beam me up Scotty!

This article will speak also about the obvious. 
The easiest way to put the code of your webapp on your server is through some versioning system like git. You just have to pick up an account on github or bitbucket and then, after finishing the development, push all you did on a repository. Then, on your webserver, you just have to clone the project and then maintaining that pulling every update you push.
There's a part of me that don't like working this way. The cleanest and most clever way to deploy a software is packing it someway and then installing it on the server. Packing and installing must be done through a clean, sharp and smart suite of scripts tailored on the software you are managing.
This is the old school procedure and as a lot of old school procedure it's still good, but I have to admit that git is easier and perfect for lazy people.
But! Remember! :
  • Think about if you need a private repository. It could be that someone doesn't want that the code of the site is exposed to everyone with a google bar. I know, I know, open is better, open is good, open i what tender kittens wants, but sometimes it's not what YOU want.
  • gitignore is the most important file of your repository. Design it thinking that nothing about your server environment should be in the git repository. For two simple reasons. One: if you have on git something related to an environment it could change the environment configuration on deploy just because on your developement server something is different and making a git pull a disaster. Two: you don't want your production database password written on a public github repository... and probably you don't want it also on a private one, if you don't know all the contributors to your project.
  • Take a branch as the production branch and never touch it before deploy time. When you have to develope a feature branch from production and develope there. On deploy, merge your new branch with production and push. You have to trust in production branch. Every time you go on your server and do "git pull origin production" you have to know exactly what will happen. Nothing, in the most cases, the deploy of something you planned, on the happy release date.
    Again, if you don't trust all your contributors, there're many ways to prohibit people to push on production. Enforce them. Your contributors are not always malicious, but they're still users. They do users' things. Users things are often stupid things. Stop them.
This project about Dancer2 deployment will be quite long. I'm putting in it a lot of random things from my experience. So I stop it now and I put a number on the title. We'll be back, i hope soon.



Sunday, 29 September 2013

404 Ghost workaround

A little recap:

Here I stated a problem about phantom 404 and  a (wrong) solution to it.
Here I confessed I was working on the issue.
Here It's the work I'm doing with a long and boring description of the problem that I'll not repeat in this blog.

...but probably you don't want to wait until the problem is really closed because you have (as me) a package that you want on CPAN to share it with the world.

Oh yes, the problem! Let me say it with few words:
You have a plugin that modify the request path in a before hook. You want that, when the user write (for example) /prefix/good/path, your plugin change it to /good/path. Obviously the code that will answer to this will be:

get '/good/path' => sub {...}

You don't define anything about /prefix/good/path because the plugin will always change it, so you'll never need to answer with it.
This works! I can swear it calling Wotan on my side. But when you try to test it with dzil test the test you wrote fails.

Long explanation for why this happen, as I said, is here. Little workaround you can use to stop this happening is adding a public directory under the t with just a dummy file. This way fix the problem and make you test suit running flawless!


Monday, 9 September 2013

In the previous episodes of Battlestar Galactica...

I didn't write anything for a lot of time because I was on holiday and (more important) a lot of things happened in the Dancer2 World and I was a bit confused about some of them.

First important news, for me, is that use Dancer2 :sytanx is no more from 0.07. I was a little puzzled about it because, as I wrote here the pure use Dancer2 is a powerful spell that do many things and sometimes it could be more than what you want.
I asked for enlightement (i'm still a barbarian, you know) and Sawyer X himself gave me this answer about it.
So, using use Dancer2 also when you're not creating an app is good.
All I said about apps  and scope is still valid, so we can say farewell to use Dancer2 :syntax (a legacy of Dancer) with no regret.

Second question I'm working about is nasty like a leprechaun with no sense of humor that hate Harry Potter. In this article I talked about phantom 404 that raise instead of the right answer while testing. I thought this issue was solved using TCP::Test. This is not true, there is still a lot of work to do about this fascinating enigma and it's exactly what I'm doing right now. It's an indispensable step to release my Dancer2 plugin (Dancer2::Plugin::Multilang) on CPAN so I'm working hard about it!

Stay tuned!

Sunday, 4 August 2013

Initiation rite

We're here to celebrate!

If you're a wotan worshipper the initiation rite is a very important part of your life. Usually, this rite is something about killing bears, drinking blood, dancing all night and having sex with virgins.. while they try to kill you with a sacred knife.

If you're a programmer you don't have to do anything of this, but you still need something that demostrate that you've grown up.

For me, it's the pull request Dancer2 team accepted today. It's something about sessions and forward that I found while developing the Dancer2::Plugin::Multilang. It's just a small thing, nothing revolutionary, but seeing it merged in the Big Project gives me a good feeling.
I think that working on a big open source project like Dancer2, also giving little contribute, is good to learn a lot about coding, collaboration and computer science. I think all people should try something like this, just to improve their skills.

Probably I'll talk about this some day, a technicle, boring post about data flows, but that time is not now. Now it's just time to celebrate!


Monday, 22 July 2013

The dumb code festival - 3

In the first two chapters of our dump code festival we just did a lot of frontend developement, something completely independent from the framework we are using and with no server-side logic.
Well, today frontend developement is very important and HTML::FormFu is a very complex and obscure thing so I think it's not so bad we spent all this time on it.

You know? I'm trying to make HTML::FormFu "cooperate" with Twitter Bootstrap. Yes, it's possible and easy enough, but you have to do tricks and you'll never have all the freedom you probably want. However, letting twitter bootstrap render the form make it nicer than not doing it, so... give it a try.

But now we have to talk about logic.
Business logic in a login form is triggered just on submit, when data start its voyage to the server and the server is committed to give an answer to the user. It's like going to the oracle. Yes, the oracle with eyes turned up to the sky and funny clothes, not the oracle database server.

Easiest and probably most elegant way to manage a login form is making is action calling again the login page. This is the meaning of the any keyword we used last time to define the route. You arrive on the login route with a GET (obtaining the form), you process the submitted data with a POST on the same route.

The four needed steps are:

  1. Create the form object
  2. obtain the DATA inserted from the user
  3. process the data
  4. Do something with the data (yes, you can throw it away and laugh, but this way you'll never become a professional programmer)

As you can see the code is very simple and it's just HTML::FormFu syntax mixed up with very few Dancer2 magic.
We saw the first two lines in the last chapter. They, with the last one, make the form appear on the screen. The params keyword retriver all the variables arrived with the request, it's a pointer to an hash that's suitable for the $form->process. Process takes every key in the hash and put it in the field of the form with the same name, triggering contraints, validators, inflators etc. etc. The result of this computation is in the submitted_and_valid variable that can be easly tested to see if the data can be processed.

Staying in the same route makes all the messages easy to manage. The process can find a violeted contraint, for example (as an empty field, as we configured), then it can write the error message directly in the form and the rendering, on the bottom of the route, will make it automatically visible.

In the GET scenario no param is present, process receive an undef and do nothing to change the form. The render will display the form untouched, ready to be used.

What about the login_valid? It's up to you, you have to decide when a user is good and when it's not.
This is a easy and dumb implementation:



Users and passwords are on a little DB table, accessible using the Dancer2::Plugin::DBIC (as always). Remember: just the MD5 for your password IS NOT a secure way to store it. But, as I said, this is just an example.

Well, there's just one more thing to say, probably the most interesting because it's the one about Dancer2 structure and server-side logic. It's just this: now we have a login page, what we need is that it appears every time someone try to navigate our admin panel. EVERY TIME.

Tuesday, 16 July 2013

Dancer2::Plugin::Multilang

So, my wonderful Dancer2 plugin to manage multilanguage sites is released on github!

I wrote in this blog something I learned developing it, so I think that this place is the right one to make the announce.

I tryed to design this plugin as "plug-and-play" as possible. With it, you should be able to make your site multilanguage with no other effort then... translate the contents.

Netx step is releasing it on CPAN but I never did something like that so I have to learn how. I hope to learn it well enough to come back and writing about it in my blog, but now I don't have enough time to do it, so this topic has to wait a little.

In the meantime download, try, fork and play with my plugin as long as you want. Feedbacks (yes, bad feedbacks too) are welcome!

Saturday, 13 July 2013

The dumb code festival - 1

One thing I usually miss when I google things about Dancer2 (or other coding stuff) is examples of simple working code doing normal things. Every time you find just little snippets that do just a thing in a way that will be useless to you ore code doing wonderful, exciting things you don't need.
For Dancer2 we have this article from Sukrieh himself that is very useful to learn how Dancer2 works, but... just this.

I want to put in this blog all the things I don't find in internet so I decided to start some articles about an obvious, stupid thing you all do blind and tied up while singing Yellow Submarine. Just because I think that out there there's someone who's not singing loud enough.

Let's talk about a little adminitration panel for your website. I'm not interestend in what your website do, there're a lot of things an administration panel has to do anyway. The most important thing is that it has to keep out all the non-authorized people and let the authorized in.

Authorized people will enter somewhere username and password. There're a lot of ways to do this (and some plugin from Dancer that could be easly migrated to Dancer2) but we'll to it from scratch in the most obvious way: a login page.

A login page is just a form with you fields, a big red button and a lot of threatening messages when you do wrong things.

Talking about forms, perl and frameworks, you should learn about HTML::FormFu, one of the most used libraries for forms in perl.
Using it, you'll translate all the logic you want in the form in a configuration file, adding to it, eventually, some libraries for validation, filtering and so on, downloaded from CPAN or written by you.
You could think this is too much complex and without real gain for your coding, especially when you are dealing with little forms.
My advice is to always follow this design path for two important reasons:

  1. If you don't use it with simple problems, you'll NEVER use it in complex one. Don't lie.
  2. In every case templating the form is keep separated from coding is logic, something you must do if you want to write tidy code.
A login box using HTML::FormFu at his minimum potential has a config file (login.yml) like this:


You can make it a little more beautiful, but all the logic you need is here.

For this time we'll stop here, i wrote the introduction to this experiment, so the article is already long enough. Numbers near the title means that we'll go on. I hope that I'll create something useful with this, useful for the internet and also for me, to make my mind a little more clear about what I do.

Friday, 28 June 2013

LWP::UserAgent: a browser

Last time we had a lot of fun playing with Test::TCP and LWP::UserAgent, but we're not done yet with these toys, there's something more I want to say about them.

So we can test our Dancer2 route:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/route/");

ok($res->is_success);
is($res->content, 'my content');

Sometime this is not enough. In the code I'm developing redirect is important as success, how can I test this? Well, I suppose...


my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->code, 302);

But result is:

# got: '200'
# expected: '302'

What's wrong?

You could think (I did, at last) that LWP::UserAgent is just a "tester", something to do "ping" on internet sites and see what come back. It's not, LWP::UserAgent is powerful because it's a complete working browser, like Firefox, Opera, Chrome or IE. It's not good in rendering pages, ok, but the data-flow it can manage is the same you can see while surfing the web.

What a normal browser like Firefox do when there's a redirect?

Simple:

CALL -> 302 -> NEW PAGE -> 200

Though you're giving back a redirect for a route you still want to give a page, a success result, at the end of the navigation chain. LWP::UserAgent follow all the chain without asking for nothing, as your browser, bringing back the final 200 response.

Ok, LWP::UserAgent is cool and I'll use it to fetch good pron, I promise, but now I'm looking for something that check my Dancer2 App doing the right redirect!

Test is just a bit more complex:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->is_success and $res->previous);

Call ends with a 200 but a previous URL was registered and we're redirecting from there. You can see this URL in $res->previous but knowing it's there is enough to say we're redirecting.

Something more about the browser powers of LWP::UserAgent?
What if you need to test sessions?  How Dancer2 trace client sessions? With a session cookie, obviously. And LWP::UserAgent can manage them too? Obviously again, but remember to give it  a cookie_jar to use:

$ua->cookie_jar({file => "cookies.txt"});

Rememeber. With LWP::UserAgent you're not just trying out your site. You're literally surfing it!

Wednesday, 26 June 2013

Dancer2::Test. undiscovered countries

If you want to do serious developement, talk with serious people and produce serious packages you have to use testing scripts A LOT. Dancer2 comes with its Dancer2::Test library, a lot of good features... and few gloomy mysteries.

Let me start with a little tip: whenever you start testing import Dancer2::Test and specify which apps you're going to use:

use Dancer2::Test apps => ['app1', 'app2'];

Probably you already knew it, but I had to dig up a little in Dancer2 docs to find that this is a "must have" for Dancer2::Test and everytime I spend time to find something I think that what I find deserves few words on this blog. When you'll search for it like me, I'll be there, on the first page of Google, waiting for you.

But I was talking about gloomy mysteries so now we'll talk about strage behaviours of Dancer2::Test, triggered by nasty (but legit) coding choices.

In my first article I talked about before hooks and forward.

See this example:

hook before => sub {
      my $context = shift;
      if (request->path_info eq '/ghost_path' )
      {
           $context->response( forward('/solid_path' ));
           $context->response->halt;
      }
}

You obviously defined a dispatcher for '/solid_path' but there's no reason you have to define a dispatcher for '/ghost_path'. Still, calling '/ghost_path' will return a page (HTTP code 200! Win!).

Now we'll write a test for this:

response_status_is ['GET' => '/ghost_path'], 200;

Result is:

#          got: '404'
#     expected: '200'


What? Aaaargh, Wotan worshipper rises is axe!

Dancer2::Test does a bit of introspection in your Dancer2 App. When you ask for a path it simply tries to find it in the defined dispatchers, where there isn't. It can't deduce that all will go well thanks to the before hook so the test fails.

But. Wotan. Worshipper. Wants. To. Test!

Well, servers never lie. Never. If the test could be done with a real server and not just... playing with pms...

Here comes Test::TCP.


use strict;
use warnings;
use Test::More;

use Test::TCP;
use LWP::UserAgent;


Test::TCP::test_tcp(
    client => sub {
        my $port = shift;
        my $ua = LWP::UserAgent->new;
        my $res = $ua->get("http://127.0.0.1:$port/ghost_path/");
        ok($res->is_success);
    },
    server => sub {
        my $port = shift;
        use Dancer2;
        use multilang;
        set(show_errors  => 1,
            startup_info => 0,
            environment  => 'developement',
            port         => $port
            );

        Dancer2->runner->server->port($port);
        start;
    },
);
done_testing;


This is a real server launched by a script and reached by a real client. It's just as the actual site. And the test goes well...

What we learned today? Running a server and click in a browser is the only reliable way to test something. Let a machine doing it by itself without all the boring part (open a browser... digit the url... go on Facebook while loading...) is good.
Obviously, in many many cases Dancer2::Test, plain and simple, is enough and it's the right way to do things, because it's a bit less complex working with it. But keep an eye on exceptions...


Monday, 24 June 2013

The dawn of the combining robots

Yesterday I wanted to say a lot of things about Dancer2 and Apps, but I was a bit sleepy, so I keep the discussion on simple topics. Now I want to talk about something I experienced in the passage from Dancer 1 to Dancer 2 and how it fooled me.

When I started using Dancer 1 I was a young (well, younger than now) and I did dirty things, as young people do. Things like putting Dancer routes directly in the bin/app.pl script.
Talking about Dancer 1 this is not a really dirty thing, Dancer 1 declarations are always global so the problem is just "esthetic".

Routes under lib directory, in a perl module, give a feeling of tidiness, nothing more.

Problem is that, working this way, you start thinking about bin/app.pl as your app, as the center of your site. This is a lot confusing when Dancer2 come along.

You should think about bin/app.pl as just your "launcher", keeping it as empty as you can, using it just to configure what code you want to load in your site.

The dancer2 init script will not help you thinking this way. Launching it you'll have:
  • bin/appl.pl with import Dancer2 
  • lib/APPNAME.pm with just import Dancer2 :syntax 
What does this mean? It means you're working just with one app, declared in the bin/app.pl and all the code you'll write in pm files will be referred to it. This app will have name "main" and your design will start in a not-so-modular way.

Evil evil Dancer2 init, wanting to keep developers in the Dancer1 global world, hiding the great powers of Dancer2 Apps!

What should you do? In my opinion you should leave import Dancer in the bin/app.pl (you can't make webapp start without it), but you should also use import Dancer in your lib/APP.pm files. Files, yes, more than one, because you should split up functionality (and navigation) in more than one perl module.

Then, your bin/app.pl will become:

import Dancer2 #a "main" app will still be created
import APP1 #some routes and logic
import APP2 #different routes and logic
and so on...

This way, just commentig one line (or uncommenting another) you can turn on/off features, behaviours, navigations and the global structure of your site is easy to understand.

A little thing to close the post. Try sometimes to launch your developement server this way:
DANCER_DEBUG_CORE=1 bin/app.pl
Bootstrap will give you notifications about th Apps start. Here you'll see site structure in a very clean way, something the code itself can't do.