Ballroom Blitz (2)

Silence in the library

When you write a perl program you don't need just the perl interpeter, but also a lot of libraries that someone else wrote. It's easy to fetch them while you're in your developement environment, but when you go to production you have to manage a lot of issues about this problem.

First of all, there's the scenario where you don't have root access for the system where you are so libraries installation appear to be an impossible task. 

A worse scenario is the one where you have root access and you install the libraries directly in the system.

The worst case scenario is the one where you're installing things in the system, the system die and someone behind you start hitting you with a club. No, the problem is not the one with the club, the problem is you. The club is the solution.

Never do installations at system level. System level is just for... the system, the libraries installed that are the libraries need by your OS, not the libraries you need for your site. In many years of developement and deployment  people found many ways to do things better.

The article I linked last time, for this task, uses perlbrew, one of the best practices for that. Perlbrew manages not only libraries, but also perl version, so it can also be used to ensure that the bizarre incompatibilty between 5.16.0 and the cutting edge 5.18.0 is not an issue because you will configure on your machine just the version that works.

But timtowtdi so in this article I'll speak about probably the oldest practice about managing libraries, the oldest but still useful, clean and sharp: local::lib.

local::lib is a library that you have to install at system level:

(as root) cpan -i local::lib

 after that you'll have the power of playing with perl libraries just moving directories. Remember the directory where we decided to place our site, /var/www/com/example/my/?

Create under this directory a directory env and then a file named perl_sandbox with these lines:

libdir=/var/www/com/example/my/env
export PS1='\[\e[1;35m\]\u@\h:\w\$\[\e[0m\] '
eval $(perl -I$libdir -Mlocal::lib=$libdir)

Now source the file:

source perl_sandbox

This way your shell will become pink (the export PS1 line) and all the operations you'll do about perl libraries will have effect just on the env directory. However, with the perl_sandbox active, every perl command you'll use will see the libraries in the env directory.

Just a last tip: remember to install cpanm

cpan -i App::cpanminus

and use that for installations, with --notest option. Life is too short to let cpan to check all the repository every time you need an environment.

Up to the plack

Plack is a server used to run PSGI applications. PSGI is a interface that we can use to communicate with the webserver that will bring our site on the World Wide Web, so Plack is something we really need to make things work.

Talking about this issue, working with Dancer makes thing really easy because the standard bin/app.pl script that run your webapp can be used as is as a PSGI startup file, as the Dancer2 Deployment page says.

bin/app.pl script must run under a perl web server supporting PSGI. My favourite for this is starman (yes, I like David Bowie, but it's not just for that).

Copy&Paste from Dancer2 Deployment page how to run plackup and make your site works, however, is not enough for something that will need maintenance in the future.

The last piece of the puzzle is Server::Starter, a perl module that provide you the start_server script that will change you plackup server in a daemon (Argh! WItchcraft!), making easy for you stop/start procedures and monitoring.

After installing that module and after creating a /tmp directory under our site directory, you can try THIS copy&paste:

WORKINGDIR=/site/directory
PROJECT=/where/bin/directory/is

source perl_sandbox #as seen in the previous paragraph 

          cd $PROJECT

start_server \
--pid-file=$WORKINGDIR/tmp/starman.pid \
--status-file=$WORKINGDIR/tmp/starman.status \
-- \
plackup -E ec2 -s Starman --workers=2 -l $WORKINGDIR/tmp/plack.sock -a bin/app.pl & 

as your start_server script.

And this as last command:

start_server \
--pid-file=$WORKINGDIR/tmp/starman.pid \
--status-file=$WORKINGDIR/tmp/starman.status \
--restart

For restart.

For the timtowtdi section, take a look at Daemon::Control for an alternative implementation of the daemon. For a PSGI webserver general purpouse, one of the most used technologies of these days is uWSGI.

Gentlemen, start your NGINX

Last step is the easiest one. NGINX is just a proxy from the internet to your plackup structure. NGINX and Plackup will communicate using the plack socket that we configured in the start script: /my/site/dir/tmp/plack.sock. Obviously we're talking about an environment where Plack and uWSGI are on the same machine, otherwise you'll have to sail for the TCP/IP, as everything on the network.
Let's see nginx configuration:

upstream siteplack {
  server unix:/your/site/directory/tmp/plack.sock;
}

server {
  server_name www.yoursite.com;

  access_log /your/site/directory/logs/www.site.it_access.log;
  error_log /your/site/directory/logs/www.site.it_error.log;

  root /your/site/directory/project/src/public;

  location / {
    try_files $uri $uri @proxy;
    access_log off;
    expires max;
  }

  location @proxy {
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://siteplack;
  }
}

Few things to say. The public directory of the Dancer2 project should be server directly, because DancerApp has nothing to do with static things like css and javascripts. All the others things are just to copy and paste.

Restarting NGINX, praying all your gods, if all you did was did well your site will rise.
If not probably it will be my fault writing something wrong in here, but I'll be too far from you to be blamed. Just solve your problems and tell me how to fix the article. Comments are there for this reason.

Ballroom blitz (1)

I read this interesting article about Dancer2 deployment. I have a lot to learn about Dancer2 deployment so I thought seriously about apply this practice to my sites. However, working on it, I didn't feel so confortable with the techniques applied there, probably because of my habist more than other reasons.
So, at the end, I came to a different solution and I want to describe it here, to clarify it in my head and to demostrate, once more, that timtowtdi.
I will skip the part about the provider subscription, I want to focus just on the technical issues and I don't want to talk about a specific environment. Let us think just about you having a server, anywhere you want, and the Power of Root is in your hands.

Who's the man?

When you're young and stupid you can think that root is enough, because root is powerful, shiny, charming and has a silver armour with gold drawings on it. Uncle Ben died to teach you that with great power comes great responsibility and you're still so stupid to use root and just root, also for pron watching!

Then you grow up and you know that there're three type of users:

• root: the boss. Period. You don't annoy the boss with stupid deployment issues.
• human users: they have to log in on your machine and do dirty things. They need a little, walled garden (some guards on the wall, too)
• application users: they're on the machine just to work. They're strictly linked to the processes they run.

So, you need to start your app deployment and you have to choose a user that will do it. For this issue, after years chasing not-so-writable directories and evoking the sacred chmod sevensevenseven, I chose the easy way: do all using the nginx application user (nginx or www-data, usually). It's not perfect, it's not the cleanest way, it's not the most secure, but it's tidy enough and give you a lot of space to manage things.

From now on, consider yourself logged as nginx. It's home directory will be /var/www.

Where's the man?

Just a small paragraph about directories. If you don't have a strict discipline your system will become a mess. It's its destiny, it's nature. Entropy is part of every job in the universe.

My favourite idea about how to organize different sites on the same machine is to use reverse domain as subdirectories.

So, mylittlepony.evilgenius.com will go under /var/www/com/evilgenius/mylittlepony and fluffy.bunny.org will be /var/www/org/bunny/fluffy.

This way it will be easy to find the site you're looking for and directory names will never be too long. The policy for domain names is also one of the strictest of the computer science, so you'll never have to manage UTF-256 characters or things like that.

Beam me up Scotty!

This article will speak also about the obvious. 

The easiest way to put the code of your webapp on your server is through some versioning system like git. You just have to pick up an account on github or bitbucket and then, after finishing the development, push all you did on a repository. Then, on your webserver, you just have to clone the project and then maintaining that pulling every update you push.

There's a part of me that don't like working this way. The cleanest and most clever way to deploy a software is packing it someway and then installing it on the server. Packing and installing must be done through a clean, sharp and smart suite of scripts tailored on the software you are managing.

This is the old school procedure and as a lot of old school procedure it's still good, but I have to admit that git is easier and perfect for lazy people.

But! Remember! :

• Think about if you need a private repository. It could be that someone doesn't want that the code of the site is exposed to everyone with a google bar. I know, I know, open is better, open is good, open i what tender kittens wants, but sometimes it's not what YOU want.
• gitignore is the most important file of your repository. Design it thinking that nothing about your server environment should be in the git repository. For two simple reasons. One: if you have on git something related to an environment it could change the environment configuration on deploy just because on your developement server something is different and making a git pull a disaster. Two: you don't want your production database password written on a public github repository... and probably you don't want it also on a private one, if you don't know all the contributors to your project.
• Take a branch as the production branch and never touch it before deploy time. When you have to develope a feature branch from production and develope there. On deploy, merge your new branch with production and push. You have to trust in production branch. Every time you go on your server and do "git pull origin production" you have to know exactly what will happen. Nothing, in the most cases, the deploy of something you planned, on the happy release date.
  Again, if you don't trust all your contributors, there're many ways to prohibit people to push on production. Enforce them. Your contributors are not always malicious, but they're still users. They do users' things. Users things are often stupid things. Stop them.

This project about Dancer2 deployment will be quite long. I'm putting in it a lot of random things from my experience. So I stop it now and I put a number on the title. We'll be back, i hope soon.

404 Ghost workaround

A little recap:

Here I stated a problem about phantom 404 and  a (wrong) solution to it.
Here I confessed I was working on the issue.
Here It's the work I'm doing with a long and boring description of the problem that I'll not repeat in this blog.

...but probably you don't want to wait until the problem is really closed because you have (as me) a package that you want on CPAN to share it with the world.

Oh yes, the problem! Let me say it with few words:
You have a plugin that modify the request path in a before hook. You want that, when the user write (for example) /prefix/good/path, your plugin change it to /good/path. Obviously the code that will answer to this will be:

get '/good/path' => sub {...}

You don't define anything about /prefix/good/path because the plugin will always change it, so you'll never need to answer with it.
This works! I can swear it calling Wotan on my side. But when you try to test it with dzil test the test you wrote fails.

Long explanation for why this happen, as I said, is here. Little workaround you can use to stop this happening is adding a public directory under the t with just a dummy file. This way fix the problem and make you test suit running flawless!

In the previous episodes of Battlestar Galactica...

I didn't write anything for a lot of time because I was on holiday and (more important) a lot of things happened in the Dancer2 World and I was a bit confused about some of them.

First important news, for me, is that use Dancer2 :sytanx is no more from 0.07. I was a little puzzled about it because, as I wrote here the pure use Dancer2 is a powerful spell that do many things and sometimes it could be more than what you want.
I asked for enlightement (i'm still a barbarian, you know) and Sawyer X himself gave me this answer about it.
So, using use Dancer2 also when you're not creating an app is good.
All I said about apps  and scope is still valid, so we can say farewell to use Dancer2 :syntax (a legacy of Dancer) with no regret.

Second question I'm working about is nasty like a leprechaun with no sense of humor that hate Harry Potter. In this article I talked about phantom 404 that raise instead of the right answer while testing. I thought this issue was solved using TCP::Test. This is not true, there is still a lot of work to do about this fascinating enigma and it's exactly what I'm doing right now. It's an indispensable step to release my Dancer2 plugin (Dancer2::Plugin::Multilang) on CPAN so I'm working hard about it!

Stay tuned!

The accent

My name is Simone Faré and I'm italian. I think you can understand how important is for me UTF-8 and character set management.

Using a framework should allow you to avoid all the boring parts of stuff and UTF8 in Dancer2 is actually like that, but you need to remember the three points where the magic spell lies and ensure to activate all of them.

The first point is the easiest, the most powerful and it comes for free.
It's just the  charset: "UTF-8" configuration key in the config.yml. Spend a moment reading the documentation page about it and you'll find how many things this little word do. You could try it also with girls in pubs, probably It will do something good also  with it as well.

The second point is about your template engine, if you're using template toolkit add:

template: "template_toolkit"

engines:
    template_toolkit:
        ENCODING: utf8

This is a useful configuration, but not as powerful as the other one. If you're in an environement where all the charset are managed for good your template engine and the browser will not need to do lucky (wrong) guess about encoding. I think that this configuration can help just when guessing is needed.

Obviously the third point is very important, very insidious and very well hidden in the documentation, because there's no gain without pain.

The first configuration is about the framework, its internal logic, the second is about HTML and templating, so who are we forgetting? Never heard of something named MySQL?

DBA are strange people, very different from developers. A database has encodings, collations, things about tables, magic and curses everywhere. What I know is that if you don't nail down your DB connection to what you want you will never save good data in it. How to do this?
In the database configuration:


options:
        RaiseError: 1
        PrintError: 1
        mysql_enable_utf8: 1


Just that. But remember it or you'll suffer. Suffering is good, but suffering forever is unfair.

Initiation rite

We're here to celebrate!

If you're a wotan worshipper the initiation rite is a very important part of your life. Usually, this rite is something about killing bears, drinking blood, dancing all night and having sex with virgins.. while they try to kill you with a sacred knife.

If you're a programmer you don't have to do anything of this, but you still need something that demostrate that you've grown up.

For me, it's the pull request Dancer2 team accepted today. It's something about sessions and forward that I found while developing the Dancer2::Plugin::Multilang. It's just a small thing, nothing revolutionary, but seeing it merged in the Big Project gives me a good feeling.
I think that working on a big open source project like Dancer2, also giving little contribute, is good to learn a lot about coding, collaboration and computer science. I think all people should try something like this, just to improve their skills.

Probably I'll talk about this some day, a technicle, boring post about data flows, but that time is not now. Now it's just time to celebrate!

The dumb code festival - 4

Authentication and authorization is something every framework must manage. Because they're boring and frameworks exist to manage boring things.
When a user can be considered authenticated? The easiest way to check it it searching is its session for a user key with some interesting value. Is this secure? Enough, because we're talking about server-side session, something a user can't touch, we have to trust it well... until a new security flaw is found in one of the layer of our system.

The "auth" thing is a pattern well known in the web world and you can find many solutions for it, looking for "Dancer::Plugin::Auth" you'll find a lot of stuff. Unfortunately Dancer2 is still a young project so the only tool we have is the Dancer2::Plugin::Auth::Tiny from Golden.
We'll not use it. Ok, the plugin is a very good piece of software but its main purpose is define which routes need login to redirect on login page the not authenticated users. I think that this is made useless by the Dancer2 App scope.
In a situation where routes for logged users and routes for not-logged users are mixed up probably D2::P::A::Tiny is the right solution, but here we have an app where every route needs auth so we'll solve our problem just with a before hook defined in the app itself.

But we have to start from: how to open the doors to a logged user? Login went well, user and password were right, what the server will do? As I said before, it will write in the user session the username.

session 'user' => $the_username;

This is what our hook will check:


Yes, just this. The most common mistake is forgetting that the login route has to skip the check because is the only page allowed for not logged users. In every other case the hook does NOTHING for logged users and redirect to the login page the guests.

I made it more longer than what is needed because I like to talk and write and because sometimes I think that also obvious things need some explanation, somewhere. Conclusion is that using the snippets from the dump code festival you have the start situation for an admin tool you can use for every purpose. Probably, working on it, you'll find a lot o things to improve it or make it smarter. But this is just the first step, the dump step, so it's right as it is.

The dumb code festival - 3

In the first two chapters of our dump code festival we just did a lot of frontend developement, something completely independent from the framework we are using and with no server-side logic.
Well, today frontend developement is very important and HTML::FormFu is a very complex and obscure thing so I think it's not so bad we spent all this time on it.

You know? I'm trying to make HTML::FormFu "cooperate" with Twitter Bootstrap. Yes, it's possible and easy enough, but you have to do tricks and you'll never have all the freedom you probably want. However, letting twitter bootstrap render the form make it nicer than not doing it, so... give it a try.

But now we have to talk about logic.
Business logic in a login form is triggered just on submit, when data start its voyage to the server and the server is committed to give an answer to the user. It's like going to the oracle. Yes, the oracle with eyes turned up to the sky and funny clothes, not the oracle database server.

Easiest and probably most elegant way to manage a login form is making is action calling again the login page. This is the meaning of the any keyword we used last time to define the route. You arrive on the login route with a GET (obtaining the form), you process the submitted data with a POST on the same route.

The four needed steps are:

1. Create the form object
2. obtain the DATA inserted from the user
3. process the data
4. Do something with the data (yes, you can throw it away and laugh, but this way you'll never become a professional programmer)

As you can see the code is very simple and it's just HTML::FormFu syntax mixed up with very few Dancer2 magic.
We saw the first two lines in the last chapter. They, with the last one, make the form appear on the screen. The params keyword retriver all the variables arrived with the request, it's a pointer to an hash that's suitable for the $form->process. Process takes every key in the hash and put it in the field of the form with the same name, triggering contraints, validators, inflators etc. etc. The result of this computation is in the submitted_and_valid variable that can be easly tested to see if the data can be processed.

Staying in the same route makes all the messages easy to manage. The process can find a violeted contraint, for example (as an empty field, as we configured), then it can write the error message directly in the form and the rendering, on the bottom of the route, will make it automatically visible.

In the GET scenario no param is present, process receive an undef and do nothing to change the form. The render will display the form untouched, ready to be used.

What about the login_valid? It's up to you, you have to decide when a user is good and when it's not.
This is a easy and dumb implementation:

Users and passwords are on a little DB table, accessible using the Dancer2::Plugin::DBIC (as always). Remember: just the MD5 for your password IS NOT a secure way to store it. But, as I said, this is just an example.

Well, there's just one more thing to say, probably the most interesting because it's the one about Dancer2 structure and server-side logic. It's just this: now we have a login page, what we need is that it appears every time someone try to navigate our admin panel. EVERY TIME.

Dancer2::Plugin::Multilang

So, my wonderful Dancer2 plugin to manage multilanguage sites is released on github!

I wrote in this blog something I learned developing it, so I think that this place is the right one to make the announce.

I tryed to design this plugin as "plug-and-play" as possible. With it, you should be able to make your site multilanguage with no other effort then... translate the contents.

Netx step is releasing it on CPAN but I never did something like that so I have to learn how. I hope to learn it well enough to come back and writing about it in my blog, but now I don't have enough time to do it, so this topic has to wait a little.

In the meantime download, try, fork and play with my plugin as long as you want. Feedbacks (yes, bad feedbacks too) are welcome!

The dumb code festival - 2

Let us spend some more words about forms and HTML::FormFu. In the preview chapter we just scratched the surface of it, doing something too essential to be enough, even in a dumb code festival.

Form is a way to have a structured stream of input coming from a human being to a machine. Human beings are bad doing data input. They insert bad data, they're lazy, they use funny mysterious languages like english and they're often stupid. You can't trust them, you have to constraint them, filter their input, inflate it and validate. (you would also bring them all and in the darkness bind them but this is not a tutorial suited for that).

Keeping us near the little example we're working on, a login, box, what kind of bad input we should manage? Luckly, very little, the only bad thing a user could try is to submit the form with no data inside. We could ignore this case, obviously it will led to a bad login, but let us make HTML::FormFu manage it for us.

Very little code added, just a contraints field with an element. The element is defined by a type and a message to me thrown up when constraint is violeted.

An interesting thing about: where that "Requeired" word come from? How can I know which words can do this kind of magic? Is there a sort of grimorioum for them?
Obviously not, the keyword you put in the type is just the name of a perl package, in the HTML::FormFu::Constraint namespace. The one invoked here is this.
What does this mean? It means that you can look at the available constraints just surfing the CPAN for contrainst classes and that you can use constraints created by your own just giving them the HTML::FormFu::Constraint namespace and putting them in the configuration file as "official" ones.
And yes, filters, inflaters and validators follow the same logic!

Just updating the form configuration you have managed the empty fields case. HTML::FormFu will reject the submission and will display errors on screen if the user try to submit without writing anything (lazy, lazy human!).

Now two important downsides:

1. Validation is server-side. The form is submitted, POST data elaborated, page reloaded with errors. If you're aiming to a validation with no page refresh and client-side javascript magic you're looking in the wrong place.
2. HTML produced by FormFu is the HTML it likes. You could not like it. You could think is a little old-style, too rude and simple. For example, I hate it doesn't use the Label for attribute.
   You can decide to work "around it" knowing how it is and putting javascript and CSS to make it looks good. The harder way is going down in the HTML::FormFu classes and redefine the rendering. This thing is not enough dumb for this article.

But... probably after so many words you want to see your form, don't you?

Open your Dancer App and render it!

The any key to acquire GET and POST will come useful in the future. Remember that the login.yml will be searched in you app root, the same directory where the Dancer config.yml is placed. Subdirectories are allowed (and recommended).
Remember to have in your template a [% form %] token where the box will be displayed.

This piece of code is very stupid, nearly useless, because there's no logic. But it's just the second part and we have a lot of things to talk about.

The dumb code festival - 1

One thing I usually miss when I google things about Dancer2 (or other coding stuff) is examples of simple working code doing normal things. Every time you find just little snippets that do just a thing in a way that will be useless to you ore code doing wonderful, exciting things you don't need.
For Dancer2 we have this article from Sukrieh himself that is very useful to learn how Dancer2 works, but... just this.

I want to put in this blog all the things I don't find in internet so I decided to start some articles about an obvious, stupid thing you all do blind and tied up while singing Yellow Submarine. Just because I think that out there there's someone who's not singing loud enough.

Let's talk about a little adminitration panel for your website. I'm not interestend in what your website do, there're a lot of things an administration panel has to do anyway. The most important thing is that it has to keep out all the non-authorized people and let the authorized in.

Authorized people will enter somewhere username and password. There're a lot of ways to do this (and some plugin from Dancer that could be easly migrated to Dancer2) but we'll to it from scratch in the most obvious way: a login page.

A login page is just a form with you fields, a big red button and a lot of threatening messages when you do wrong things.

Talking about forms, perl and frameworks, you should learn about HTML::FormFu, one of the most used libraries for forms in perl.
Using it, you'll translate all the logic you want in the form in a configuration file, adding to it, eventually, some libraries for validation, filtering and so on, downloaded from CPAN or written by you.
You could think this is too much complex and without real gain for your coding, especially when you are dealing with little forms.
My advice is to always follow this design path for two important reasons:

1. If you don't use it with simple problems, you'll NEVER use it in complex one. Don't lie.
2. In every case templating the form is keep separated from coding is logic, something you must do if you want to write tidy code.

A login box using HTML::FormFu at his minimum potential has a config file (login.yml) like this:

You can make it a little more beautiful, but all the logic you need is here.

For this time we'll stop here, i wrote the introduction to this experiment, so the article is already long enough. Numbers near the title means that we'll go on. I hope that I'll create something useful with this, useful for the internet and also for me, to make my mind a little more clear about what I do.

Database around the clock

Probably this article is about something that everyone in the world knows. Problem is I did mistakes about this TWO times, without learning from them, so writing down it could be useful for this head of mine to remember how to do things right.

Writing a timestamp whenever a database row is created or updated is a standard task and everyone need it.  Many database systems can do it by themselves (MySQL), dumber ones cannot (SQLite). Are we interested in this? Obviously not, DBIX::Class (the one and only) give us a good abstraction about that and it's easy to use.

When you know the trap to avoid.

Syntax to have a timestamp that set itself on create is really simple. In your model class:


__PACKAGE__->add_columns(
  "timestamp",
  { data_type => "datetime", is_nullable => 0, set_on_create => 1 },
);


It's something so used that there's a flag to manage it. You set it to true, the timestamp will be written.

IF and only IF you remember this:

__PACKAGE__->load_components("TimeStamp");

and you installed DBIx::Class::Timestamp!

Are you hearing the Wotan worshipper roaring? Why should he? It's a procedure so clean!

Well, perl is a language that worship optimization (more than Wotan! Grrrrr). Often optimization means a fine grane library control. So in many important modules you see methods different from use... to load libraries. Most of these methods have a little side-effect: they don't give errors.
In this particular case you can write the first piece of code, forget the second or write the second and forget to install the library. The code will run. What will happen about the timestamp? Absolutly nothing.

But now I wrote this. Now this thing is encarved in my blog and in my mind. I will never be tricked again.

What were we talking about?

LWP::UserAgent: a browser

Last time we had a lot of fun playing with Test::TCP and LWP::UserAgent, but we're not done yet with these toys, there's something more I want to say about them.

So we can test our Dancer2 route:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/route/");

ok($res->is_success);
is($res->content, 'my content');

Sometime this is not enough. In the code I'm developing redirect is important as success, how can I test this? Well, I suppose...


my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->code, 302);

But result is:

# got: '200'
# expected: '302'

What's wrong?

You could think (I did, at last) that LWP::UserAgent is just a "tester", something to do "ping" on internet sites and see what come back. It's not, LWP::UserAgent is powerful because it's a complete working browser, like Firefox, Opera, Chrome or IE. It's not good in rendering pages, ok, but the data-flow it can manage is the same you can see while surfing the web.

What a normal browser like Firefox do when there's a redirect?

Simple:

CALL -> 302 -> NEW PAGE -> 200

Though you're giving back a redirect for a route you still want to give a page, a success result, at the end of the navigation chain. LWP::UserAgent follow all the chain without asking for nothing, as your browser, bringing back the final 200 response.

Ok, LWP::UserAgent is cool and I'll use it to fetch good pron, I promise, but now I'm looking for something that check my Dancer2 App doing the right redirect!

Test is just a bit more complex:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->is_success and $res->previous);

Call ends with a 200 but a previous URL was registered and we're redirecting from there. You can see this URL in $res->previous but knowing it's there is enough to say we're redirecting.

Something more about the browser powers of LWP::UserAgent?
What if you need to test sessions?  How Dancer2 trace client sessions? With a session cookie, obviously. And LWP::UserAgent can manage them too? Obviously again, but remember to give it  a cookie_jar to use:

$ua->cookie_jar({file => "cookies.txt"});

Rememeber. With LWP::UserAgent you're not just trying out your site. You're literally surfing it!

Dancer2::Test. undiscovered countries

If you want to do serious developement, talk with serious people and produce serious packages you have to use testing scripts A LOT. Dancer2 comes with its Dancer2::Test library, a lot of good features... and few gloomy mysteries.

Let me start with a little tip: whenever you start testing import Dancer2::Test and specify which apps you're going to use:

use Dancer2::Test apps => ['app1', 'app2'];

Probably you already knew it, but I had to dig up a little in Dancer2 docs to find that this is a "must have" for Dancer2::Test and everytime I spend time to find something I think that what I find deserves few words on this blog. When you'll search for it like me, I'll be there, on the first page of Google, waiting for you.

But I was talking about gloomy mysteries so now we'll talk about strage behaviours of Dancer2::Test, triggered by nasty (but legit) coding choices.

In my first article I talked about before hooks and forward.

See this example:

hook before => sub {
      my $context = shift;
      if (request->path_info eq '/ghost_path' )
      {
           $context->response( forward('/solid_path' ));
           $context->response->halt;
      }
}

You obviously defined a dispatcher for '/solid_path' but there's no reason you have to define a dispatcher for '/ghost_path'. Still, calling '/ghost_path' will return a page (HTTP code 200! Win!).

Now we'll write a test for this:

response_status_is ['GET' => '/ghost_path'], 200;

Result is:

#          got: '404'
#     expected: '200'

What? Aaaargh, Wotan worshipper rises is axe!

Dancer2::Test does a bit of introspection in your Dancer2 App. When you ask for a path it simply tries to find it in the defined dispatchers, where there isn't. It can't deduce that all will go well thanks to the before hook so the test fails.

But. Wotan. Worshipper. Wants. To. Test!

Well, servers never lie. Never. If the test could be done with a real server and not just... playing with pms...

Here comes Test::TCP.


use strict;
use warnings;
use Test::More;

use Test::TCP;
use LWP::UserAgent;


Test::TCP::test_tcp(
    client => sub {
        my $port = shift;
        my $ua = LWP::UserAgent->new;
        my $res = $ua->get("http://127.0.0.1:$port/ghost_path/");
        ok($res->is_success);
    },
    server => sub {
        my $port = shift;
        use Dancer2;
        use multilang;
        set(show_errors  => 1,
            startup_info => 0,
            environment  => 'developement',
            port         => $port
            );

        Dancer2->runner->server->port($port);
        start;
    },
);
done_testing;


This is a real server launched by a script and reached by a real client. It's just as the actual site. And the test goes well...

What we learned today? Running a server and click in a browser is the only reliable way to test something. Let a machine doing it by itself without all the boring part (open a browser... digit the url... go on Facebook while loading...) is good.
Obviously, in many many cases Dancer2::Test, plain and simple, is enough and it's the right way to do things, because it's a bit less complex working with it. But keep an eye on exceptions...

The dawn of the combining robots

Yesterday I wanted to say a lot of things about Dancer2 and Apps, but I was a bit sleepy, so I keep the discussion on simple topics. Now I want to talk about something I experienced in the passage from Dancer 1 to Dancer 2 and how it fooled me.

When I started using Dancer 1 I was a young (well, younger than now) and I did dirty things, as young people do. Things like putting Dancer routes directly in the bin/app.pl script.
Talking about Dancer 1 this is not a really dirty thing, Dancer 1 declarations are always global so the problem is just "esthetic".

Routes under lib directory, in a perl module, give a feeling of tidiness, nothing more.

Problem is that, working this way, you start thinking about bin/app.pl as your app, as the center of your site. This is a lot confusing when Dancer2 come along.

You should think about bin/app.pl as just your "launcher", keeping it as empty as you can, using it just to configure what code you want to load in your site.

The dancer2 init script will not help you thinking this way. Launching it you'll have:

• bin/appl.pl with import Dancer2 
• lib/APPNAME.pm with just import Dancer2 :syntax 

What does this mean? It means you're working just with one app, declared in the bin/app.pl and all the code you'll write in pm files will be referred to it. This app will have name "main" and your design will start in a not-so-modular way.

Evil evil Dancer2 init, wanting to keep developers in the Dancer1 global world, hiding the great powers of Dancer2 Apps!

What should you do? In my opinion you should leave import Dancer in the bin/app.pl (you can't make webapp start without it), but you should also use import Dancer in your lib/APP.pm files. Files, yes, more than one, because you should split up functionality (and navigation) in more than one perl module.

Then, your bin/app.pl will become:

import Dancer2 #a "main" app will still be created
import APP1 #some routes and logic
import APP2 #different routes and logic

and so on...

This way, just commentig one line (or uncommenting another) you can turn on/off features, behaviours, navigations and the global structure of your site is easy to understand.

A little thing to close the post. Try sometimes to launch your developement server this way:
DANCER_DEBUG_CORE=1 bin/app.pl
Bootstrap will give you notifications about th Apps start. Here you'll see site structure in a very clean way, something the code itself can't do.

Doctor Dancer2Apps

...or: How I learned to stop worrying and love the framework.

Apps are one of the most gorgeous things introduced by Dancer 2. They allow developers to take very sexy design decisions and encourage modular developement, with a lot of reusable code released on CPAN to make people happy.

Apps are not something new in web frameworks (nothing is actually new in this world). Django, one of the most important Python frameworks, is based on them.
Apps are a really explicit concept in Django. Every time you start a project you must define apps, there's no other way to do it (as in Python zen).

Dancer2 is pure perl, so apps  are a bit more subtle and if you come from Dancer1 you could miss them. There's no App->new, no creation moment, no actual ritual to conjure an app in this dimension.

How do you create an App?

Easy: every module has Import Dancer2 is an App.

Yes, I know, there's nothing simpler than importing a module and you cannot believe something so simple is so powerful but so it is. You import Dancer2 => you create an App.

Then?

Then the world of the modular design is open to you, you can start thinking clever about writing your webapp. You can help world to be a better place... well, a better place where write Dancer websites, at least.

Everything decided inside an App is just inside it and cannot leave it.
Let us thinking about a site where every navigation starting with /funnylayout need a particular layout. You could  check all the routes but why? This is just a "module" of your webapp with a particular behaviour.

Create a packege with a geeky-but-professional name: package MySite::Funny
Make it an app (no chicken blood needed!): import Dancer2
Decide a common prefix for all your navigations: prefix => '/funnylayout'
Set the layout: set layout => 'funny'
That's all, monks!

Now write all the routes you want, knowing they will have the same behaviour. Knowing that if you move them to another site, they'll still have the same behaviour.

Why aren't you excited? Can't you understand? Well, let me speak a little more about Django and one of the most importand feature of it: Django Admin. It's a very fast way to have a backend for a site and let users manage their data.
Dancer's most similar thing is SimpleCRUD. It's just a Dancer Plugin and it does wonderful things but it has problems like HTML interfaces hardcoded in the subs and very little space for improvement.
What if someone start developing and app with all the code of SimpleCRUD in it, made cleaner by a complete use of Dancer2 framework, with templates, plugins and all that stuff? You would just import it to access all your ORM data. You would change it to rewrite its interface or change its flow. Sky would be the limit!

Understand this or go with the Wotan worshipper on the hill, barking to the moon.

Dancer2: changing path_info in a before hook

Dancer2 is a great piece of software, easy to use and smart. But it's still evolving and working with it could be a little rude. Easy tasks could take a lot of time (and patience) with no possibility to straight google your solution out of few keywords.

I wanted to preprocess the calls my Dancer2 site receives in a before hook and change them to have a cleaner dispatching. For example I wanted to receive

/foo/bar

keep /foo for me and make the call processed by

get '/bar'  => sub

(i'll tell you why another day, but you can easly guess).

It's a standard task and this page  has an example about exactly what i want. An example that... well... doesn't work.

I admit I went a bit on rage, howling against all the Dancer2 Gods when I understood that something was wrong but when the angry moment finished I started digging in the documentation finding this issue.

"Argh!" said the blood thirsy Wotan worshipper inside me "They knew it. They fixed it. And It's still not working!"

Probability that the fix wasn't in the CPAN package I downloaded was high, as I said, Dancer2 is still evolving, it' version is still starting with a double 0 and we're all beta-testers, but the time of the pull request said otherwise so I went on digging.

We're lucky people because we live in a test-driven world. Let us see how the test for this fix is written...:

hook before => sub { 

my $context = shift;
return if $context->request->path eq '/default';
$context->response( forward('/default') );
$context->response->halt;

}; 

...and that's the way to do it! I hope someone will find this post before they have to start the voyage I did to find the solution. This will keep the Wotan worshipper calm a bit more...