LWP::UserAgent: a browser

Last time we had a lot of fun playing with Test::TCP and LWP::UserAgent, but we're not done yet with these toys, there's something more I want to say about them.

So we can test our Dancer2 route:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/route/");

ok($res->is_success);
is($res->content, 'my content');

Sometime this is not enough. In the code I'm developing redirect is important as success, how can I test this? Well, I suppose...


my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->code, 302);

But result is:

# got: '200'
# expected: '302'

What's wrong?

You could think (I did, at last) that LWP::UserAgent is just a "tester", something to do "ping" on internet sites and see what come back. It's not, LWP::UserAgent is powerful because it's a complete working browser, like Firefox, Opera, Chrome or IE. It's not good in rendering pages, ok, but the data-flow it can manage is the same you can see while surfing the web.

What a normal browser like Firefox do when there's a redirect?

Simple:

CALL -> 302 -> NEW PAGE -> 200

Though you're giving back a redirect for a route you still want to give a page, a success result, at the end of the navigation chain. LWP::UserAgent follow all the chain without asking for nothing, as your browser, bringing back the final 200 response.

Ok, LWP::UserAgent is cool and I'll use it to fetch good pron, I promise, but now I'm looking for something that check my Dancer2 App doing the right redirect!

Test is just a bit more complex:

my $ua = LWP::UserAgent->new;
my $res = $ua->get("http://127.0.0.1:$port/redirecting_route/");

is($res->is_success and $res->previous);

Call ends with a 200 but a previous URL was registered and we're redirecting from there. You can see this URL in $res->previous but knowing it's there is enough to say we're redirecting.

Something more about the browser powers of LWP::UserAgent?
What if you need to test sessions?  How Dancer2 trace client sessions? With a session cookie, obviously. And LWP::UserAgent can manage them too? Obviously again, but remember to give it  a cookie_jar to use:

$ua->cookie_jar({file => "cookies.txt"});

Rememeber. With LWP::UserAgent you're not just trying out your site. You're literally surfing it!

Dancer2::Test. undiscovered countries

If you want to do serious developement, talk with serious people and produce serious packages you have to use testing scripts A LOT. Dancer2 comes with its Dancer2::Test library, a lot of good features... and few gloomy mysteries.

Let me start with a little tip: whenever you start testing import Dancer2::Test and specify which apps you're going to use:

use Dancer2::Test apps => ['app1', 'app2'];

Probably you already knew it, but I had to dig up a little in Dancer2 docs to find that this is a "must have" for Dancer2::Test and everytime I spend time to find something I think that what I find deserves few words on this blog. When you'll search for it like me, I'll be there, on the first page of Google, waiting for you.

But I was talking about gloomy mysteries so now we'll talk about strage behaviours of Dancer2::Test, triggered by nasty (but legit) coding choices.

In my first article I talked about before hooks and forward.

See this example:

hook before => sub {
      my $context = shift;
      if (request->path_info eq '/ghost_path' )
      {
           $context->response( forward('/solid_path' ));
           $context->response->halt;
      }
}

You obviously defined a dispatcher for '/solid_path' but there's no reason you have to define a dispatcher for '/ghost_path'. Still, calling '/ghost_path' will return a page (HTTP code 200! Win!).

Now we'll write a test for this:

response_status_is ['GET' => '/ghost_path'], 200;

Result is:

#          got: '404'
#     expected: '200'

What? Aaaargh, Wotan worshipper rises is axe!

Dancer2::Test does a bit of introspection in your Dancer2 App. When you ask for a path it simply tries to find it in the defined dispatchers, where there isn't. It can't deduce that all will go well thanks to the before hook so the test fails.

But. Wotan. Worshipper. Wants. To. Test!

Well, servers never lie. Never. If the test could be done with a real server and not just... playing with pms...

Here comes Test::TCP.


use strict;
use warnings;
use Test::More;

use Test::TCP;
use LWP::UserAgent;


Test::TCP::test_tcp(
    client => sub {
        my $port = shift;
        my $ua = LWP::UserAgent->new;
        my $res = $ua->get("http://127.0.0.1:$port/ghost_path/");
        ok($res->is_success);
    },
    server => sub {
        my $port = shift;
        use Dancer2;
        use multilang;
        set(show_errors  => 1,
            startup_info => 0,
            environment  => 'developement',
            port         => $port
            );

        Dancer2->runner->server->port($port);
        start;
    },
);
done_testing;


This is a real server launched by a script and reached by a real client. It's just as the actual site. And the test goes well...

What we learned today? Running a server and click in a browser is the only reliable way to test something. Let a machine doing it by itself without all the boring part (open a browser... digit the url... go on Facebook while loading...) is good.
Obviously, in many many cases Dancer2::Test, plain and simple, is enough and it's the right way to do things, because it's a bit less complex working with it. But keep an eye on exceptions...

The dawn of the combining robots

Yesterday I wanted to say a lot of things about Dancer2 and Apps, but I was a bit sleepy, so I keep the discussion on simple topics. Now I want to talk about something I experienced in the passage from Dancer 1 to Dancer 2 and how it fooled me.

When I started using Dancer 1 I was a young (well, younger than now) and I did dirty things, as young people do. Things like putting Dancer routes directly in the bin/app.pl script.
Talking about Dancer 1 this is not a really dirty thing, Dancer 1 declarations are always global so the problem is just "esthetic".

Routes under lib directory, in a perl module, give a feeling of tidiness, nothing more.

Problem is that, working this way, you start thinking about bin/app.pl as your app, as the center of your site. This is a lot confusing when Dancer2 come along.

You should think about bin/app.pl as just your "launcher", keeping it as empty as you can, using it just to configure what code you want to load in your site.

The dancer2 init script will not help you thinking this way. Launching it you'll have:

• bin/appl.pl with import Dancer2 
• lib/APPNAME.pm with just import Dancer2 :syntax 

What does this mean? It means you're working just with one app, declared in the bin/app.pl and all the code you'll write in pm files will be referred to it. This app will have name "main" and your design will start in a not-so-modular way.

Evil evil Dancer2 init, wanting to keep developers in the Dancer1 global world, hiding the great powers of Dancer2 Apps!

What should you do? In my opinion you should leave import Dancer in the bin/app.pl (you can't make webapp start without it), but you should also use import Dancer in your lib/APP.pm files. Files, yes, more than one, because you should split up functionality (and navigation) in more than one perl module.

Then, your bin/app.pl will become:

import Dancer2 #a "main" app will still be created
import APP1 #some routes and logic
import APP2 #different routes and logic

and so on...

This way, just commentig one line (or uncommenting another) you can turn on/off features, behaviours, navigations and the global structure of your site is easy to understand.

A little thing to close the post. Try sometimes to launch your developement server this way:
DANCER_DEBUG_CORE=1 bin/app.pl
Bootstrap will give you notifications about th Apps start. Here you'll see site structure in a very clean way, something the code itself can't do.

Doctor Dancer2Apps

...or: How I learned to stop worrying and love the framework.

Apps are one of the most gorgeous things introduced by Dancer 2. They allow developers to take very sexy design decisions and encourage modular developement, with a lot of reusable code released on CPAN to make people happy.

Apps are not something new in web frameworks (nothing is actually new in this world). Django, one of the most important Python frameworks, is based on them.
Apps are a really explicit concept in Django. Every time you start a project you must define apps, there's no other way to do it (as in Python zen).

Dancer2 is pure perl, so apps  are a bit more subtle and if you come from Dancer1 you could miss them. There's no App->new, no creation moment, no actual ritual to conjure an app in this dimension.

How do you create an App?

Easy: every module has Import Dancer2 is an App.

Yes, I know, there's nothing simpler than importing a module and you cannot believe something so simple is so powerful but so it is. You import Dancer2 => you create an App.

Then?

Then the world of the modular design is open to you, you can start thinking clever about writing your webapp. You can help world to be a better place... well, a better place where write Dancer websites, at least.

Everything decided inside an App is just inside it and cannot leave it.
Let us thinking about a site where every navigation starting with /funnylayout need a particular layout. You could  check all the routes but why? This is just a "module" of your webapp with a particular behaviour.

Create a packege with a geeky-but-professional name: package MySite::Funny
Make it an app (no chicken blood needed!): import Dancer2
Decide a common prefix for all your navigations: prefix => '/funnylayout'
Set the layout: set layout => 'funny'
That's all, monks!

Now write all the routes you want, knowing they will have the same behaviour. Knowing that if you move them to another site, they'll still have the same behaviour.

Why aren't you excited? Can't you understand? Well, let me speak a little more about Django and one of the most importand feature of it: Django Admin. It's a very fast way to have a backend for a site and let users manage their data.
Dancer's most similar thing is SimpleCRUD. It's just a Dancer Plugin and it does wonderful things but it has problems like HTML interfaces hardcoded in the subs and very little space for improvement.
What if someone start developing and app with all the code of SimpleCRUD in it, made cleaner by a complete use of Dancer2 framework, with templates, plugins and all that stuff? You would just import it to access all your ORM data. You would change it to rewrite its interface or change its flow. Sky would be the limit!

Understand this or go with the Wotan worshipper on the hill, barking to the moon.

Dancer2: changing path_info in a before hook

Dancer2 is a great piece of software, easy to use and smart. But it's still evolving and working with it could be a little rude. Easy tasks could take a lot of time (and patience) with no possibility to straight google your solution out of few keywords.

I wanted to preprocess the calls my Dancer2 site receives in a before hook and change them to have a cleaner dispatching. For example I wanted to receive

/foo/bar

keep /foo for me and make the call processed by

get '/bar'  => sub

(i'll tell you why another day, but you can easly guess).

It's a standard task and this page  has an example about exactly what i want. An example that... well... doesn't work.

I admit I went a bit on rage, howling against all the Dancer2 Gods when I understood that something was wrong but when the angry moment finished I started digging in the documentation finding this issue.

"Argh!" said the blood thirsy Wotan worshipper inside me "They knew it. They fixed it. And It's still not working!"

Probability that the fix wasn't in the CPAN package I downloaded was high, as I said, Dancer2 is still evolving, it' version is still starting with a double 0 and we're all beta-testers, but the time of the pull request said otherwise so I went on digging.

We're lucky people because we live in a test-driven world. Let us see how the test for this fix is written...:

hook before => sub { 

my $context = shift;
return if $context->request->path eq '/default';
$context->response( forward('/default') );
$context->response->halt;

}; 

...and that's the way to do it! I hope someone will find this post before they have to start the voyage I did to find the solution. This will keep the Wotan worshipper calm a bit more...