Excellent news and a tonne of new features.

• new look: nicer html styling and new graph colours with better contrast
• scalability: including running multiple rrd-graph instances in parallel
• multigraph: allowing one plugin to create multiple graphs
• “drill down” graphs: aggregate, then refine data in graphs
• SSL (TLS) support: finally!

Congratulations to all involved. I’m looking forward to playing with the 1.4.x releases.

WhatDoTheyKnow getting too popular! Yay!

Solving the (memory) performance issues more traffic can cause? Boo.

• RailsSpawnMethod conservative
• Monit to kick apache when memory (leaks) got too high
• Profile (memory) usage.
• More efficient memory usage (“text.gsub!” rather than “text = text.gsub”)
• common sense ;)

Your own Amazon ec2 powered open street map tile rendering app (which you can then layer onto top of google maps and more) in just a few clicks.

nifty! (via)

codepad.org

Kinda like pastebin/nopaste, but with interpreter/compiler/execution built in.

You paste code into the website, it is run & executed and you see the results. [that intentionally is designed to fail].

The about page is an interesting read - not least how everything is locked down to make it (hopefully) safe.

From the department of “this took me a while to figure out and googling didn’t help much, so here’s what I did”.

Problem

Combining mod_rewrite (rewriterule / rewritecond) with proxypass directives inside a single virtualhost block in your apache configuration.

Background

This isn’t unique to the problem/solution, but fyi, we have:

• Front end apache, serving static files
• Backend “application” apache, running our mod_perl application

We proxy requests for the application to the backend apache and deliver the rest of the files (css, js, ssi includes) from the frontend.

ProxyPass        /art         !
ProxyPass        /css         !
ProxyPass        /favicon.ico !
ProxyPass        /robots.txt  !
ProxyPass        /            http://localhost:8001/
ProxyPassReverse /            http://localhost:8001/

We were trying to set an environment variable (or setting a cookie would be another use-case) using RewriteRule.

We wanted to conditionally set the rule based on a querystring, so SetEnvIf wasn’t sufficient because it cannot see the query string component of the request.

In general, this can be achieved with:

RewriteEngine on
RewriteCond %{REQUEST_URI} ^/some/foo/
RewriteCond %{QUERY_STRING} .*qs_param=.*
RewriteRule (.*) $1 [E=my_env_var:1]
....
ProxyPass        /art         !

The RewriteRule line is basically saying: set this environment variable but otherwise do nothing.

The problem was that requests for /some/foo/ where 404’ing rather than being proxy passed.

Solution

Can be found in the apache docs for RewriteRule and specifically the section about PassThrough:

This flag is just a hack to enable post-processing of the output of RewriteRule directives, using Alias, ScriptAlias, Redirect, and other directives from various URI-to-filename translators.

… such as ProxyPass. So the fix for us was:

RewriteRule (.*) $1 [PT,E=my_env_var:1]

ps

When trying to figure out what is going on with Rewrite rules, the RewriteLog and RewriteLogLevel options are well worth taking advantage of.

I learnt today (thanks to irc.perl.org #moose) that you can do this in Moose:

has cookies => (
    is  => 'rw',
    isa => 'Bool|Int',  # Bool or Int type
);

You can also state that the type should be an instance of a specific class:

has web_browser => (
    is  => 'rw',
    isa => 'WWW::Mechanize',
);

However, you cannot use the above OR construct with class names:

has web_browser => (
    is  => 'rw',
    isa => 'WWW::Mechanize|My::Test::Mechanize', # FAIL
);

At least, not directly out of the box ….

class_type

The quickest way to crack this nut seems to be with Moose::Util::TypeConstraints class_type.

use Moose::Util::TypeConstraints;
class_type q[WWW::Mechanize];
class_type q[My::Test::Mechanize];
no Moose::Util::TypeConstraints;

and now this will work:

has web_browser => (
    is  => 'rw',
    isa => 'WWW::Mechanize|My::Test::Mechanize',
);

OR … Sub types

Moose::Manual::Types tells us we can create our own sub types, giving an example of a PositiveInt type.

That in turn points to Type Constraint Naming in Moose::Util::TypeConstraints which says:

Since the types created by this module are global, it is suggested that you namespace your types just as you would namespace your modules. So instead of creating a Color type for your My::Graphics module, you would call the type My::Graphics::Types::Color instead

I don’t know if this is the right way to do things, but I now have the following setup:

Defining My::Types

In lib/My/Types.pm I have:

package My::Types;

use Moose::Util::TypeConstraints;

subtype 'My::Types::PositiveInt'
    => as 'Int'
    => where { $_ > 0 }
    => message { "'$_' is not a int > 0" };

subtype 'My::Types::Mech'
    => as 'Object'
    => where {
        $_->isa('WWW::Mechanize')
        || 
        $_->isa('My::Test::Mechanize')
    }
    => message {
        "'$_' is neither WWW::Mechanize or My::Test::Mechanize"
    };

no Moose::Util::TypeConstraints;

1;

Using My::Types

And then within my other code / modules / classes, I can have:

use My::Types;

has web_browser => (
    is  => 'rw',
    isa => 'My::Types::Mech',
);

Which will ensure $thing->web_browser is either a WWW::Mechanize object or my own My::Test::Mechanize object (used for testing purposes).

OR …. MooseX::Types

An (third) alternative approach is MooseX::Types, which allows for having custom types and choosing which ones you import in any one place.

To define your types, in lib/My/Types.pm

package My::Types;

use MooseX::Types
    -declare => [qw<PositiveInt Mech>];

use MooseX::Types::Moose qw<Int Object>;

subtype PositiveInt,
    as Int,
    where { $_ > 0 },
    message { "'$_' is not a int > 0" };

subtype Mech,
    as Object,
    where {
        $_->isa('WWW::Mechanize')
        ||
        $_->isa('My::Test::Mechanize')
    },
    message {
        "'$_' is neither WWW::Mechanize or My::Test::Mechanize"
    };

1;

and then to use it in your code:

use My::Types qw<Mech>;
has web_browser => (
    is  => 'rw',
    isa => Mech,
);

Note that you do not want the quotes around Mech here else it will fail.

Which one to use?

I haven’t a clue. I think I currently favour defining my own subtype through Moose::Util::TypeConstraints but that is possibly just because it was my first working solution.

Comments welcome :)

ps. thanks again to the friendly people on #moose for putting up with my dumb questions about all this.

Easy as 1,2, 3:

• Grab a webkit nightly download
• View the demo
• Tap on your arrow keys, use Space to toggle zoom mode.

Background and videos at satine.org.

Or, for a closed source version but with nicer polish try Cooliris or the Cooliris Firefox extension

Now would someone please find the managers who refuse to let their IT departments upgrade from IE6 and mock and taunt this stupidity until they see sense.

I had cause to want a more recent version of Perl’s libwww-perl (specifically 5.829) than ships in Ubuntu 9.04 (Jaunty).

This post explains how I built a new .deb file to replace libwww-perl_5.820-1_all.deb that ships in Jaunty.

Note that the resulting file:

• passes the libwww-perl tests.
• uploads to my own local apt repository (optional, not required)
• installs cleanly via apt-get or aptitude install on my Jaunty setup

However:

• It is not suitable for uploading to a Debian or Ubuntu apt package repository (mostly because I’ve not gone through all the required steps to ensure it’s compatible with their policies and standards).
• Is supplied as-is, without any warranty. It is what I use, and I’m not aware of any thing bad, but there is at least the possibility this could mess up your system. Caveat emptor.

Overview

The process is thus:

• Grab the current .deb file that ships in Ubuntu 9.04 Jaunty.
• Grab the latest upstream source for libwww-perl.
• Do some un-tarring, directory moving/renaming
• The original deb package contains all the changes debian/ubuntu made to the original source. We re-apply these changes to the newer up-stream source.

It occassionally happens that this doesn’t work cleanly. For instance, if debian had applied a bug fix which wasn’t originally in the upstream source, but the latter version of upstream you are building has applied the patch. The original debian patches will fail because they’ll be trying to re-apply an already applied patch.

If this happens you need to fiddle with the debian patches until they apply cleanly. I’m not covering this here as in my experience this is fairly uncommon, at least when upgrading cpan packages.

Then we:

• Tweak the debian/control and debian/changes meta-files

Process

Ensure you have the deb-src lines in /etc/apt/sources.list and you’ve run “aptitude update”.

Namely, /etc/apt/sources.list contains something like:

deb http://gb.archive.ubuntu.com/ubuntu/ jaunty \
  main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ jaunty \
  main restricted universe multiverse
deb http://gb.archive.ubuntu.com/ubuntu/ jaunty-updates \
  main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ jaunty-updates \
  main restricted universe multiverse

sudo aptitude update
sudo aptitude safe-upgrade

mkdir -p /tmp/jaunty-packages
cd /tmp/jaunty-packages
/usr/bin/apt-get source libwww-perl

wget http://search.cpan.org/CPAN/authors/id/ \
  G/GA/GAAS/libwww-perl-5.829.tar.gz
tar zxvf tar zxvf libwww-perl-5.829.tar.gz

apt-get source libnet-oauth-perl
wget http://search.cpan.org/CPAN/authors/id/ \
  K/KG/KGRENNAN/Net-OAuth-0.19.tar.gz
tar zxvf Net-OAuth-0.19.tar.gz
mv Net-OAuth-0.19 libnet-oauth-perl-0.19

    
    cp: cannot stat `demo/': No such file or directory
    dh_installexamples: command returned error code 256
    make: *** [install-stamp] Error 1
    dpkg-buildpackage: failure: fakeroot debian/rules 
      binary gave error exit status 2

Next, create two copies of the directory, one for testing and one that is required for building.

cp -ai libwww-perl-5.829 libwww-perl-5.829.orig
cp -ai libwww-perl-5.829 libwww-perl-5.829.test

Apply the debian patches:

cd libwww-perl-5.829
gzip -dc ../libwww-perl*.diff.gz | patch -p1
cd ../

Ensure the debian/rules file in the main build dir is executable:

chmod u+x libwww-perl-5.829/debian/rules

The next step isn’t essential, but I find it helps with resolving issues. It generates the debian meta files, and these can be useful to compare and contrast with what you have inherited from the earlier release.

/usr/bin/dh-make-perl libwww-perl-5.829.test
cd libwww-perl-5.829.test
perl Makefile.PL
make test
cd ../

That will shake out any required dependencies and highlight any failing tests. You should take care of those before you continue.

Now back to the primary libwww-perl-5.829 directory.

Take a glance over the debian/control file. You’re primarily looking for missing dependencies or where version numbers are too low. Namely, the earlier version of your module might have depended on Acme::FooBar version 1.2, but the newer version you are building now depends on Acme::FooBar 1.3. You’ll want to ensure the debian/control file reflects this.

vi libwww-perl-5.829/debian/control

Provided you ran the dh-make-perl step above:

• libwww-perl-5.829.test/debian/control

will contain a decent first guess which you can use as a reference. The Makefile.PL or META.yml or similar files in the upstream source directories are also useful places to look for dependency information (and dependency versions).

Now you need to update libwww-perl-5.829/debian/changelog. Copy the previous entry, and update for your needs.

Package version numbers

A note about about the version number in the first line of the changelog file

• (5.829-1~0yourname)

The 5.829 is the upstream version number. -1 is the first package built from that. What about the ~0yourname part?

If debian/ubuntu were to build an official package for this specific version, it would have the version number 5.829-1. This ought to be “newer” than yours (because of the ~0yourname bit), and thus apt will prefer the official version over yours.

If you find a mistake and need to re-build your package, you will need to give it a newer version, but you don’t want to mess with what upstream or debian/ubuntu would use.

• (5.829-1~1yourname)
• (5.829-1~2yourname)
• …

Would be a way to achieve this. The “yourname” part is largely optional, but it helps to make the packages you have built stand out which is sometimes of use.

Finally …

cd libwww-perl-5.829
dpkg-buildpackage -rfakeroot -uc -us
cd ../

If that works, you should find libwww-perl_5.829-1~0yourname_all.deb along with some other files you would need where you going to upload to a local apt repository.

You can install your deb with:

sudo dpkg -i libwww-perl_5.829-1~0yourname_all.deb

All done!

Comments, feedback, questions or suggestions welcome. Preferably in the comments below but otherwise drop us an email.

PS

I’m aware of http://debian.pkgs.cpan.org/ but it doesn’t always have what I want. I also find that the differently named packages sooner or later cause some kinda clash with the packages I get from Debian/Ubuntu.

PPS

I’m also aware of dh-make-perl however that has caused me to trip up previously with version numbers.

For example the version of Spreadsheet::ParseExcel in Ubuntu Jaunty is 0.3300. The latest version on cpan is 0.49 and to make that work it’s version in the debian package needs to be 0.4900. But dh-make-perl picks 0.49.

dpkg --compare-versions 0.3300 lt 0.4900 && echo yes || echo no

Plus, sometimes you want to tinker with the debian control files a little, and I’ve not yet figured out how to have it upgrade an existing module rather than build a new version (and loose all the changelog history from the debian package)

A fabulous article about the Guardian’s MP expenses crowd sourcing experiment

Fabulous in so many ways, not least that the whole is greater than the sum of it’s parts:

• mp expenses
• guardian (not-for-profit org = win!)
• open source
• web frameworks (django, php, ruby on rails, catalyst)
• rapid development
• crowd sourcing
• amazon ec2

On that last point: entire server/hardware cost: £50 (using Amazon ec2) vs “The Guardian has lead time of several weeks to get new hardware”.

And something not explicitly mentioned - really good developers are really worth it. Not just in pay, but in listening to, supporting & giving them a brief, then getting the hell out of the way and letting the sparks fly.

I confess a (very personal) weak spot for the application of IT in media environments. It’s fabulous.

This ought to work for Airport Express devices too, although I’ve not tested it. Ditto it should be similar on a BSD like machine.

On your debian / ubuntu box

• If you have a firewall / iptables setup, enable UDP port 514 from your local network (or at least the IP of the Airport)

• Add the following line to /etc/syslog.conf

  local0.* /var/log/AirPort.log

• To prevent the Airport messages also appearing in /var/log/messages, find the stanza in /etc/syslog.conf that controls that file and add

  !local0.*;\

• Restart your syslog deamon

  sudo /etc/init.d/sysklogd restart

Access the Airport via your Airport Utility

• Advanced tab
• Syslog Destination Address is the IP or hostname of the Debian / Ubuntu linux box that you want to contain the logs. Start with the IP address to get it working, then flip to a hostname if you prefer.

See also AirPort Extreme: Remotely logging base station activity