A way to ...
The word testing means different things to different people. In many cases, testing is just a means to an end. When we are talking about testing in Perl, we are actually talking more about a development philosophy.
Exercise the code by hand to see if it breaks
There is nothing wrong with ad hoc testing, unless it is the only form of testing you use. This approach (obviously) requires a lot of person-hours to perform. It is also not really suitable for use in verifying code repeatedly. It is very repetitious, which generally means error-prone when executed by humans.
Have the computer do the boring work.
Most of the testing we want to talk about is automated. When the procedure is to run the same test day after day, you really want to hand the test off to a computer. This makes the test more reproducable and, quite frankly, the computer does not get bored and wander off topic.
Many different people have created testing frameworks in hopes of automating away the tedium of tests. Some are complicated. Some are simple. Others are incredibly flexible and customizable.
In keeping with Perl's philosophy, there are many ways to test Perl code as well. We will focus on the most widely used and standardized system in the Perl ecosystem: TAP.
1..4
ok 1 - Initial sanity verified
ok 2 - Sanity still exists
not ok 3 - Sanity has left the building
# Failed test 'Sanity has left the building'
# at examples/sanity.t line 11.
# got: '4'
# expected: '5'
ok 4 - Sanity has been restored
# Looks like you failed 1 test of 4.
This is a very simple example of the output from a TAP-based test. It shows all of the major parts of a TAP test.
#
#!/usr/bin/perl
use Test::Simple tests => 4;
use strict;
use warnings;
my $var = 1;
ok( $var == 1, 'Initial sanity verified' );
ok( $var =~ m/^\d+$/, 'Sanity still exists' );
ok( 2+2 == 5, 'Sanity has left the building' );
ok( 2+2 != 5, 'Sanity has been restored' );
The simplest Perl module supporting TAP is Test::Simple.
It only supplies the ability to set the test plan and a single function
ok(). The ok() function tests a scalar value,
printing ok on true and not ok for a false value.
It also provides the assertion number automatically, and adds a label if provided as a second argument.
#!/usr/bin/perl
use Test::More tests => 4;
use strict;
use warnings;
my $var = 1;
is( $var, 1, 'Initial sanity verified' );
like( $var, qr/^\d+$/, 'Sanity still exists' );
is( 2+2, 5, 'Sanity has left the building' );
isnt( 2+2, 5, 'Sanity has been restored' );
The more advanced Test::More module provides a more
complete set of assertion functions. These don't really do much more
than the straight ok() function (which is also provided)
if the assertion succeeds. But, if the assertion fails, they provide
more useful diagnostic output.
ok() - same as in Test::Simpleis() and isnt() - equalitylike() and unlike() - regex matchcmp_ok() - arbitrary binary comparisonis_deeply() - compare arbitrary data structuresisa_ok() and can_ok() - objects and classesThese are the major test assertion functions provided by
Test::More. The module does provide other functionality, but
we will not be exploring that in an introduction.
Test::DifferencesTest::DeepTest::LongStringTest::Exception or Test::FatalTest::WarnTest::NoWarningsYou are not limited to Test::Simple and
Test::More. There are a large number of specialized modules
that provide other assertions and extend functionality. All of these still
output TAP, so they are compatible with any tools that would recognize that
output.
Test::TimerTest::Perl::CriticTest::WWW::MechanizeTest::WWW::SeleniumThere are also test modules that go far beyond just testing what your methods return.
Testing cannot find all bugs, but not testing cannot find any bugs.
Well-written tests show how the module should be used.
Writing tests makes you use an interface.
Any bug that is covered by a unit test will not come back
(without you knowing).
The old Extreme Programming methodology used the first approach to decide what should be tested.
Most people remember to test the obvious success case. It is equally important to look for places on the edges of good behavior or changes in behavior and test around those areas as well. Edges are a rich source of bugs.
Fewer developers remember to test the failure cases. Do you know how the code acts if it is passed a bad parameter? How about if a file it depends on does not exist? How about permission failures? Testing these areas makes certain code is in place to handle the sorts of surprises that happen in the real world. These corners of code are often never tested except by unit tests. So the first time they may execute is in production.
Almost nobody remembers to test their assumptions. These assertions almost never fail. When they do, they tend to generate very strange behavior that may take days to unravel.
Look at test code from SVG::Sparkline.
These files are real unit tests copied from the
SVG::Sparkline module. They are tests for real code, but are
not much more complex than the simple examples from before.
The Devel::Cover module helps determine what code is executed.
Together with a good test suite, we can increase confidence in our code.