Bind DBI columns for speed

From xoa

Jump to: navigation, search

When in a tight loop of many records, using $sth->fetchrow_hashref can be expensive.

For a batch of 500,000 records, the timings look like this:

hashref     took 31.5048 wallclock secs (17.95 usr +  0.48 sys = 18.43 CPU) @  0.05/s (n=1)
array       took 8.83724 wallclock secs ( 6.08 usr +  0.44 sys =  6.52 CPU) @  0.15/s (n=1)
arrayref    took 5.5308 wallclock secs ( 4.44 usr +  0.03 sys =  4.47 CPU) @  0.22/s (n=1)
direct_bind took 4.46956 wallclock secs ( 3.83 usr +  0.02 sys =  3.85 CPU) @  0.26/s (n=1)

Here's the benchmark program I used:

#!/usr/bin/perl

use strict;
use warnings;

use Benchmark ':hireswallclock';
use FLR::DB qw(:sqldo);

FLR::DB::set_instance('TTDEV') || die "Can't set_instance('TTDEV'): $!";

our $sth;

sub prep_handle {
    my $sql = <<"EOF";
    select interestlevel, av_flag, isbn, title
    from testbook
    limit 500000
EOF
    return sqldo_handle( $sql );
}

sub hashref {
    while ( my $row = $sth->fetchrow_hashref ) {
        my $interestlevel = $row->{interestlevel};
        my $av_flag = $row->{av_flag};
        my $isbn = $row->{isbn};
        my $title = $row->{title};
    }
    $sth->finish;
}

sub array {
    while ( my @row = $sth->fetchrow_array ) {
        my ($interestlevel, $av_flag, $isbn, $title) = @row;
    }
    $sth->finish;
}

sub arrayref {
    while ( my $row = $sth->fetchrow_arrayref ) {
        my $interestlevel = $row->[0];
        my $av_flag = $row->[1];
        my $isbn = $row->[2];
        my $title = $row->[3];
    }
    $sth->finish;
}

sub direct_bind {
    $sth->bind_columns( \my $interestlevel, \my $av_flag, \my $isbn, \my $title );
    while ( my $row = $sth->fetch ) {
        # no need to copy
    }
    $sth->finish;
}

for my $func ( qw( hashref array arrayref direct_bind ) ) {
    $sth = prep_handle();
    my $t = timeit( 1, "$func()" );
    print "$func took ", timestr($t), "\n";
}
Personal tools