I was just debugging my crawler again for the Parked Domains Project. Every now and then it still dies despite appropriate eval wraps around areas that could die.
For a super weird example, try this (it's in Perl):
The print statement never executes! I'm not really sure as to why, though I traced it pretty far as is explained in the bug report I just filed. It seems to have something to do with line 208 of LWP/Protocol/http.pm:
Follow me on Twitter.
Or in Google reader.
Or by email.
Oh, and tell me what I should blog about: yegg@alum.mit.edu
use LWP::UserAgent;
my $ua = new LWP::UserAgent();
my $response = '';
eval {
local $SIG{ALRM} = sub { die "alarm\n" };
alarm(10);
my $request = HTTP::Request->new('GET','https://www.whatsyourriskscore.info/');
$response = $ua->request($request);
alarm(0);
};
print "TEST!\n";
# Since this just writes out the header block it should almost
# always succeed to send the whole buffer in a single write all.
my $n = $socket->syswrite($req_buf, length($req_buf));
This, in turn, has something to do with this line in IO/Socket/SSL.pm:
$written = Net::SSLeay::write_partial( $ssl,$offset,$length,$$buffer );
Exactly what, I'm not sure, and am not inclined to dig deeper and find out until I hear back about my bug report. If you have any ideas, please let me know. Otherwise I'll put an update in the comments when I find out more.
