Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Perl mechanize response is only "<HTML></HTML>" with https

I'm kind of new with perl, even newer to Mechanize. So far, when I tried to fetch a site via http, it's no problem.

Now I need to fetch a site with https. I've installed Crypt::SSLeay via PPM.

When I use $mech->get($url), though, this is the only response I get:

"<HTML></HTML>"

I checked the status and success, both were OK (200 and 1).

Here's my code:

use strict;
use warnings;
use WWW::Mechanize;
use Crypt::SSLeay;
$ENV{HTTPS_PROXY} = 'http://username:pw@host:port';
//I have the https_proxy env variable set globally too.

my $url = 'https://google.com';
//Every https site has the same response, 
//so I don't think google would cause problems.

my $mech = WWW::Mechanize->new(noproxy => 0);
$mech->get($url) or die "Couldn't load page";

print "Content:\n".$mech->response()->content()."\n\n";

As you can see I'm behind a proxy. I tried setting

$mech->proxy($myproxy);

but for no avail. I even tried to fetch it into a file, but when I checked it, I got the same response content.

Any kind of advice would be appreciated, since I'm just a beginner and there is still a lot to learn of everything. Thanks!

like image 263
TCruise Avatar asked Aug 30 '26 08:08

TCruise


1 Answers

I think the answer lies here: How do I force LWP to use Crypt::SSLeay for HTTPS requests?

 use Net::SSL (); # From Crypt-SSLeay
 BEGIN {
   $Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL
   $ENV{HTTPS_PROXY} = 'http://10.0.3.1:3128'; #your proxy!
   $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
 }
like image 193
user1126070 Avatar answered Aug 31 '26 22:08

user1126070



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!