Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations bkrike on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Help with HTTP requests...

Status
Not open for further replies.

Numbski

MIS
Mar 27, 2001
56
US
I've got a doozie on my hand. I need to be able to do the following:

1. Request a text document via HTTP, open it and loop through it searching for strings. I've been reading the documentation for ActivePERL and I haven't been able to get it working.

2. Request a binary file via HTTP and save it to a pre-specified location on the hard drive.

3. Be able to set up a list of requests, and be able to queue them so only 2 go at a time. If I set up an array of URL's I want to download and just use a foreach() command to do them all, it'll try to do them all at once. I want them to fire two at a time, when one completes begin the next.

I have absolutely NO clue of how to do these things. :( I've looked around for code samples and have run into very limited success. Anyone?
 
I'm guessing that more info from you migh help.

1. What platform are you on? I assume Windows or NT
since you need ActivePerl, but am I right?

2. package LWP::Agent or LWP::Simple might be useful for
retrieving web pages - then there are other packages
that can help you parse the HTML.

On Perl 5.6.0 on Linux, I can do "perldoc LWP::Simple" to get a description of the LWP::Simple module. Once you have ActivePerl installed, I believe you can see the perldocs for LWP::Simple by bringing up a DOS window and doing "perldoc LWP::Simple" at the command line - assuming LWP::Simple is included in the version of ActivePerl that you're installing. If not, you can get LWP::Simple from CPAN(
3. not sure why you need to queue requests - you could set up an array of request URL's, and request each of them in turn, but I don't know how you'd queue them. Can you give more info on this?

HTH.
Hardy Merrill
Mission Critical Linux, Inc.
 
I need to queue the requests because the server on the other end can only send 2 at a time. These are rather large binary files that will take some time to download. If I try to get them all going at once from array cycling in turn, most of them will time out.

I think I've figured out how to save the binary files. If I recieve the binary file in an array, I would open it's filename in the location I need it. Then do a foreach{print} on it. Close the file.

You don't happen to have a code snippet on how to request a document and store it in an array, each line an element?
 
FYI...it's being coded in active perl on NT, and will be compiling a binary for Win32 and Linux.
 
We'll make this a little more simple. The only thing I need to do now is save a binary file to the hard drive. I got everything else figured out. Using LWP::Simple is great BTW. :)

Here's what's happening:

$binary_file=get("
I try to do this:

open (ROMFILE,">$rom_filename") || die "Can't Open $rom_filename: $!\n";

print ROMFILE $binary_file;
close(ROMFILE);

This does in fact save the data to a specified filename, HOWEVER...it's corrupted. I've tried doing binmode(print ROMFILE $binary_file);, but that doesn't help. :(
 
Have you tried just doing binmode on the filehandle?
Code:
open (ROMFILE,">$rom_filename") || die "Can't Open $rom_filename: $!\n";
binmode ROMFILE;
print ROMFILE $binary_file;
close(ROMFILE);
Tracy Dryden
tracy@bydisn.com

Meddle not in the affairs of dragons,
For you are crunchy, and good with mustard.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top