Writing HTML to File via Array

[chitownclone]

I am attempting to create a spider to do some data mining on the internet. I want my .pl program to open a page on the web, put the text into an array, and write it to a text file.

Using the code below, I can't seem to get any data into the array, but if I change $url to
"C:/website/myweb.com/file.html" is works fine. What do I have to do to make it work for a web page that I do not know the root C: directory and just know the URL.




$url = "

http://myweb.com/file.html";

open (READ, $url)
@raw_data=<LOGIN>;
close(READ);

open (WRITE, ">>C:/Websites/myweb.com/files/write.txt")
foreach $log(@raw_data) {
print WRITE "$log\n";
}
close (WRITE);

 

[duncdude]

open ([blue]READ[/blue], $url)
    @raw_data=<[red]LOGIN[/red]>;
close([blue]READ[/blue]);

open (WRITE, ">>C:/Websites/myweb.com/files/write.txt")
    foreach $log(@raw_data) {
        print WRITE "$log\n";
    }
close (WRITE);


Kind Regards
Duncan

 

[chitownclone]

Sorry...that wasn't the mistake. I just edited my real code to make it simpler to read. They are all LOGIN in the actual code. So unfortunantly that isn't it.

 

[PaulTEG]

LWP is what you're looking for

Have a look at the documentation on search.cpan.org, also perldoc should have some lit on it

HTH
--Paul

 

[ishnid]

Specifically, look for LWP::Simple. For all kinds of web transactions like this, lwpcook is an invaluable resource (a quick google search'll find it for you).