User Agent Automation -> Where to start

[1DMF]

Hello,

I've been asked if I can write a script to visit a website, and navigate a section of it as if it was a human, so I can do some screen scraping and compile a DB of the results.

Is this possible with PERL, and where would I start?

How easy is it to do?

I assume I'd use LWP or MECHANISE to automate the clicks as if a human had clicked the hyperlink.

is this correct?

Any guidance and advice is appreciated, i'm for ever being thrown in the deep end!

Oh well guess it's never a dull momment at work

regards,

1DMF

"In complete darkness we are all the same, only our knowledge and wisdom separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!

 

[1DMF]

Well I'm trying Win32-IE-Mechanize

but all I keep getting is

Can't call method "LocationURL" on unblessed reference at C:/Perl/site/lib/Win32/IE/Mechanize.pm line 283.

Why everytime I try any win32 module it errors with some unblesssed reference?

"In complete darkness we are all the same, only our knowledge and wisdom separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!

 

[1DMF]

I got

http://WWW:Mechanize

to work in the end.

I give up with Win32 Modules , the only one I've ever got to work is the Win32::ODBC module!

"In complete darkness we are all the same, only our knowledge and wisdom separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!