LWP and parsing a URL

[awingnut]

I need to follow some links using LWP (simple). However, some of the links are relative. Does anyone have an easy way to parse a relative URL to complete it? TIA.

 

[PaulTEG]

Can you give us some examples?

If this is a case of "/cgi-bin/another/dir", you just need to maintain the current domain, and simply prepend to the desired url

$url_full=$domain.$given_url;

a relative url normally starts as "/.." where as fully qualified domain should be "

http://..."

HTH
--Paul

Paul
------------------------------------
Spend an hour a week on CPAN, helps cure all known programming ailments ;-)

 

[Rieekan]

Unless the developer was lazy and made their relative links start with ./blah/blah.html (start from the current directory and move from there). To capture that, you'd want to also store the path from the doc root to the current file.

- George

 

[awingnut]

In this case the relative links start with ../blah/blah. What is not clear is how much of the full URL I need to store. For example:

http://www.domain.com/dir1/dir2/dir3/page.html

On that page is the URL:

../dir3a/another.html

Then suppose the URL is:

../../dir2a/third.html

Is it legal to simple append the relative URL to the current one sans the file name?

http://www.domain.com/dir1/dir2/dir3/../dir3a/another.html

 

[PaulTEG]

IN a situation like this, suck it and see, but there's no guarantee it'll work on all servers because they may be configured to redirect differently.

It's such a general problem, my guess is you're going to either have to spend a lot of time looking for a generic answer, or play about with a few variants, and put them in a compound if, and depending on the status (ie 404) try another tack.

Just another €0.02

Paul
------------------------------------
Spend an hour a week on CPAN, helps cure all known programming ailments ;-)

 

[awingnut]

Thanks. Not very satisfying but at least I was right in my understanding of the problem. You are correct, it will require a lot of complex code to try and cover as many bases as possible.