Can't get a web-file with Curl in certain ways

[sulacco]

Hi dear sirs. Would be short. I have a problem getting a file from net with Curl.

First and interesting when i write

http://www.yell.ru/quickrubr.asp

in my browser MSIE 6.0 i get the page all right.

but when i'm trying it to like this:

//=================
<?php
echo ( GetCurlPage(&quot;

http://www.yell.ru/quickrubr.asp&quot;));

function GetCurlPage ($pageSpec) {
$ch = curl_init();
$user_agent=&quot;Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)&quot;
curl_setopt ($ch, CURLOPT_URL, &quot;$pageSpec&quot;
curl_setopt ($ch, CURLOPT_USERAGENT, $user_agent); //i have to make as if i'm MSIE
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$tmp = curl_exec ($ch);
curl_close ($ch);
$tmp = preg_replace('/(?s)<meta http-equiv=&quot;Expires&quot;[^>]*>/i', '', $tmp);
return $tmp;
}
?>

//=================

I get nothing but error 404 and frames. Try it yourself if you like this snippet is ready.
And if you will try to get page

http://www.yell.ru

you will get it with my script.
Do you know what's the problem?

 

[sleipnir214]

You get frames because the URL to which you are pointing is a frameset.

If you want the text from one of the two frames, I recommend that you point your code to the &quot;src&quot; attribute of the frame from which you want content.


Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[sulacco]

Well, how one can make it?

 

[sleipnir214]

I do not understand your question.

Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[sulacco]

I want to get the content from one of the frames but how do
i know what attribute the frame have? I just know

http://www.yell.ru/quickrubr.asp

which i'd like to get. But
with Curl i can't (maybe cat but don't know how). As you
can see in broweser you can easily get the needed page but
not with Curl. I think i miss something maybe there some
kind of session which don't accept me. Maybe cookies.

 

[sleipnir214]

Not cookies. Not sessions.

When your browser hits a page that is a frameset, it looks at the &quot;src&quot; attributes of each of the <frame> tags in the frameset and automatically fetches and displays the sub-pages. cURL does not do that.


Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[sulacco]

Well i see. I've found this src=&quot;center_search.asp&quot; in one
of the frame so i've changed

http://www.yell.ru/center.search.asp

but still nothing. You were speaking that curl don't get
contents in frames. So how get off the mess?

 

[sleipnir214]

I don't know what to tell you. It seems to work for me.

I took your posted code and modified it to point to

http://www.yell.ru/center_search.asp

and to write the HTML to a file instead of the browser. I then compared the content of the file the script produced to what I see when I perform &quot;View Source&quot; on the right-hand frame.

The two HTML streams were identical.

Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[sulacco]

Let me see...

 

[sulacco]

Don't understand. I've got just 1 after connecting with
changed code. Can you show me your modified code, please?

 

[sleipnir214]

Sure.

<?php
error_log (GetCurlPage(&quot;[URL unfurl="true"]http://www.yell.ru/center_search.asp&quot;),[/URL] 3, '/tmp/rus.txt');

function GetCurlPage ($pageSpec)
{
	$ch = curl_init();
	$user_agent=&quot;Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)&quot;;
	curl_setopt ($ch, CURLOPT_URL, &quot;$pageSpec&quot;); 
	curl_setopt ($ch, CURLOPT_USERAGENT, $user_agent); //i have to make as if i'm MSIE
	curl_setopt ($ch, CURLOPT_HEADER, 0); 
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	$tmp = curl_exec ($ch); 
	curl_close ($ch); 
//	$tmp = preg_replace('/(?s)<meta http-equiv=&quot;Expires&quot;[^>]*>/i', '', $tmp);
	return $tmp;
}
?>

When I examine the file /tmp/rus.txt, I get exactly the same HTML I do when I perform &quot;View Source&quot; on the right-hand frame.

Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[sulacco]

It works!! Thank you very much. I grabbed the page!!!
Tnx