×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

simple dom problem

simple dom problem

simple dom problem

(OP)
Im search some html for the £4.50 out of this...

CODE

<span id="fshippingCost" class="notranslate sh-cst "><span>£4.50</span> 

using this code

CODE

foreach($html->find('span[id=fshippingCost]') as $shippingcost)
	{
		$arrayresults[] = $shippingcost->plaintext;
		$shippingcoststr = $shippingcost->plaintext;
	} 

strangest result back is £17.33 why???

RE: simple dom problem

Using what DOM parser??

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
Simple dom html parser, got other things working im baffled

RE: simple dom problem

Is it possible there's another span with that ID somewhere else in he HTML that has that value?

Are you doing anything else with the array before outputting to screen?

How many elements end up in the $arrayresults array?

----------------------------------
Phil AKA Vacunita
----------------------------------
Ignorance is not necessarily Bliss, case in point:
Unknown has caused an Unknown Error on Unknown and must be shutdown to prevent damage to Unknown.

Web & Tech

RE: simple dom problem

(OP)
I have just changed to this which still comes up with this £17.33 .?? it doesn't exist see above

CODE

foreach($html->find('span[class=sh-cst]') as $key => $info) 
{
	echo ($key + 1).'. '.$info->plaintext."<br />\n";
} 

RE: simple dom problem

(OP)
If I was to make an efficient attempt to attract the £4.50 what would be the simplest method? I have other ones to track also so if I could clinch it that would be marvellous

RE: simple dom problem

(OP)
Ive just done this to make sure im not going mad and I get the correct result so what is the problem with above?

CODE

$html1 = str_get_html('<span id="fshippingCost" class="notranslate sh-cst "><span>£4.50</span>');
	$html1->find('span[class=sh-cst]');
	echo "<br>meeeee".$html1->plaintext."<br>"; 

RE: simple dom problem

(OP)
Ok, what do i search when locating span elements? Should i just use all elements and pull out the id of the one i want? thats not very good if modificatios are made

RE: simple dom problem

Quote:

Should i just use all elements and pull out the id of the one i want?
Yes

Quote:

thats not very good if modificatios are made
And??

That's just the way it is, your task in that case is to detect something has changed and handle it in some way.

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
I'm still getting the wrong value out, im baffled, I use span span span to bring the list down and still its the wrong value

RE: simple dom problem

Have you tried removing the '£' and casting the resulting string to a floating point value before trying any mathematical operations?

Have you done any "debugging 101" steps, such as writing the 'scraped' string value to the browser to verify that you are getting the results you expect from the DOM extraction?

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
I did you first suggestion and it turns out has 17.33 TODAY?? its 17.34?

I did this
http://www.tek-tips.com/viewthread.cfm?qid=7281504

when you say scraping you mean looking for a word? yes and it comes back negative... heres the code, does it need any adjustment?

CODE

if(scraping_for_text($url,'Buy it now'))
   $binstr = "Buy it now";
else
   $binstr = "Not Found";*/

function scraping_for_text($iUrl,$iText)
{
	echo "iUrl=".$iUrl."<br />";
	echo "iText=".$iText."<br />";

    // create HTML DOM
    $html = file_get_html($iUrl);

    // get text elements
    $aObj = $html->find($iText);
    if (count($aObj) > 0)
    {
	   echo "<BR>Num = " . count($aObj) . "<br>";
       $condition = True;
	   echo "<h4>Found ".$iText."</h4>";
    }
    else
    {
	   echo "<BR>Num = " . count($aObj) . "<br>";
       $condition = false; 
	   echo "<h4>No ".$iText." found"."</h4>";
    }    

    // clean up memory
    $html->clear();
    unset($html);

    return $condition;
} 

RE: simple dom problem

(OP)
Ive tried everything, got stupid bug I cannot solve.... how does £4.50 become NOW.... £17.34

RE: simple dom problem

"scraping" is a term used to describe the action of requesting a URL then extracting the text from the response rather than rendering and displaying it in a browser.

What is the URL of the page you are extracting from?

and when you said it is "simple dom html parser", should that be Simple HTML DOM Parser meaning this component
http://simplehtmldom.sourceforge.net/

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum?

or this http://php.net/manual/en/domdocument.loadhtml.php ?

or something else?

RE: simple dom problem

(OP)
Im extracting of many pages of similar format, ie: <span id=thesameineverypage> etc and yes its the component above im using

RE: simple dom problem

Quote:

Im extracting of many pages of similar format,

Just a representative one will do. Without that how exactly are we supposed to figure out all the possible problems?

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
this has never failed...

CODE

foreach($html->find('span[itemprop=price]') as $price)
	{
		$arrayresults[] = $price->plaintext;
		$pricestr = $price->plaintext;
	} 

the above is and its the same idea as above, pull out the span element that has itemprop = price

RE: simple dom problem

Quote:

this has never failed...

Which demonstrates what exactly?

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
I have no idea, both ate there one works one doesnt

RE: simple dom problem


Quote:

I have no idea, both ate there one works one doesnt

Okay, I'll point out what it probably demonstrates then.


It demonstrates that there are differences in the source of the data.

If it works with 'A' but not with 'B' then 'B' is different to 'A'

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
So if there is one instance of span in one file and one instance in another why should it fail chris?

RE: simple dom problem

Quote:

and one instance in another why should it fail

I have absolutely no idea, as I have said a few times (in different threads) I prefer to see evidence and leave absolute certainty without evidence for religion. ... So, ... as stated earlier in the thread, to determine why it fails would require access to the original source data, one that succeeds and [ideally] two that fail.

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
looks at an source file and the areas are same... heres two pages, I cant list the full page its massive

CODE

<div class="u-flL sh-col">
					<div class="sh-gspFirstLine" style="">No additional import charges on delivery.</div>
					<div class="sh-gspSecondLine " >
						This item will be sent through the Global Shipping Programme and includes international tracking. <a href="http://pages.ebay.co.uk/buy/globalshippingprogram/" target="_blank">Learn more<b class="hideforacc">- opens in a new window or tab</b></a></div>
					<div class="sh-gspDashedBorder"></div>
					</div>	
	</div>	

						
<div>
	<div class="u-flL lable" id="shippingPlaceHolderId">Postage:</div>
	<div class="u-flL sh-col">
	<span id="shSummary">
		<!-- TODO:check whether to display none or not -->
					<span id="fshippingCost" class="notranslate sh-cst ">
						<span>£2.85</span>
						</span>
				<span id="fShippingSvc">
				Economy Delivery<!-- GSP -->
						</span>
		<span> | </span>
		<span> 

this is the other page, both give wrong postage

CODE

<div class="u-flL sh-col">
					<div class="sh-gspFirstLine" style="">No additional import charges on delivery.</div>
					<div class="sh-gspSecondLine " >
						This item will be sent through the Global Shipping Programme and includes international tracking. <a href="http://pages.ebay.co.uk/buy/globalshippingprogram/" target="_blank">Learn more<b class="hideforacc">- opens in a new window or tab</b></a></div>
					<div class="sh-gspDashedBorder"></div>
					</div>	
	</div>	

						
<div>
	<div class="u-flL lable" id="shippingPlaceHolderId">Postage:</div>
	<div class="u-flL sh-col">
	<span id="shSummary">
		<!-- TODO:check whether to display none or not -->
					<span id="fshippingCost" class="notranslate sh-cst ">
						<span>£4.50</span>
						</span>
				<span id="fShippingSvc">
				Economy Delivery<!-- GSP -->
						</span>
		<span> | </span>
		<span> 

RE: simple dom problem

Quote:

heres two pages, I cant list the full page its massive


URLs please, so the whole page can be seen not just the bit that you assume it should only find.

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
Not sure if this helps but the original problem, that figure is now £17.35

RE: simple dom problem

(OP)
seems i may of stumped everyone with this problem, i gone back to this that gets the text but its wrong value

CODE

foreach($html->find('span[id=fshippingCost]') as $shippingcost)
	{
		$shippingcoststr = $shippingcost->plaintext;
	} 

RE: simple dom problem

Assuming the one URL provided is a working one, how about ones that do not?

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

Seems to me this is a DOM vs originally loaded HTML issue. The source code browsers show is an initial code, it's not current DOM. Ebay is showing shipping costs depending on your location detected after the pages have loaded. So original value might be for US users only or UK or whatever. Then the browser gets acitve, the shipping costs change and to get the price displayed you'd have to parse the runtime DOM.

With $html = file_get_html($iUrl); you request the original HTML code, you don't get the browsers DOM.No javascript runs and adjusts the DOM to what you see online.

Bye, Olaf.

RE: simple dom problem

(OP)
Ok, OlafDoschke I get what your saying, am I not viewing and/or scraping the source that is delivered to me? ie: the page above, click view source I thought that is wehat I'm working with, what are the mechanics using ebays api's? example to getshippingcost etc.? is this http://php.net/manual/en/domdocument.loadhtml.php pointed out by Chris not better than simpledomparser?

RE: simple dom problem

(OP)
To use this are fees involved?

$apicall = "$f_endpoint?OPERATION-NAME=findItemsAdvanced"
. "&version=$f_version"
. "&GLOBAL-ID=$globalID"
. "&SECURITY-APPNAME=$appID" // replace this with your AppID
. "&RESPONSE-DATA-FORMAT=$responseEncoding"
. "&itemFilter(0).name=Seller"
. "&itemFilter(0).value=$sellerID"
. "&itemFilter(1).name=ListingType"
. "&itemFilter(1).value=$itemType"
. "&paginationInput.entriesPerPage=$maxEntries"
. "&sortOrder=$itemSort"
. "&affliate.networkId=9" // fill in your information in next 3 lines
. "&affliate.trackingId=123456789"
. "&affliate.customId=456";

RE: simple dom problem

(OP)

Quote (chrishirst)

Assuming the one URL provided is a working one, how about ones that do not?

The id of the span element finds what im looking for but last few threads show a different light on the subject

RE: simple dom problem

Browser view source show initial HTML. Once that's loaded into a Browser, it starts processing it, including starting javascripüt functionaliites updateing content here and there.
This is not happening, if you only request the HTML, also if you put it in a DOM model, this is not an active Browser with all its whistles and bells.
So you can only see the initial HTML or it's DOM representation.

Pricing mainly is free, see: http://developer.ebay.com/join/pricing/
You pay for support only, and you "pay" with a rate limit you can vastly expand by passing a compatibility check.

You could also switch to scrape data via client side programming, automating a browser to get 1:1 what you see online, too. The difficulty should be obvious frmo this example: Even if you do so, you also onl get one locale shipping cost. The API gives you a concrete paramterization, you don't rely on the html/css not changing, you querey the official data, etc. So overall this has much more benefits than downsides, even in cases an API costs.

Bye, Olaf.

RE: simple dom problem

(OP)
shipping cost is the only thing left I need, I get everything else, why this is different daily is beyond me... update like the weather man, today its £17.35 its started at £17.33

RE: simple dom problem

(OP)

Quote (OlafDoschke )

Thanks for the info

RE: simple dom problem

The simple fact is that the "foreach" must be finding more than one element that matches the criteria.

Chris.

Indifference will be the downfall of mankind, but who cares?
Time flies like an arrow, however, fruit flies like a banana.
Webmaster Forum

RE: simple dom problem

(OP)
There is only one id for that shipment but commenta off OlafDoschke start another problem, the browser html is not getting exactly what i need according to OlafDoschke, i captcha everything else, een us ifrane to go into the inner page to get the image, shipping is the only problem i can even captcha buy it now

RE: simple dom problem

For example, the asics shoes give me a shipping cost of £11.38 International Priority Shipping to Germany. This will differ for your location, you better use the API simply for the fact this enables you to query the shipping cost for any specific country without the browser making a guess about it via some geolocation machanism via IP or browser request headers, etc. If you do this from your server, it's likely you don't get the shipping cost for the user using your application anyway. And if your PHP runs on a cloud service, you might get different quotes because of making the request from different locations, because your VM(s) can move in many data centers of AWS or Azure or whatever you use.


Chris, the span id really just exists once. Additional to that span there are two javascript calls mentioning the id:

CODE

if(powerBid && powerBid > 0  && (powerBid == 2 ||(powerBid > 5 && powerBid < 15)) ){
  new bidSection("bidBtn_btn", "_OPN_POWB_LAYER", "MaxBidId", "fshippingCost", "impchSummary", "_OPN_ONLOAD_POWB_LAYER", isRedirect,true,{trackBidBtn:"true", bidTrackingId:"p2047675.l4023"},"false","Enter your max. bid");
}else{
  new bidSection("bidBtn_btn", "_OPN_PB_LAYER", "MaxBidId", "fshippingCost", "impchSummary", "_OPN_ONLOAD_PB_LAYER", isRedirect,true, {trackBidBtn:"true", bidTrackingId:"p2047675.l4023"},"false","Enter your max. bid");
} 
These two plus the span id are the three only occurances of fshippingCost, though the shipping cost also is in another html section in the "Postage and payments" tab below the bid section.

This javascript is most probably having a runtime effect on this HTML span. I won't deep dive into how the bidSection initialisation influences this, but most probably this updates or creates the span at runtime.

Bye, Olaf.

Edit: I loaded the html via ajax and it starts with the £11.38 cost, before javascript has hands on it, so that JS might only render this HTML or change it's layout. Anyway, the price is locale specific, as you can see from my test.

RE: simple dom problem

(OP)
I suppose, if im creating this on the fly, i should ask for shipping cost as the only input from the user

RE: simple dom problem

You could do that, but the shipping cost ebay displays to each user depends on auction source country and bidder destination country, so it varies.
Who are your users? Auctioners or Bidders?

Bye, Olaf.

RE: simple dom problem

(OP)
bidders have a control panel that allows them to place ebay url and it imports everything into another bidding site saving time copying pasting etc

... is it possible to use fragments of the api to replace simple dom? ...is loadhtml for php any better that simple dom or they similar

RE: simple dom problem

A bidder is making bids, he's not the owner of the product, so I think you mean auctioners are copying their auctions to other sites. That obviously has the prerequisite it's not the auction of a single item, but since eBay rather has become a shopping mile anyway, that's perhaps not the problem.

I don't know how to exactly answer your question about API vs dom parser, as they don't compare at all. The intention of any API (application programming interface) is to have an open interface where eBay itself interfaces to it's data, but also further applications can extend eBay.

The API is a programming interface answering your questions via methods foreseen to give the detail infos you need in a better quality than you can scrape off, but the approach totally differs from scraping information. With allowance of users, the API even enables starting auctions, changing auctions, everything you might do on the behalf of the eBay user. The API methods are official, directly "wired" to eBays database, so they give concrete infos as eBay stores in their backend(s) to put together the auctions HTML themselves.

Via the API you go directly at what you need (and what eBay offers). I don't think you can get any detail from scraping an auctions HTML, which you can't get from the API. The limit of 500 will hit you with a few users, so you would need to make your application apply to the rules specified, for example about OWASP safety guidance. It's not at all a bad idea to apply these rules to any web application anyway (https://go.developer.ebay.com/compatible-applicati...)

Bye, Olaf.

RE: simple dom problem

(OP)
Hi can I possibly see a very simple call with the api pulling a bit data out by item number please? I have registered but not a clue where to start

RE: simple dom problem

(OP)
below is sample code, but all I want is what I did above, the result of what im searching for, how can I do this with below and break it down to look for lets say, the shipping cost..

CODE

<html>
<head>
<title>eBay Search Results</title>
<style type="text/css">body { font-family: arial,sans-serif;} </style>
</head> 
<body>
<h1>eBay Search Results</h1>
<div id="results"></div>

<script>
function _cb_findItemsByKeywords(root)
{
  var items = root.findItemsByKeywordsResponse[0].searchResult[0].item || [];
  var html = [];
  html.push('<table width="100%" border="0" cellspacing="0" cellpadding="3"><tbody>');

  for (var i = 0; i < items.length; ++i)  
  {
    var item     = items[i];
    var title    = item.title;
    var pic      = item.galleryURL;
    var viewitem = item.viewItemURL;

    if (null != title && null != viewitem)
    {
      html.push('<tr><td>' + '<img src="' + pic + '" border="0">' + '</td>' + 
        '<td><a href="' + viewitem + '" target="_blank">' + title + '</a></td></tr>');
    }
  }
  html.push('</tbody></table>');
  document.getElementById("results").innerHTML = html.join("");
}
</script>


<script src=http://svcs.ebay.com/services/search/FindingService/v1?SECURITY-APPNAME=XXXXX&OPERATION-NAME=findItemsByKeywords&SERVICE-VERSION=1.0.0&RESPONSE-DATA-FORMAT=JSON&callback=_cb_findItemsByKeywords&REST-PAYLOAD&keywords=iphone%203g&paginationInput.entriesPerPage=3>
</script>
</body>
</html> 

RE: simple dom problem

(OP)
the tutorial is not clear at all with parsing the data, it gives a result back for a keyword search, that's not what I want

RE: simple dom problem

You don't get the gist of this.

The code you show is not only making a call, it's also processing the response. It's a full example.
Focus on this line:

CODE

<script src=http://svcs.ebay.com/services/search/FindingService/v1?SECURITY-APPNAME=XXXXX&OPERATION-NAME=findItemsByKeywords&SERVICE-VERSION=1.0.0&RESPONSE-DATA-FORMAT=JSON&callback=_cb_findItemsByKeywords&REST-PAYLOAD&keywords=iphone%203g&paginationInput.entriesPerPage=3>
</script> 

And the core of that aside of the callbasck is not only doable in Javascript, it's a http request of:

CODE

http://svcs.ebay.com/services/search/FindingService/v1?SECURITY-APPNAME=XXXXX&OPERATION-NAME=findItemsByKeywords&SERVICE-VERSION=1.0.0&RESPONSE-DATA-FORMAT=JSON&REST-PAYLOAD&keywords=iphone%203g&paginationInput.entriesPerPage=3 

What other Service APIs are available and what methods they offer is within the reference of all the APIs, what parameters you pass in and what response you get out is described there.
You won't make me read for you. Now is reading time, take a tour, get an overview, look at what you want.

Bye, Olaf.

RE: simple dom problem

(OP)
Now I get it... next question, where do my credentials get entered above? my file is on my server so I need to to knock on the door and let ebay let me in for data

RE: simple dom problem

Well, your PHP will do the service calls, so these urls will never be seen by clients. As with any credentials, eg as with MySQL credentialy, you put them in some config.php you include/require and then use the URLs via PHPs cURL integration. http://php.net/manual/de/book.curl.php

Or do this as ebay demonstrates for PHP: http://developer.ebay.com/devzone/merchandising/do...
I'd not work with globals, but that aside look how they compose the URL call and then use simplexml_load_file($apicalla) to make the call and load the response xml and how you get at the inner data via the result $resp object.

You will need to change from asking an auction URL to asking the users ebay account and let him allow your api account to query in his auctions. You let him pick from the list of his auctions and then can get any info about the picked auction.

Bye, Olaf.

RE: simple dom problem

(OP)
Thank you to everyone for the help.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close