Coldfusion web-mining

[hpvic03]

I'm tying to create a progam that I can use to get data off of other websites - like things such as weather, stock prices, etc - Things that are variable on the other websites. I found this one program that will do it, but it's written in PHP, so it won't work for me (

http://www.shuetech.com/minetheweb/).

And I know you can request whole web pages using coldfusion, but I just need selected data from selected pages. Anyone have any ideas? Help would be greatly appreciated.

 

[imstillatwork]

there are generally web services or xml feeds made for this sort of thing.

Just mining web data is usually done without permission, and I think that's lame.




Kevin

http://www.lot-o-nothin.com/cfml/

Phase 1: Read the CFML Reference
Phase 2: ???
Phase 3: Profit!

 

[hpvic03]

Ok, say I have permission. Where do I get these web services or xml feeds and can it possibly done in coldfusion?

 

[imstillatwork]

As long as you know what data you want you can get it.

You mentioned you know how to get the full page - you'll have to do that first.

now you have a variable that contains the full page. You can search within that variable for the data you need. refind() will be your friend here I think.

there are numerous sources for data feeds - you've got to start googling for the type you want. weather, sports scores, virus threats, stock tickers, mortgage rate, etc etc... there are brazillions of them out there.


Kevin

http://www.lot-o-nothin.com/cfml/

Phase 1: Read the CFML Reference
Phase 2: ???
Phase 3: Profit!

 

[Sarky78]

I agree with Kevin's first post that it is wrong to do this without permission, especially when there are loads of web services available to do almost all of the things that you see on other websites. Not only this but what if the site owner changes their implementation your site will fail as well.

Take a look at the x-methods site (

http://www.xmethods.com/)

there are loads of examples of web services here that will return xml for you to do with as you please. Failing that Yahoo have quite a few feeds as well and weather.com offer a free weather feed (

http://www.weather.com/services/oap.html?from=homewxanywhere)

hope this helps!

Tony