Data Mining Tek-Tips 2

[jnicho02]

Just a general question.

Would it be possible to data mine the back catalogue of Tek-Tips threads?...what would be the problems involved?....and how would you go about it?

The idea would be to produce a Tek-Tips Expert application. My home ---->

http://www.firstlightassociates.co.uk/

visit me for Java and Data Warehousing resources

 

[Predictor]

Yes, this falls under the heading of "text mining". At the least, one could run all of the past messages through a system like WizDoc (from WizSoft) or dtSearch (from the company of the same name). This would allow one to search very quickly and easily, even using conceptual searches (as opposed to just keyword AND/OR searching). There are other, fancier text mining systems but I don't know that they'd provide much more utility in this context than the above.

 

[jnicho02]

Tek-Tips have already indexed the forums, but I'm thinking about how they make a sort of 'ask Jeeves'.

Keyword searches tend to be a bit hit or miss, but then again it is quite complicated to parse sentences and extract sense from them.

Are 'text mining' systems essentially fancy indexers? My home ---->

http://www.firstlightassociates.co.uk/

visit me for Java and Data Warehousing resources

 

[Predictor]

Ask Jeeves is a conceptual search (which is available in both of the tools I mentioned): the system makes some attempt to interpret the user's query, instead of searching for keywords.