Stop Words for Search 1

[Itshim]

Greetings,

I am writing a search function for a client and I was wondering if anyone knew of a place I could find a list of common stop words for searches?

I searched on Google and a few other engines, but I cannot find a basic list of words.

Thank for any help in advance.
Itshim

 

[Olavxxx]

stop words?
banned wordlist?

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[ChrisHirst]

DaButcher, stop words are the common words not included in searches. banned words are usually profanities and words related to gambling, p0rn, drugs etc.

Believe it or not it actually illegal to supply profanity lists in some countries so you will never find one published but creating your own shouldn't be difficult
My stop word reg exp pattern is;

$WordList = "can|also|up|what's|where|are|we|by|in|no|and|or|of|it|all|if|at|the|for|what|to|a|has|is|on"

Chris.

Indifference will be the downfall of mankind, but who cares?

http://www.candsdesign.co.uk

A website that proves the cobblers kids adage.

http://www.cram-system.com

Nightclub counting systems

So long, and thanks for all the fish.

 

[Olavxxx]

ChrisHirst: ok.. I havent heard about this before.

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[ericbrunson]

You should add "shit" to the list.

Oops. I just said "shit".

 

[ChrisHirst]

That would be a banned word rather than a stop word <grin>



Chris.

Indifference will be the downfall of mankind, but who cares?

http://www.candsdesign.co.uk

A website that proves the cobblers kids adage.

http://www.cram-system.com

Nightclub counting systems

So long, and thanks for all the fish.

 

[Itshim]

Thanks for the replies, and the word list ChrisHirst.

I actually had not heard of the term "stop words" either, until I got this project.

Thanks again,
Itshim

 

[hads]

I remebered seeing a list of words in an article in passing, here's a link

http://www.wdvl.com/Authoring/Langu...SearchEngine/conclusion.htmle/conclusion.html

also, according to here;

http://www.asp101.com/articles/john/indexserver/noisewords.asp

there is a 'noisewords' list on Windows for the index server. May be worth checking out.

HTH


01101000011000010110010001110011

 

[ChrisHirst]

the wdvl link is broken so it's here instead.



Chris.

Indifference will be the downfall of mankind, but who cares?

http://www.candsdesign.co.uk

A website that proves the cobblers kids adage.

http://www.cram-system.com

Nightclub counting systems

So long, and thanks for all the fish.

 

[Olavxxx]

How does this stoplist work?
Do you remove all occurrances of the <stopword> in the <searchstring> ?

eg. if user searches for:
"can fall", it removes "can" from the searchstring and only searches for "fall" ?

I never thought about implementing this in my searchfunction, but now that it's mentioned, I'm interested..

ps. If I will make a function like this, I will base it on a mysql table, like my smilies script.

it's just as easy and more dynamic

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[ChrisHirst]

that's about it, just replace the stopword with ""

Chris.

Indifference will be the downfall of mankind, but who cares?

http://www.candsdesign.co.uk

A website that proves the cobblers kids adage.

http://www.cram-system.com

Nightclub counting systems

So long, and thanks for all the fish.

 

[hads]

Huh, thanks for fixing that link ChrisHirst, I knew I should have previewd that post!


01101000011000010110010001110011

 

[Bastien]

http://www.weberdev.com/get_example-3855.html

is a search snippet I wrote...

Bastien

Cat, the other other white meat