Thursday, December 4, 2008

Text mining stop words

Stop Words in text mining are words ignored in a query because they are so commonly used that they can't contribute to relevancy. I think this is a key indicator of why texting, twitter, and sometimes blogging is much more efficient. Essentially we get rid of a lot of the stop words to create more precise statements. One issue with this is that you need to really understand the language very well as well as the person you are communicating with.

Instead of: John went to the store to buy a can of peas it would be: John went store buy can peas. There are much better examples, but that just came off first. Hmmm...

No comments: