Information Overload

I have been hearing and reading a lot about this term "Information overload" recently. Just today, there was a post on Greg Linden's blog about it.There is no doubt that is a problem that is increasing in magnitude every single day, even as I write this post adding to the problem.

Most of the suggestions to solve this problem have been mainly related to better understanding the intent of the user, using the user's history and clickstream to further present the most relevant results. Google has recently started this and Findory has been doing this for quite some time. Although these work well in certain cases, I personally do not think this is the holy grail.

We here at i411 believe that there is only so much you can do to infer a user's intent. We prefer to give the user options and engage him in a dialog and give the user options so they can guide themselves to the right set of results. This approach works very well , especially when there is some structure in the data which you can take advantage of to guide the user. This is a trickier problem to solve for the web but not many people have attempted it. Vivisimo with their Clusty have attempted a solution with not much success and Google and MSN have also been researching with clustering technology but have not been able to bring it to the mainstream yet.

Some other potential combinations of solutions to this problem would also include

• New visualization techniques that would enable us to get a good sense of all the information presented to us quickly and intuitively.
• Authoritative source of information - The web has made each one of us a journalist and there is no pecking order among the sources of information. Maybe we need authoritative sources of information which take precedence.

An interesting quote to end the post -

"We are drowning in information but are starving for knowledge"