A new search engine in the making which could beat Google at its game

When it comes to searching information, we are automatically programmed to go to search engines. Even as the current search engine algorithms are great at working with fact-based queries and providing structured answers, they are surprisingly ineffective at answering subjective and personal questions.

It is this gap that IIT Kharagpur professors and Microsoft are trying to fill. They are trying to build a system that can form the basis for a deeper and more meaningful search engine.

Queries based on human experiences and personal opinions are not easy for a standard search engine to comprehend and hence at several instances they fail to answer questions such as – ‘How to make small talk with new friends’, ‘People’s favourite memories from school’, ‘How does it feel to immigrate to a new country and many more’.

The researchers at IIT-K are conducting a study to extract meaningful information from social conversations to help search engines answer social list queries better by deploying artificial intelligence and machine learning. For instance, the team is using multi-word hashtags, basically idioms, from Twitter to conduct a detailed study so that the search engine can give answers more accurately.

The researchers have infact collected around four million hashtags that were trending between January 2015 and June 2015, and used a SVM (Support Vector Machine) classifier to conduct this research. Some of these hashtags were #foreveralone, #awkwardcompanynames, #childhoodfeels, and #africanproblems.

“While traditional search engines may struggle with such deeply human queries there are online platforms specifically tailored for personal opinions and conversations — social media. Twitter, specifically, has become a forum for people to create sustained online conversations held together by a common hashtag,” Microsoft's Senior Applied Researcher Manish Gupta told the media.

"The algorithm used to conduct this study forms the basis for a better search engine for social platforms which can assist users looking for subjective information and trusted opinions,” he said.

Microsoft currently owns search engine bing. Earlier this year, Apple switched the default provider of its web searches for Siri on iOS and Spotlight on the Mac to Google from Microsoft's Bing. That means now if you ask Siri to search something on the internet, you will get Google results instead of Bing on iOS and Spotlight -- a system-wide desktop search feature of Apple's macOS and iOS operating systems. Apple's web browser Safari on Mac and iOS already use Google search as the default provider, thanks to a deal worth billions to Apple (and Google) over the last decade.

Given Microsoft's work to advance the field of artificial intelligence, the company has been confident that Bing would be at the forefront of providing a more intelligent search experience for its customers and partners. However, by associating with IIT-K to roll out a more meaningful search engine, the tech giant is looking to take on rival Google head-on, and find its place back in the market.