Sentiment analysis” delivers context-specific search results

When Mountain View CA search juggernaut Google Inc. materialized out of nowhere eight years ago, the instant results returned by its search engine seemed uncanny in their precision. People were initially stunned at its seeming ability to read their minds. Now that the company has grown into a billion-dollar business, its omniscient and omnipotent presence on the Web is described as God-like. But God is slipping. Google’s brute-force statistical approach to search may not be up to the task much longer. The total amount of digital information available is increasing exponentially, doubling every two years. About two billion new gigabytes are generated annually: 250 megabytes for every man, woman and child on earth. Type the term "information overload" into Google and 14 million results will be returned in .04 seconds. "In five years, if nothing changes we run the risk of living in a chaotic world of information that we can’t make sense of," says Carmi Levy, senior research analyst at the London ON-based Info-Tech Research Group. He points out the big three search engines – Google, Yahoo Inc. and Microsoft Corp.’s MSN – essentially use similar statistical algorithms to rank and list results. Although search technology has been tweaked and refined, no new player has emerged with a radically new mass-market tool to help mortals sift through millions of results. Part of the reason for that is that there is no compelling reason – yet – for consumers to pay for such a tool. Google’s search results are still good enough, and it’s hard to compete with free. Google’s genius was not just in inventing its technology but pioneering the pay-per-click business model that charges advertisers, not users, for content – much like traditional television. "You don’t make money from search," says Levy. "In and of itself, it doesn’t generate revenue, but it does lay the groundwork for the generation of other revenue streams." Consumers may not have reached the breaking point yet, but enterprise customers are prepared to pay handsomely for the next generation of search tool. Businesses are drowning in data and are desperate for tools that can help their people manage information efficiently for competitive advantage. Levy believes that once one search leader is established as an enterprise standard, the crossover to home use will begin, and then to the masses – much like Microsoft Office’s evolution. And the enterprise market is already percolating with some interesting new search players. Enterprise: the next frontier for search?Enterprise search has many names and flavours – business intelligence, enterprise content management, knowledge management – with big players such as IBM Corp., Cognos Inc., and Open Text Corp. offering it as a component of broader business applications. But a promising sub-set for crossover from the business to consumer realms is contextual search, and one Canadian player is making waves with its innovative approach: Nstein Technologies Inc. of Montreal. Nstein’s converged search suite was named one of the top four trend-setting products of 2006 by a leading industry publication, KMWorld. The company’s revenues skyrocketed by 574% from 2001 to 2005, earning it 33rd place in Deloitte & Touche LLP’s 2006 Canadian Technology Fast 50 rankings. Nstein has relationships with IBM and Cognos, and a growing stable of major clients who make their living from content creation and analysis, such as Time Warner Inc., United Press International Inc. and the British Broadcasting Corp. "When I started up the company in 1997, the first person I hired was a linguist from Laval University, not a programmer," says Mario Girard, Nstein’s CEO, explaining what’s at the heart of the company’s approach. Nstein’s patented linguistic DNA (LDNA) technology decomposes content into its smallest building blocks, or "entities" – a place, person, company and so on – to automatically generate an index. It then applies linguistic rules to it, dynamically linking the entities together to extract the meaning of a keyword within its context. For example, if the word "virus" appears in a document that also references "Microsoft" and "patch," the system associates its use with the world of computing instead of medicine. "Traditional search counts the number of times a keyword appears in a document but it’s completely stupid," says Girard. Nstein’s intelligent search derives the possible meanings of a keyword, and returns them to the user to hone the search. Aggregating and presenting the results in a digestible form is another feature of intelligent search, says Laurent Proulx, CTO at Nstein. "At the first interaction, I have to use a shovel in traditional search," he grumbles. Users want to be able to distinguish results in various ways, such as by document type – article, annual report, Q&A, or blog. "I want to be able to see the main clusters and peaks. I don’t care if there are five million results – what does it mean?" he says. Nstein’s latest sentiment analysis technology extends its human-like search capabilities to ascertain not just the meaning of content but its tone as well. With the explosion of consumer-generated content in blogs, organizations are anxious to know what’s being said about their products and services. "We can now separate what is being said objectively and subjectively, positive or negative," says Michel Lemay, VP of marketing at Nstein. He believes an important related trend to watch is the "retribalization" of society, where people create online communities of interest. "Increasingly, they trust each other’s opinions about products more than the companies pushing messages to them," he says. While Nstein is focused on enterprise search, Lemay believes the company’s tools could be adapted to the consumer market. "But I’m not going to pretend we’re the next Google," he says, pointing out there are many challenges in making the crossover. The Tipping PointHowever, cross-fertilization is already occurring in one industry: e-publishing. A case in point is Montreal-based CEDROM-SNi, an aggregator of news content derived from publications in Canada and France for libraries and companies that need media monitoring and research, and part of the Transcontinental Inc. media empire. CEDROM-SNi has moved from a clientele of librarians skilled in the intricacies of Boolean logic and taxonomies to general knowledge workers without any special expertise in finding information, according to François Aird, president of CEDROM-SNi. With Nstein’s tools, indexes are automatically created that allow these customers to analyze results based on a range of factors such as tone, proximity, and placement to sniff out significant news items. Once accustomed to a business tool, these are the types of users who typically bring it home from the office. While text search technology may be evolving steadily, Aird says there is another crucial area that has yet to see a breakthrough: at present, there are no tools that can automatically sift through multimedia files. "Customers are asking us to monitor the news on television and radio," he says. "At some point, we will need a better way to search multimedia files, which are becoming a big part of the media world." To search a multimedia file for a reference to say, "Bombardier Inc.," a manual transcript has to be made first, he explains. Then the text is searched and matched to a time-code in the system where the reference is made, and then the user has to switch to video to capture it. This inelegant process is very much like first-generation search indices were years ago, he says. Without captions, transcripts, and other meta-data that is typically created manually, the source files are not directly searchable. "We have rights to all the CBC’s multimedia content, but we can’t exploit it much today," says Aird. This may have serious implications, as much of the multimedia content generated today may be lost to the future. A parallel problem is being created in the consumer sphere, says Info-Tech’s Levy. People are creating vast repositories of personal digital photos and videos, but this data type is not being managed properly. "All this data is growing exponentially everywhere on hard drives, networks and the Web. Wherever it resides, we will need tools to manage this, as the problem will only get worse in a few years," he says. Since this is one area where both enterprises and consumers can make common cause, this may well be the crossover point where an enterprise search tool that handles multi-media indexing and searching well will be adopted en masse, says Levy. People may even be willing to pay for this search tool or capability directly, instead of the circuitous ad-based model currently in place. This type of search is not entirely blue-sky speculation, at least not on the technology front, says Levy. People hate classifying and meta-tagging files, but the information could be derived automatically. A digital camera with GPS capability can record the time and place the image was taken, for instance, or facial recognition and pattern analysis could be used to identify family members and friends. All this could be done in the background to automatically index the file within the search engine. "Now that’s next-generation search," says Levy. Most of the technology is already there. But the problems lie in the integration of the various components and the economics of mass adoption, Levy says: "So can we roll this out to the billions using the Web at a cost they’re willing to pay? Or have them pay nothing but build an ad-based model around it? Those are questions that will take a few years to shake out."

Sentiment analysis” delivers context-specific search results

Broadcast | October 18, 2006

Unlock all the Canadian telecom, broadcasting and digital media news you need.

FREE TRIAL

SUBSCRIBE

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us