Wednesday, February 4, 2009

Google's approach to the semantic web

Google has never expressed a strong interest in the W3C Semantic Web approach. They are interested in developing systems that can understand the content indexed to a greater degree, however. They have to be, if they take the long view, and Google (still) has the resources to take the long view. The most common reason I have heard from Googlers about why they are not working with RDF and OWL is that they still don't seen enough content out there expressed in these languages. I guess if you are Google, tens or hundreds of billions of triples is still not very much.

IDG news service has a story sketching how Google Researcher Targets Web's Structured Data. This is not directed at data published in mahine understandable form (e.g., in RDF), but on other kinds of structured data accessible on the web.
"Internet search engines have focused largely on crawling text on Web pages, but Google is knee-deep in research about how to analyze and organize structured data, a company scientist said Friday. "There's a lot of structured data out on the Web and we're not doing a good job of presenting it to our users," said Alon Halevy during a talk at the New England Database Day conference at the Massachusetts Institute of Technology,

Halevy was referring in part to so-called "deep Web" sources, such as the databases that sit behind form-driven Web sites like or Google has been submitting queries to various forms for some time, retrieving the resulting Web pages and including them in its search index if the information looks useful.

But the company also wants to analyze the data found in structured tables on many Web sites, Halevy said, offering as an example a table on a Web page that lists the U.S. presidents. And there are reams of those tables -- Google's index turned up 14 billion of them, according to Halevy. He "realized very quickly that over 98 percent of these are not that interesting," but even after significant filtering there remain about 154 million tables worth indexing, he said.
ReadWriteWeb also has a story (Google: "We're Not Doing a Good Job with Structured Data")on that Google is or isn't doing with structured data, including an interesting admission by Google researcher Halevy.
"During a talk at the New England Database Day conference at the Massachusetts Institute of Technology, Google's Alon Halevy admitted that the search giant has "not been doing a good job" presenting the structured data found on the web to its users. By "structured data," Halevy was referring to the databases of the "deep web" - those internet resources that sit behind forms and site-specific search boxes, unable to be indexed through passive means."
For some technical details on the issues and current work, see the paper Google’s DeepWeb Crawl by researchers from Google (including Halevy), UCSD and Cornell published in the Proceedings of VLDB 2009.

No comments: