Here's Waldo: content-based image retrieval - applications

A picture is worth a thousand words, and object for object, pictures are several orders of magnitude larger and more subtle information carriers than written language. Video, a dense stream of still images, increases the difficulty of information retrieval several magnitudes beyond text or even individual images or graphics. But content-based image retrieval is critical to many applications, and similar in principle to much text retrieval, and useful image search systems are emerging from such vendors as IBM, Excalibur, and Virage.

All offer products or toolkits that effectively address tasks essential to image retrieval, including preparing image indices and abstracting appropriate attributes such as color, texture, and shape. Many systems also include keywords associated with each image, since we still often search by categories such as artist's name, by date, or by concepts like romance.

IBM'S QBIC: MORE THAN ALPHABET SOUP

IBM's image management system, Query by Image Content (QBIC), provides searching of still graphics and video collections based on properties such as shape, texture, sketches, and other attributes. As with any search and retrieval application, results range from surprisingly good to off-the-wall. A query asking to "find me more like this" and using a picture of a solitary red apple could as likely yield more red fruit as it could a red Corvette.

IBM supplements its pure image searching with text searching, and the result--as implemented at its showcase site, the Imagebase at the Fine Arts Museums of San Francisco--supplies what it calls a "word soup" describing each graphic. The soup contains important standard information on each work, such as artist, title, and date, for example, and many of the works also have a free-form description of the image and its contents. This free-form text--the "soup"--is created by having knowledgeable museum volunteers describe the works in a 60-second stream-of-consciousness session, leaving out art and museum jargon. QBIC is also NASA's image retrieval product-of-choice.

QBIC is available for download (http://www.qbic.almaden.ibm.com) with a free 90-day trial license. The download package includes the image indexing and search engine (for AIX, Linux, Windows 95/NT, and OS/2), a Web front end, APIs for embedding QBIC in other applications or extending QBIC with new query functions, and even a sample image collection.

EXCALIBUR'S VISUAL RETRIEVALWARE: NEURAL-LOGICAL IMAGE MANAGEMENT

Since 1980, Excalibur has pioneered content-based retrieval products. Excalibur's flagship products, Excalibur RetrievalWare and Excalibur Visual RetrievalWare, incorporate advanced neural network technologies in the generalized search and retrieval business.

Neural network systems approach search-and-retrieval from an angle that differs from that of traditional text retrieval vendors. Instead of inverted indices pointing to occurrences of words, Excalibur's neural network technology uses what it calls Adaptive Pattern Recognition Processing (APRP). Similar to the way biological systems learn, APRP systems develop an increasingly rich essential notion of objects by analyzing many instances of these objects at different angles or renditions.

APRP applies to patterns, whether in text or images. When the domain is words, then the technique isn't stymied by spelling errors introduced by either humans or OCR processes. The downside is that without a proper set of samples to study, the system may produce unexpected results. Produce the right sample set however, and the system allows you to pose a query by essentially saying "find me more like this."

Excalibur's recent acquisition of Interpix Software Corporation and deals with Yahoo have extended its multimedia search and retrieval reach to Web servers. Users can try out a sample version on the Interpix Web site at http:\\www.interpix.com.

VIRAGE'S IMAGE SEARCH ENGINE LIBRARY: IMAGE (SYSTEM)-BUILDING

A third player named frequently in image content-based retrieval systems, Virage Corporation, is relatively unknown. Virage's product is not an application per se but rather the Virage Image Search Engine, described by the company as an "open framework for building systems." The engine is a static or dynamically linkable library on Sun, Silicon Graphics, Windows, and Macintosh platforms. Moreover, Virage says the engine has no dependencies on specific file systems, I/O mechanisms, or operating systems. This engine offers primitive functions that can be used to enter images into a searchable collection and then to query that collection. What distinguishes the Virage engine is its efficiency and precision in managing image attributes. The four primary attributes in a Visage image collection are color distribution, color placement, structure, and texture.

Virage's most visible licensors include Kodak and PhotoDisc Inc.; also applying the technology is Viisage (http://www.visage.com), a startup provider of biometric identification systems that has developed a successful facial recognition system based on the Virage toolkit. Seven state motor vehicle departments have already begun using Viisage's Virage-based integration services, building "face bases" for future use in law enforcement.

WHERE'S WALDO? THE ANSWER'S IN THE QUESTIONS

Possible uses for still-image retrieval are limitless; the most circumstances in which image management is applied today include stock photos, art galleries and museums, medical diagnoses, and law enforcement and security systems.

None of the solutions available today, whether IBM's, Excalibur's, or Virage's, is perfectly suited to every application. And none is as yet guaranteed to track down that prized hidden face in the crowd--to "find Waldo," as it were. But image content-based retrieval is well on its way, having moved from the lab to the enterprise and the Web to serve effectively a healthy range of practical graphic media management applications.

Robert J. Boeri and Martin Hensel are columnists for Information Insider. Boeri is Information Systems Publishing Consultant at Factory Mutual Engineering of Norwood, Massachusetts. Hensel is founder of Martin Hensel Corporation, a Newton, Massachusetts-based consulting firm that builds SGML-based editorial and production systems for publishers, corporations, interactive services, and compositors.

Comments? Email us at letters@onlineinc.com, or check the masthead for other ways to contact us.

COPYRIGHT 1998 Online, Inc.
COPYRIGHT 2000 Gale Group