T-6: Semantic Indexing and Retrieval of Video
Monday Afternoon, June 23, 14:30 - 17:30
Presented by
Marcel Worring and Cees Snoek, University of Amsterdam
Abstract
The semantic gap between the low level information that can be derived from the visual data and the conceptual view the user has of the same data is a major bottleneck in video retrieval systems. It has dictated that solutions to image and video indexing could only be applied in narrow domains using specific concept detectors, e.g., “sunset” or “face”. This leads to lexica of at most 10-20 concepts. The use of multimodal indexing, advances in machine learning, and the availability of some large, annotated information sources, e.g., the TRECVID benchmark, has paved the way to increase lexicon size by orders of magnitude (now 100 concepts, in a few years 1,000). This brings it within reach of research in ontology engineering, i.e. creating and maintaining large, typically 10,000+ structured sets of shared concepts. When this goal is reached we could search for videos in our home collection or on the web based on their semantic content, we could develop semantic video editing tools, or develop tools that monitor various video sources and trigger alerts based on semantic events.
This tutorial lays the foundation for these exciting new horizons. It will cover basis video analysis techniques and explain the different methods for video indexing. From there it will explore how users can be given interactive access to the data. For both indexing and interactive access TRECVID evaluations will be considered.
Speaker Biographies
Marcel Worring is Associate Professor of Computer Science at the University of Amsterdam, The Netherlands. His main research interests are in (semantic) video indexing and interactive search techniques. He is the chair of the IAPR TC12 on Multimedia and Visual Information Systems. He was co-chair of the Conference on Image and Video Retrieval (CIVR 2007), co-organizer of the First International Workshop on Image Databases and Multi Media Search (1996), the International Conference on Visual Information Systems (1999) and the Conference on Multimedia & Expo (ICME, 2005). He is an associate editor of IEEE Transactions on Multmedia. He was guest editor of the special issue on Semantic Image and Video Indexing in Broad domains for IEEE Transactions on Multimedia (2007). He is leading the successful MediaMill team which has been participating from the beginning of the TRECVID benchmark.
Cees Snoek received the M.Sc. degree in business information systems (2000) and the Ph.D. degree in computer science (2005) both from the University of Amsterdam, The Netherlands, where he is currently a senior researcher at the Intelligent Systems Lab Amsterdam. He was a Visiting Scientist at Informedia, Carnegie Mellon University, USA in 2003. His research interests focus on multimedia signal processing and analysis, statistical pattern recognition, content-based information retrieval, and large-scale benchmark evaluations, especially when applied in combination for multimedia understanding. Dr. Snoek is the lead architect of the award-winning MediaMill Semantic Video Search Engine, which obtained state-of-the-art performance in recent NIST TRECVID evaluations. He was the local chair of the 2007 International Conference on Image and Video Retrieval in Amsterdam.