|Authors: ||Rangachar Kasturi, Dmitry Goldgof, Vasant Manohar, Padmanabhan Soundararajan, John Garofolo, Rachel Bowers, Travis Rose, Jonathan Fiscus, and Martial Michel
2006 NIST/USF Evaluation Resources for the VACE Program - Meeting Data Test
Set Part 2, Linguistic Data Consortium (LDC) catalog number LDC2011V06 and ISBN
1-58563-598-7, was developed by researchers at the Department
of Computer Science and Engineering, University of South Florida (USF),
Tampa, Florida and the Multimodal Information
Group at the National Institute of Standards and Technology (NIST). It contains
approximately twenty hours of meeting room video data collected in 2005 and
2006 and annotated for the VACE (Video Analysis and Content Extraction) 2006
face and person tracking tasks.
VACE program was established to develop novel algorithms for automatic video
content extraction, multi-modal fusion, and event understanding. During VACE
Phases I and II, the program made significant progress in the automated detection
and tracking of moving objects including faces, hands, people, vehicles and
text in four primary video domains: broadcast news, meetings, street surveillance,
and unmanned aerial vehicle motion imagery. Initial results were also obtained
on automatic analysis of human activities and understanding of video sequences.
Three performance evaluations were conducted under the auspices of the VACE
program between 2004 and 2007. In 2006, the VACE program and the European Unions
in the Human Interaction Loop (CHIL) collaborated to hold the
Classification of Events, Activities and Relationships (CLEAR) Evaluation. This was an
international effort to evaluate systems designed to analyse people, their identities,
activities, interactions and relationships in human-human interaction scenarios,
as well as related scenarios. The VACE program contributed the evaluation infrastructure
(e.g., data., scoring, tools) for a specific set of tasks, and the CHIL consortium,
coordinated by the Karlsruhe
Institute of Technology, contributed a separate set of evaluation infrastructure.
To the extent possible, the VACE and CHIL programs harmonized their evaluation
protocols and metrics.
LDC has previously released:
The meeting room data used for the 2006 test set was collected by the following
sites in 2005 and 2006: Carnegie Mellon University (USA), University of Edinburgh
(Scotland), IDIAP Research Institute (Switzerland), NIST (USA), Netherlands
Organization for Applied Scientific Research (Netherlands) and Virginia Polytechnic
Institute and State University (USA). Each site had its own independent camera
setup, illuminations, viewpoints, people and topics. Most of the datasets included
High-Definition (HD) recordings, but those were subsequently formatted to MPEG-2
for the evaluation.
The VACE evaluation tools have been integrated into NISTs downloadable Framework
for Detection Evaluation (F4DE) Toolkit. The toolkit contains small example
files for each of the task/object/domain scoring combinations.
None at this time.
Portions © 2011 Trustees of the University of Pennsylvania