Introduction
The AQUAINT Corpus, Linguistic Data Consortium (LDC) catalog
number LDC2002T31 and ISBN 1-58563-240-6 consists of newswire text data in English,
drawn from three sources: the Xinhua News Service (People's
Republic of China), the New York Times News Service, and the
Associated Press Worldstream News Service. It was prepared by
the LDC for the AQUAINT Project, and will be used in official
benchmark evaluations conducted by National Institute of
Standards and Technology (NIST).
Data
The data files contain roughly 375 million words correlating to
about 3GB of data. The text data are separated into directories by source
(apw, nyt, xie); within each source, data files are subdivided
by year, and within each year, there is one file per date of
collection. Each file is named to reflect the source and
date, and contains a stream of SGML-tagged text data
presenting the series of news stories reported on the given
date as a concatenation of DOC elements (i.e. blocks of text
bounded by and tags).
All data files are published in compressed form, using the
GNU "gzip" utility; as such, all files have a ".gz" extension,
and will have null file name extension when uncompressed in
the usual way (i.e. just the base file name, consisting of
"YYYYMMDD_SRC").
While all the data files are covered by a single DTD, it is
not the case that they all have a single pattern of markup.
Rather, all files share a core markup structure, with minor
variations in the peripheral regions of each DOC element, and
the DTD has been written to accommodate the variations.
Updates
19980614_NYT.gz was left off in the conversion from CD to DVD. An update was issued on 09/13/2012. All copies ordered after this date will be complete. Contact ldc@ldc.upenn.edu for more information.
Content Copyright
Portions © 1998-2000 New York Times, © 1998-2000 The Associated Press, © 1996-2000 Xinhua News Agency, © 2002 Trustees of the University of Pennsylvania |