Spanish Gigaword Third Edition, Linguistic Data Consortium (LDC) catalog number
LDC2011T12 and ISBN 1-58563-596-0, was produced by LDC. It is a comprehensive
archive of Spanish newswire text data that has been acquired over several years
by LDC. Spanish Gigaword Third Edition includes all of the content of the second
and adds data collected from January 1, 2009 through December 31, 2010.
The three distinct international sources of Spanish newswire in this
edition, and the time spans of collection covered for each, are as
- Agence France-Presse, Spanish (afp_spa) May 1994 - Dec 2010
- Associated Press, Spanish (apw_spa)
Nov 1993 - Dec 2010
- Xinhua News Agency, Spanish (xin_spa) Sep 2001 - Dec 2010
The seven-letter codes in the parentheses above include the
three-character source name abbreviations and the three-character
language code (spa) separated by an underscore (_) character. The
three-letter language code conforms to LDCs internal convention based on the
ISO 639-3 standard.
All text data are presented in SGML/XML form, using a very simple,
minimal markup structure all text consists of printable ASCII,
whitespace, and printable code points in the Latin1 Supplement
character table, as defined by both ISO-8859-1 and the Unicode
Standard (ISO 10646) for the accented characters used in Spanish.
The Supplement/accented characters are rendered using UTF-8 encoding.
For all of the documents in this corpus, we have applied a rudimentary
(and _approximate_) categorization of DOC units into four distinct
types. The classification is indicated by the type=string
attribute that is included in each opening DOC tag. The four
- story : This is by far the most frequent type, and it represents the
most typical newswire item: a coherent report on a particular topic
or event, consisting of paragraphs and full sentences.
- multi : This type of DOC contains a series of unrelated blurbs, each of
which briefly describes a particular topic or event this is typically applied
to DOCs that contain summaries of todays news, news briefs in ... (some
general area like finance or sports), and so on.
- advis : (short for advisory) These are DOCs which the news service addresses
to news editors -- they are not intended for publication to the end users
(the populations who read the news). This type contains formulaic,
repetitive content (contact phone numbers, etc).
- other : This represents DOCs that clearly do not fall into any of
the above types -- in general, items of this type are intended for
broad circulation (they are not advisories), they may be topically
coherent (unlike multi type DOCS), and they typically do not
contain paragraphs or sentences (they arent really stories)
these are things like lists of sports scores, stock prices,
temperatures around the world, and so on.
An update to Spanish Gigaword Third Edition was issued to fix an issue with 26
consecutive months of data files from Xinhua Spanish: xin_spa_200601 through xin_spa_200802 i.e.
all files from 2006 and 2007, plus the first two files from 2008. The problem was that all letters with diacritic marks had been omitted
in the text data for that portion of the collection. For example, the
word ano was presented as ao (minus the n with tilde character),
aspiracion appeared as aspiracin, and similarly for all accented
characters (UTF-8 letters outside the ASCII range). All copies of Spanish
Gigaword Third Edition ordered after February 2013 will have this update included.
More information is included in the
readme associated with this update.
Portions © 1994-2010 Agence France Presse, © 1993-2010 The Associated
Press, © 2001-2010 Xinhua News Agency, © 2006, 2009, 2011, 2013 Trustees
of the University of Pennsylvania