Chinese Gigaword Fourth Edition, Linguistic
Data Consortium (LDC) catalog number LDC2009T27 and isbn 1-58563-527-8, is a comprehensive archive of newswire text
data that has been acquired over several years by the LDC. This edition includes
all of the contents in
Chinese Gigaword Third Edition (LDC2007T38) as well as newly collected data.
In addition, four entirely new sources have been added in the fourth edition,
Central News Service, Guangming Daily, People's Liberation Army Daily, and People's
The eight distinct international sources of Chinese newswire included
in this edition are the following:
- Agence France Presse (afp_cmn)
- Central News Agency, Taiwan (cna_cmn)
- Central News Service (cns_cmn)
- Guangming Daily (gmw_cmn)
- People's Daily (pda_cmn)
- People's Liberation Army Daily (pla_cmn)
- Xinhua News Agency (xin_cmn)
- Zaobao Newspaper (zbn_cmn)
The seven-letter codes in the parentheses above are used for the
directory names and data files for each source, and are also used (in
ALL_CAPS) as part of the unique DOC "id" string assigned to each news
The original data received by the LDC from AFP, People's Liberation
Army Daily, Xinhua, and Zaobao were encoded in GB-2312, those from CNA
were in Big-5, and those from GMW, CNS, and People's Daily were in a
combination of GB-2312 and GB-18030. To avoid the problems and confusion
that could result from differences in character-set specifications, all
text files in this corpus have been converted to UTF-8 character encoding.
New in the Fourth Edition
- Two years worth of new articles (January 2007 through December 2008)
have been added to the Xinhua, Agence France Presse, and CNA data sets.
- Four new data sources have been added - Guangming Daily, Central News
Service, People's Daily and People's Liberation Army daily, covering
a timespan from November 2006 through December 2008.
This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.
Portions © 2000-2008 Agence France Presse,© 1991-2008 Central News Agency
(Taiwan),© 2006-2008 China Military Online, © 2006-2008 Chinanews.com, © 2006-2008 Guangming Daily, © 2006-2008 People's Daily, © 1998, 2000-2003
SPH AsiaOne, Ltd., © 1990-2008 Xinhua News Agency, © 2003, 2005, 2007, 2009
Trustees of the University of Pennsylvania