Introduction

This file contains documentation on the Arabic Treebank: Part 3 (full corpus) v 2.0 (MPG + Syntactic Analysis), Linguistic Data Consortium (LDC) catalog number LDC2005T20 and ISBN 1-58563-341-0.

The goal of the Arabic Treebank project is to support the development of data-driven approaches to natural language processing (NLP), human language technologies, automatic content extraction (topic extraction and/or grammar extraction), cross-lingual information retrieval, information detection, and other forms of linguistic research on Modern Standard Arabic in general. The LDC was sponsored to develop an Arabic POS and Treebank of 1,000,000 words, and this corpus is part three of that project. In this release, we provide both syntactic (treebank) annotation and annotation on part of speech (POS), gloss, and word segmentation.

Treebanks are language resources that provide annotations of natural languages at various levels of structure: at the word level, the phrase level, and the sentence level. Treebanks have become crucially important for the development of data-driven approaches to natural language processing (NLP), human language technologies, automatic content extraction (topic extraction and/or grammar extraction), cross-lingual information retrieval, information detection, and other forms of linguistic research in general.

This corpus is designed for those who study and use languages either professionally or academically, and who need text corpora in their work. The Penn Arabic Treebank is particularly suitable for language developers, computational linguists and computer scientists who are interested in various aspects of natural language processing.

The Penn Arabic Treebank, which was part of the DARPA TIDES project, started in the Fall of 2001 with the objective of annotating via human intervention and automatically a large Arabic machine-readable text corpus. As in previous Penn Treebanks, two different kinds of information need to be produced by two different (human and computer) processes. The Arabic Treebank project consists therefore of two distinct phases: (a) Part-of-Speech (=POS) tagging, which divides the text into lexical tokens and gives relevant information about each token such as lexical category, inflectional features, and a gloss (referred to as POS for convenience, although it includes morphological and gloss information not traditionally included with part-of-speech annotation), and (b) Arabic Treebanking (=ArabicTB), which characterizes the constituent structures of word sequences, provides categories for each non-terminal node, and identifies null elements, co-reference, traces, etc.

Both tasks started in November 2001 with an initial pilot consisting of 734 files representing roughly 166K words of written Modern Standard Arabic newswire from the Agence France Presse corpus, which has since been released as "Arabic Treebank: Part 1 v 3.0," LDC Catalog No. LDC2005T02. The second part was released as the 168K-word corpus "Arabic Treebank: Part 2 v 2.0," LDC Catalog No. LDC2004T02.

The current Arabic Treebank: Part 3 corpus consists of 600 stories from the An Nahar News Agency. This corpus is also referred to as ANNAHAR. The new features include complete vocalization of all Imperfect Verb mood endings: Indicative, Subjunctive, and Jussive.

The POS only annotation of this ANNAHAR corpus was released in 2004 under the catalog number LDC2004T11 (Arabic Treebank: Part 3 v 1.0). In addition to the treebank annotation, this release (i.e., Arabic Treebank: Part 3 v 2.0) also includes the POS annotation in LDC2004T11.

Samples

• POS
• Treebank

Content Copyright

Portions © 2002 An Nahar News Agency, © 2004 Trustees of the University of Pennsylvania