WORKSHOP REPORT
Workshop on Web-Based Language Documentation and Description
and the
Open Language Archives Community
J. Albert Bickford
SIL-Mexico and University of North Dakota
albert_bickford@sil.org
The Workshop on Web-Based Language Documentation and Description (December
12-15, 2000, University of Pennsylvania) brought together linguists,
archivists, software developers, publishers and funding agencies to discuss
how best to publish information about language on the internet. This
workshop, together with the Open Language Archives Community which is
developing out of it, seem especially important in providing useful
information about linguistics and less-commonly studied languages for both
scholars and the wide general audience that can be found on the web. I hope
that this report will be useful in understanding these new developments in
the linguistics publishing and archiving field.
The aim of the workshop was to establish an infrastructure for electronic
publishing that simultaneously addresses the needs of users (including
scholars, language communities, and the general public), creators,
archivists, software developers, and funding agencies. Such an
infrastructure would ideally meet a number of requirements important to
these different stakeholders, such as:
- provide a single entry point on the internet through which all materials
can be easily located, regardless of where they are stored (on the internet or
in a traditional archive). Essentially, this would be a massive union catalog
of the whole internet and beyond.
- identify every language uniquely and precisely, so that all materials
relevant to a particular language can be located
- make available software for creating, using, and archiving data (especially data in special formats); this includes software to help convert data from older formats to newer ones
- serve as a forum for giving and receiving advice about software, archiving
practices, and related matters
- provide opportunity for comments and reviews of materials published within
the system
The workshop was organized by Steven Bird (University of Pennsylvania) and Gary
Simons (SIL International).1 It included approximately 40 presentations and
several working sessions on a variety of topics.
There was general agreement among the participants that a system for organizing
the wealth of language-related material on the internet is needed, and that an
appropriate way to establish one is by following the guidelines of the Open
Archives Initiative (OAI) [http://www.openarchives.org]. (These guidelines provide a general framework for creating systems like this for specific scholarly communities.) An OAI publishing and archiving system contains the following elements:
- data providers, which house the materials that are indexed in the system
- a standardized set of cataloguing information for describing each of the materials, also known as "metadata" (i.e., data about data)
- service providers, which collect the metadata from all the data providers and allow users to search it in various ways so as to locate materials of interest to them
In the case of linguistics, the system will be known as the Open Language
Archives Community (OLAC). The Linguist list [http://www.linguistlist.org] has
agreed to serve the system as its primary service provider. It will be the main
source that people will use to find materials through the system. Further
information about OLAC can be found at [http://www.language-archives.org]. The
agreement to establish OLAC is probably the most important accomplishment of
the workshop.
This agreement was solidified through working sessions which met during the
workshop and started the process of working through the details in various
areas, such as:
- Character encoding: Unicode, fonts, character sets, etc.
- Data structure for different types of data (lexicons, annotated text, etc.)
- Metadata (cataloguing information that should be common to the whole community and how it should be represented in the computer) and other concerns of archivists
- Ethics, especially the responsibilities that archivists and publishers have to language communities
- Expectations of users, creators (e.g. authors), software developers
These and other issues will continue to be discussed on email lists in the
coming months, ultimately culminating in recommendations for "best practice" in
each area, together with a preliminary launch of the whole system, hopefully
within a year. (Prototypes of the system are available now at the OLAC address
above, along with various planning documents.)
There were also a number of conference papers, which provided a foundation for
making the working sessions productive. Rather than list or review all the
presentations here, I will summarize them, since they are all available on the
conference website [http://www.ldc.upenn.edu/exploration/expl2000]. The topics
covered included the following:
- Proposals for various aspects of the OLAC system
- Concerns of various stakeholders, such as archivists, sponsors, language communities
- Descriptions and demonstrations of specific software, research projects, and web publishing systems
- Metadata and metadata standards
- Technical issues, such as Unicode, the OAI, sorting, data formats for
different types of language materials (e.g. dictionaries, annotated text,
example sentences in linguistic papers, and audio)
One insight that I gleaned from these presentations was a better understanding
of glossed interlinear text. Interlinear text is not a type of data, but rather
just one possible way of displaying an annotated text. The annotations on a
text can consist of many types of information: alternate transcriptions,
morpheme glosses, word glosses, free translations, syntactic structure (and
possibly several alternative tree structures for the same text), discourse
structure, audio and video recordings, footnotes and commentary on various
issues, etc. What ties them all together is a "timeline" that proceeds from the
beginning to the end of a text, to which different types of information are
anchored. Aligned interlinear glosses are one way of displaying some of this
information, but not the only way, and not even the most appropriate way for
some types of information. The traditional arrangement of Talmudic material,
for example, with the core text in the center of the page and commentary around
the edges, is another possible display of annotated text, in which the
annotations are associated more with whole sentences and paragraphs than with
individual morphemes. There are also some sophisticated examples available for
presenting audio alongside interlinear text. (For example, check out the LACITO
archive [http://lacito.archivage.vjf.cnrs.fr]!)
Throughout, it was very clear that those at the conference had a great deal in
common with each other:
- a primary concern for descriptive (as distinguished from theoretical) linguistics
- a desire to make language materials available, to communities of speakers and the general public as well as scholars
- an interest in taking advantage of the Internet, which provides a means of publishing such materials that by-passes the limitations of traditional publication (since the costs are so much lower, and thus appropriate for materials that have smaller audiences)
- awareness that many materials may be less than fully-polished yet still valuable to some people and worth archiving
- a sense of frustration with the currently confused state of the art in data formats, especially fonts and character encoding, and the lack of good information about how best to archive and publish on the web
- awareness of the large amount of data that is in data formats which will be obsolete in a few years (and thus a willingness to accept data in whatever form it is in, while also seeing a need for software to help convert data to newer formats)
- a strong suspicion toward and distrust of rigid requirements, yet a
willingness to adopt standards voluntarily when their usefulness has been
demonstrated
Finally, the conference pointed out several trends that will be increasingly
important in future years.
- The speakers of lesser-known languages will be more actively involved in
the production and use of materials in and about their languages, and their
concerns will increasingly have to be considered by scholars. These include
carefully documenting permissions and levels of access to materials, making
sure that language materials are available to the communities themselves, and
being careful that scholars do not inadvertently aid commercial interests in
exploiting native knowledge-systems (such as medicinal use of plants) without
appropriate compensation.
- The boundary between publishing, libraries, and archiving is being blurred
by the shift to the digital world. Materials can be "archived" on the web,
which is a type of publication. Electronic "libraries" are springing up in many
places. Published and unpublished works from around the world can be listed
together in one common catalog. The same technology is important in both
spheres of activity. In short, these activities are merging under a new
umbrella that could be called "scholarly information management". A corollary
to this trend is that archiving is not just something done at the end of a
research project; it's part of the ongoing process of managing the information
that the project produces.
- In such a world, and with huge numbers of resources available to sift
through, metadata becomes increasingly important. A freeform paragraph
description in a publications catalog is no longer good enough. It is the
metadata that users will consult in order to find materials of interest to
them, so the metadata must be carefully structured, accurate and current. More
and more, scholars will have to think not just about producing materials but
also about how to describe them so as to make them accessible to others.
- Unicode [http://www.unicode.org] is the way of the future for
representation of special characters in computers. The days of special fonts
for each language project are numbered. Instead, Unicode will make possible a
single set of fonts that meets virtually everyone's needs in the same
package. Over the next few years, most people will be switching their computers
over to using Unicode almost exclusively (that is, if they want to take
advantage of newer software).
- Language data will increasingly need to be structured carefully so that not
only can people view it and use it, but machines will be able to understand and
manipulate it in various ways. This will most likely be done using XML
(Extensible Markup Language) which is already widely-supported in the computer
industry, with more support becoming available regularly.2
All in all, it was a workshop that was both stimulating and practical, one
which will have an unusual amount of influence in months and years to come.
Footnotes:
1 Funding was provided by the Institute for Research in Cognitive Science
(IRCS) of the University of Pennsylvania, the International Standards in
Language Engineering Spoken Language Group (ISLE), and Talkbank.
2 Since XML's development has been closely-associated with the World Wide Web
consortium [http://www.w3.org/XML/], it has been widely regarded as the
successor to HTML for web pages. However, this is just a small part of its
usefulness; it is a general-purpose system for representing the structure of
information in a document or database, which can be customized for myriads of
purposes. Many software tools are currently available for creating and
manipulating data in XML, with more being created all the time. One, Extensible
Stylesheet Language Transformations [http://www.w3.org/TR/xslt], can do complex
restructuring of XML data.
Return to Index