Éric de La Clergerie has continued to improve the coverage, quality and efficiency of the French meta-grammar FRMG. This work is made progressively easier, using the various testing scripts developed in 2009 (confusion matrices for instance). A small but very practical shell ( frmg_shell) has been written to ease the use of FRMG, in particular for testing sentences.

Preliminary experiments have been tried to integrate, in the post-parsing disambiguation phase, some quantitative information provided by the knowledge acquisition experiments (cf.  ) about for instance the strength of a dependency through some preposition between a governor noun and a governee noun. Weights for several kinds of dependencies have been used, exploiting their frequency of a large corpus but also the semantic proximity between the governors or between the governees, as computed by the knowledge acquisition experiments. However, while real, the benefits of these weights for a better disambiguation remain very small. Further experiments have to be tried to investigate the situation, with maybe a better calibration of the imported weights wrt the various weights manually attached to the disambiguation rules.

In the context of the PASSAGE and SCRIBO actions, we have continued the explore the use of distributed computing for processing of very large corpora, largely using GRID 5000 and a local cluster at INRIA Rocquencourt. For the SCRIBO action (cf.  ), GRID 5000 was in particular used to parse with FRMG a 700 million words corpus (including Wikipedia, Wikisource, and 30 months of AFP news). Several problems were again identified and corrected to get more and more robust treatments.

Distributed computing is also used on local clusters to collect and count patterns in the output of FRMG, using an adapted version of the map-reducealgorithm. These patterns with counts are then used as input for knowledge acquisition (cf.  ).

The parsing results provided by FRMG over a very large corpus covering various style and domains (30 months of AFP news [2007, 2009, and 2010], wikipedia, wikisource, Est Republicain, EuroParlement, JRC) have been used as input for the acquisition of semantic knowledge. This work was part of the SCRIBO action (cf.  ), in order to explore the techniques and methodologies that may be used to enrich a lexicalized seed ontology in a supervised way, i.e. using some human validation at some point.

The guiding principle for most of the experiments that were tried is the Harris distributional hypothesis, stating that semantically close words occur in similar contexts, syntactic contexts in our case.

The experiments may be classified around 2 main directions, those trying to identify and organize concepts, and those trying to identify and organize classes of “events”, an event class being (often) used to establish some relationship between named entities (for instance, to denote that “someone” becomes president of “something”). These event class may properly denote events, but more generally, may denote the attachment of a property to an entity.

This work involves the automatic construction of distributional thesauri for French, with an eye toward use in statistical parsing. The distributional hypothesis states that words occurring in the same contexts tend to have similar meanings, as posited by Harris (1954). Additionally, distributional similarity based on syntactic contexts naturally indicates shared selectional preference, which may be useful for statistical parsing. Finally, distributional lexical resources are appealing because they can be constructed automatically from raw text corpora using unsupervised approaches, avoiding the problem of limited lexical coverage found in hand-built resources.

Probabilistic parsers are trained on treebanks, namely syntactically annotated sentences, and this training allows to capture syntactic regularities. Yet, though lexical information is known to play a crucial role in determining the syntactic structure of a sentence, many lexical phenomena cannot be learned simply by training on a treebank of a few thousands of sentences (the French treebank we use contains about 12000 sentences). First because treebanks cover only a small part of the French vocabulary. Second, because lexical data is very sparse : a corpus contains a few very frequent words, and a lot of rare words. Compared to English, this is even truer for French, or more generally inflected languages : morphological marks for gender, number, tense etc... drastically augment the vocabulary size.

To cope with this inherent limitation of statistical parsing techniques, we have investigated the use of word clusters instead of words as input to the parser. Our work was inspired by [Koo et al. 08] , who have shown that word clusters obtained with unsupervised techniques could improve statistical dependency parsing, when used as features for classifiers determining the weights of dependency arcs. We tried in 2009 to use word clusters within the framework of generative statistical parsing [Candito and Crabbé, 09]. We continued this investigation en 2010: we tested the impact on parsing performance of two morphological clustering techniques on both lexicalized models and PCFG-LA models : the "desinflection" process proposed in 2009 (a lexicon-based morphological clustering), and "lemmatisation" Conducted by a data driven lemmatizer which we adapted for French during a visit at Saarbrucken in 2009., which is the more classic technique that groups inflected word forms into part-of-speech+lemma pairs. Though with this second technique the oracle obtained with gold part-of-speech and gold lemmas is higher, the results obtained in a realistic setting (predicted part-of-speech and lemmas) are comparable to the rougher desinflection method. We have also analyzed the improvement in performance for both techniques with respect to word frequency. We found that replacing word forms with clusters improves attachment performance for words that are originally either unknown or low-frequency, since these words are replaced by cluster symbols that tend to have higher frequencies. Furthermore, clustering also helps significantly for medium to high frequency words, suggesting that training on word clusters leads to better probability estimates for these words.

It shall be noted that this work serves as a basis of a grant proposal to the FP7 ICT-Call, named parse4real, jointly made by Dublin City University, Alpage, Uppsala University and others European leaders in parsing morphologically-rich languages on which augmenting the lexical coverage of statistical parsers if of crucial importance. Djamé Seddah is Alpage's leader for that proposal, to which Marie Candito, Éric de La Clergerie and Benoît Sagot also participate.

Dependency trees are often preferred to syntagmatic trees for many NLP tasks, such as information extraction, question answering, lexical acquisition. We started in 2008, and continued in 2009, to work on the conversion of the syntagmatic trees of the French treebank into surface dependency trees. We have now a stabilized version of a dependency treebank : the French treebank converted to dependencies .

The constituent-to-dependency conversion procedure can also be applied to syntagmatic trees as output by a parser trained on the syntagmatic treebank. Hence, we have various ways to obtain a parser outputting dependency trees : (i) training a parser on syntagmatic trees, and converting the output of this parser into dependencies. And (ii) directly using existing algorithms to train a dependency parser on the treebank converted to dependencies. We have performed a comparison of the two approaches (i) and (ii). Approach (i) is tested with an architecture where a parser is trained on the French treebank (using Petrov's algorithm), and output trees from this parser are converted to dependencies. Approach (ii) is tested with two dependency parser training algorithms : MST [McDonald and Pereira, 06] and MaltParser [Nivre et al., 06]. First bare results showed that directly training a dependency parser with the MST algorithm [McDonald and Pereira, 06] outperforms the architecture base on Petrov's algorithm. We then performed a more sophisticated benchmark between the three architectures (Petrov, MSTparser, MaltParser), with the integration of morphological features (lemmas, gender, number...) and unsupervised word clustering features (cf. results "Improving the lexical coverage of statistical parsers"). We found that with these additional information, the differences between parsing architectures are generally small, and there is no consistent trend favoring either constituency-based or dependency-based methods. The best performance is achieved using MSTParser, enhanced with predicted part-of-speech tags, lemmas, morphological features, and unsupervised clusters of word forms. MaltParser achieves slightly lower labeled accuracy, but is probably the best option if speed is crucial

Understanding the properties of formal languages more sophisticated than Context-Free Grammars has been a research topic for several Alpage members for a long time. Indeed, although probablistic models based on simple CFG backbones or TAG-based systems are quite successful for parsing tasks, they fail to model correctly some linguistic constructions that are complex but not necessarily rare. So-called Mildly Context-Sensitive formalisms are a particular class of grammars that have reasonable expressive power (strictly higher than TAGs, for example, but strictly less than Range Concatenation Grammars that cover PTIME) although they have a reasonable parsing complexity (polynomial). Studying these formalisms is therefore a very important research track.

Following up on work started back in 2006 , Djamé Seddah and Benoît Sagot have introduced a new extension of the so-called Multi-Component TAGS, and more precisely an extension of the MCTAGs with Local Shared Deriva- tion introduced in which can handle non local elliptic coordinations. Based on a model for control verbs that makes use of so-called ghost trees, one can show that this extension leads to an analysis of argument cluster coordinations that provides an adequate derivation graph. This is made possible by an original interpretation of the MCTAG derivation tree mixing the views of and .

MCTAGs and some of their variants are equivalent to a well-studied MCS formalism, that of Linear Context-Free Rewriting Systems or LCFRSs . Having NLP applications and language modeling in mind, studying the formal properties of LCFRSs is relevant. In collaboration with Giorgio Satta, from the University of Padova (Italy), Benoît Sagot has finalized and presented at ACL 2010 an optimal algorithm for reducing the rank of a sub-class of LCFRSs (namely those with fan-out 2) .

An important task in temporal processing is to recover the chronology of the events that are described in a text. Most recent work has focused on learning temporal relations (e.g., precedence, inclusion) between given events in a text, while rarely ensuring that these separate pieces of information remain consistent. The target representations also differ, according to the distinctions they make or the inferences they allow.

In , we investigate the impact of using different temporal algebras for learning temporal relations between events. Specifically, we compare three interval-based algebras: Allen (1983) algebra, Bruce (1972) algebra, and the algebra derived from the TempEval-07 campaign. These algebras encode different granularities of relations and have different inferential properties. They in turn behave differently when used to enforce global consistency constraints on the building of a temporal representation. Through various experiments conducted on the English TimeBank/AQUAINT corpus, we show that although TempEval smaller relation set leads to the best classification accuracy performance, it is too vague to be used for enforcing consistency. By contrast, the other two relation sets are similarly harder to learn, but more useful when global consistency is important. Overall, the Bruce algebra is shown to give the best compromise between learnability and expressive power. In forthcoming work, we consider the use of yet another temporal algebra, namely point algebra, in the context of a global inference problem to address the problem of predicting temporal structure for texts.

This first work has been performed on English, for until recently there was no available TimeBank for French. One of the contributions of André Bittar's PhD dissertation, supervised by Laurence Danlos and co-supervised by Pascal Denis that was successfully defended this year, was to develop such a resource and to make it publicly available to the community. The current version of the French TimeBank consists of 109 journalistic texts (16,208 tokens) from 7 different sub-genres which have been annotated according to the ISO-TimeML standard. Bittar's dissertation also suggests a number of improvements to the ISO-TimeML schema in order to account for linguistic phenomena which apply across languages, as well as adaptations necessary for the processing of French texts. The French TimeBank is soon to be made available (via the INRIA Forge) to the wider scientific community and will provide a useful basis for studying the linguistic expression of temporal phenomena in French, as well as providing data for evaluating automatic temporal annotation systems.

Discourse interpretation is often automated in two steps: a segmentationstep where discourse units (DU) are extracted, and a parsingstep where these DUs are related to derive a discourse structure. The relations are labelled with functions reflecting the underlying intention of the producer of the discourse.

Within that perspective, one can proceed bottom-up, and isolate the elementary discourse units (EDU) before building a structure made of groupings of such EDUs. Alternatively, a text can be segmented in a top-down fashion, separating topically coherent parts (this is called text segmentationor text tiling) before applying the more fine-grained approach mentioned before. Ideally the two methods complement each other, while this is rarely done in practice.

Some strong punctuation signs are “wrongly” used instead of weak punctuation signs, leading to graphic sentences which are not grammatical sentences, see the following discourse which includes five graphic sentences only made up of adverbial phrases.

On avait donné dans le Nord un grand coup de pied dans la fourmilière, et les fourmis s'en allaient. Laborieusement. Sans panique. Sans espoir. Sans désespoir. Comme par devoir. [Saint Exupéry, Pilote de guerre]

In , we present a corpus study of this phenomenon — sometimes called “épexégèse” — and a tool in the early stages to automatically detect wrong strong punctuation signs. The goal of this tool is to automatically categorize these punctuation signs as weak, so that a parser can make a standard syntactic and semantic analysis.

In an attempt to cope with lexical incompleteness within the EDyLex project ( ), a typology of unknown words (i.e. words, forms, tokens) has been proposed. This typology reflects both linguistic structures and operational issues regarding unknown words. It is indeed intended to guide automatic processing modules in the handling of out-of-lexicon words . Unknown words are defined relatively to the kind of unknown tokens which compose them. Those can be either productive sequences such as dates and recognizable for example by a local grammar, productive lexical creations ( red-hair), lexicalized forms ( cupboard), proper names, borrowings, dependant components ( priori) or errors (spelling, misprint).

In order to verify and improve this typology, a corpus annotation of unknown words based on it has been carried out in two phases. First the corpus, consisting of xxx news items from Agence France Presse in French, English and Spanish, have been automatically annotated with the Alpage surface processing chain SxPipe, able to recognize a series of out-of-lexicon sequences and forms (dates, URLs, numerical values, proper names, some prefixed compositions...); this automatic annotation then marked remaining out-of-lexicon forms, based on the lexicons available in the three languages within the Alexina framework (Lefff, Enlex and Leffe). In the second phase the latter forms have been reviewed by human annotators who assigned to each of them the appropriate category from the typology. A high inter-annotators agreement showed the relevance of this typology and can therefore help the building of a complete and modular processing chain fo lexical resources enrichment.

Within the Alpage surface processing chain SxPipe, a series of modules based on local grammars enable the analysis of sequences at the token- and form-level such as dates, URLs, addresses... A new version of NP, the named entity recognition tool, has been integrated to SxPipe in order to cope with named entities of different types (person, location, organization and company names). NP's local grammar consists of 130 rules, including contextual patterns and triggers, for detection and typing of named entities. In order for those rules to operate efficiently, a lexicon of proper nouns can be associated to it. The lexicon consists in a list of forms acceptable by the grammar as named entities variants and can be obtained by several means.

In this prospect, a database called Aleda has been built, based on two main external and freely available resources: Wikipedia and Geonames. Aleda intends to gather information and knowledge on entities and not only a list of entity names. This knowledge (precise type of the entity (for example capital, country, museum are subtypes of location), decomposition and variation of its name (title, last name, pseudonym, short form...), main sector of organizations and companies...) is reflected in its structure by dedicated typed fields. The base can then be converted into a lexicon, linking each entity to its variants with a unique identifier. This structure allows for the ulterior use of information contained in Aleda by the client application.

In order to make named entities recognition more complete and usable as an information extraction module, the denotational aspect of named entities must be handled, i.e. the entity to which a recognized name refer to must be identified. This process of entity resolutioncan be based on an entity base such as Aleda. After the recognition phase, the result of which being a set of entity mentions in text, the resolution phase intends to link each mention to the adequate entity in the reference base. This operation must deal with possible ambiguities among homonyms or names refering to entities absent from the base. Disambiguation can benefit from the knowledge gathered in Aleda, for example by comparing information associated to entities and named entities contexts in the text.

In 2010, Alpage has taken part actively in the PerGram French-German project (co-funded by ANR and DFG, see ). In particular, the development of an Alexina description of the morphology of Persian and the development of an associated Persian Alexina lexicon, named PerLex, has been pursued following a first preliminary version in end-2009 . This work, in collaboration with Géraldine Walther (from LLF, Université Paris 7) and Pollet Samvelian, Pegah Fagiri and Ariel Gutman (MII, CNRS and Université Paris 3), has also lead to adapting the shallow processing chain SxPipe to Persian . Ongoing work has started for the manual validation of PerLex and for its extension to the syntactic level (sub-categorization frames, entries for “complex predicates”...).

Benoît Sagot and Géraldine Walther have then pursued their efforts for resourcing Western Iranian languages. First, they formalized a methodology for developing a morphological lexicon for a language lacking any resource, and applied its first steps to Sorani Kurdish, hence creating a preliminary small-scale Alexina lexicon for this language . Second, they performed experiments on later stages of this methodology, but in Kurmanji Kurdish , which is another Kurdish variant for which lexical resources do exist, although not formalized. Their work include preliminary experiments on developing a POS tagger using a lexicon but no annotated training corpus. These experiments will be pursued in 2011, and both Kurdish lexicons should reach a significant size.

We study the problem of choice in the ordering of French words building upon two case studies. Both studies try to identify the factors that come into play when one has two choose among several possible orderings : the first inquiry is dedicated to the position of attributive adjectives wrt the noun. The second is dedicated to the relative order of postverbal dependants. Both questions have almost never been adressed for French.

In collaboration with Gwen Fox (Université Paris 3), the first investigation in this direction has been led towards identifying the importance of constraints that drive the placement of attributive adjectives wrt the noun in the noun phrase in French. This study brings an additional element to Bresnan’s thesis, according to which the syntactic competence of human beings is indeed probabilistic. This year we enhanced our previous statistical models with a qualitative study trying to shed light (1) on semantic effects and (2) on word independant versus word specific constraints.

Another study on preferences in verbal complementation has started this year. We are planning to study the preferences between postverbal dependants (direct objects and indirect objects). This year we started by extracting data suitable for statistical modeling mainly from the French Treebank and started to annotate the treebank with missing information (of semantic nature). The expected outcome from this work is first to identify factors that are relevant in French and beyond in a crosslinguistic perspective we plan to compare the constraints observed in other languages such as german. This is intended to be realised in collaboration with psycholinguistic teams both in Paris 5 (LPNCog) and in Germany (Frias, Freiburg) and the Laboratoire de Linguistique Formelle (LLF) in Paris 7 also intends to set up experimental work in connection with our modeling results in this framework for French.

As can be seen from the outline above, this line of research brings us closer to cognitive sciences and more specifically to frameworks inspired by construction grammar. We hope in the very long run that these investigations will bring – among other – further insights on the design of probabilistic parsers. In NLP the framework that is closest to implementing construction grammar is Data Oriented Parsing.

This is an exploratory work on modelling the acquisition of allophonic rules. It is made in collaboration with the Laboratoire de Sciences Cognitives de Paris, LSCP). It explores the acquisition of the phonological system made by children relying upon the assumption that word segmentation and phonological learning are dependant processes feeding each other. Hence this work splits into two subproblems, that of segmentation and that of phonological clustering.

State-of-the-art models of the acquisition of word segmentation have been evaluated using phonemically transcribed corpora. As such, they implicitly assume that children know how to undo phonetic variation when they learn to extract words from fluent speech. Moreover, whereas models of language acquisition should perform similarly across languages, evaluation is often limited to English samples. We first argue that online learning is a sound desideratum for any model of language acquisition and use this criterion to select candidate segmentation models. Then, using child-directed corpora of English, French and Japanese, we evaluate the models' performance given inputs where phonetic variation has not been reduced. To do so, we propose a parametric benchmark where segmentation robustness can be measured across different levels of noise, simulating uniform errors in phoneme recognition or systematic allophonic variation. We show that statistical models do not resist noisy inputs and do not generalize to typologically different languages. From the perspective of early language acquisition, the results strengthen the hypothesis according to which phonological knowledge is acquired in large part before the construction of a lexicon.

It is recognized that infants learn phonemes through some kind of unsupervized clustering of the speech signal. Previous work has shown the feasibility of unsupervized clustering, using as input manually segmented phonetic parameters . showed that an HMM state-splitting algorithm run on conversational speech coded using standard MFCC coefcients automatically grows a network of HMM states which successfully encoded speech sounds with no loss of informaton compared to supervized HMM training. However the obtained states did not map one-to-one to phonemes: phoneme-size strings of states did not yield abstract phonemes, but rather, context dependant allophones. This problem is serious enough to impede subsequent unsupervized learning of words . To adress this issue, we will recluster allophonic variants using higher order information in additon to acoustic distance in order to improve the metric used in the clustering algorithm. Two such informaton have been shown to be helpful: distributional information about adjacent segments and (pseudo)-lexical informaton obtained through approximatng a lexicon using frequent n-grams .