CLEA1500

Revision as of 17:03, 4 August 2014

CLEA1500 certifies that you have reached a standard of knowledge concerning the development of natural language grammars for UNLization and may officially participate in the UNL Research. The certificate consists of a set of 100 sentences that must be UNLized with IAN.

Goal

In CLEA1500, you are expected to provide the lingware (dictionary and grammars) necessary to UNLize a translated version of UCA1.

Instructions

Corpus

1. Translate (manually) the sentences of the corpus UCA1 from English into your native language. Be as close as possible to the original, and provide one single translation for each sentence. This will be your input document file, and your goal will be to provide (automatically, through IAN) the UNL graphs for each sentence
   1. Save the translated text (without the English original) in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>NL FILES.
2. Dictionary
   1. Localize the dictionary available at eng_unl_dic.txt. Note that "localization" is not the same as "translation". You may need other features (in English, for instance, nouns do not have gender or case) or other entries. In any case, the resulting dictionary should fit your translated version of the corpus (i.e., all the entries appearing in your translated version of the corpus should appear in the dictionary). For further information on localization, see Localization. For information on the dictionary structure, see Dictionary Specs. For an explanation of the structure of the English dictionary, see English Dictionary. In case you need additional features, use only the tags available at the tagset.
   2. Save the NL-UNL dictionary in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>DICTIONARIES.
   3. Upload the Default Dictionary to UNLWEB>UNLDEV>IAN>DICTIONARIES.
3. Grammar
   1. Create the NL-UNL (analysis) transformation grammar necessary to analyze, in UNL, the natural language sentences of the translated corpus. This grammar is the most difficult (and the actual goal) of the whole analysis task. In order to prepare the grammar, study the Grammar Specs. Next, take a look at the structure of the English Grammar for a detailed example. Note that the English Grammar is actually made of three different sub-grammars: the Standardization grammar, the English Grammar itself, and the Default grammar. In many cases, it is simpler just to localize the English grammar to your own locale rather than creating a whole grammar from the scratch. See the instructions at Localization. The Standardization Grammar and the Default Grammar are supposed to be language-independent, and you may simply use them "as is", i.e., without any change. The grammar is normally developed inside IAN, because we have to test it every time. But you may also prepare a draft grammar, save it in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>T-RULES.
   2. Create the NL-UNL (analysis) disambiguation grammar in order to improve the results of the transformation and to control the process of tokenization. Again, this grammar is normally developed inside IAN (at the D-RULES tab), because we have to test it every time. Consult the English Grammar for a detailed example of a disambiguation grammar.
   3. Test the grammar against the corpus and provide the necessary changes.

Assessment

The actual output must be evaluated against the expected outputs using the F-Measure, which can be automatically calcuated at UNLWEB>UNLARIUM>TOOLS>F-MEASURE In order to obtain the F-measure you will have to provide the following files:

• Actual output: the output provided by IAN, in your language, with the resources that you have provided, for the translated version of UCA1
• Expected output: UC-A1 in UNL

Samples and Examples

The following resources have been used to deal with UCA1 in English and may be used as a sample of what is expected to be provided

Recommended Readings

Before starting the activity, and in order to fully understand what is expected to be done, it is important for you to be acquainted with the following documentation:

• Tagset, because you are expected to use only the tags included in the tagset
• UNL Dictionary Specs, which is essential to understand the dictionary structure
• UNL Grammar Specs, which is essential to understand the grammar structure
• X-bar
• Default Dictionary
• English dictionary
• Default grammar
• English grammar

It is also interesting to make a test drive with IAN.

Notes

1. ↑ Two dictionaries are necessary for each language: the language-specific dictionary, and the Default Dictionary, which contains language-independent entries, such as punctuation signs and regular expressions. The default dictionary must be loaded after the language-specific dictionary.
2. ↑ Three t-grammars are necessary for each language: the Standardization grammar, the language-specific grammar, and the Default grammar. The standardization grammar and the default grammar are language-independent. The grammars must be loaded in this order: 1) standardization, 2) language-specific, and 3) default.