CLEA1500: Difference between revisions

From UNLwiki
Jump to navigationJump to search
← Older editNewer edit →
VisualWikitext
imported>Martins
No edit summary
imported>Martins
No edit summary
Line 15: Line 15:
#Grammars
#Grammars
##Create the NL-UNL (analysis) grammars necessary to UNLize the natural language sentences of the translated corpus. These grammars are the most difficult (and the actual goal) of the whole analysis task. There can be three different types of grammar:
##Create the NL-UNL (analysis) grammars necessary to UNLize the natural language sentences of the translated corpus. These grammars are the most difficult (and the actual goal) of the whole analysis task. There can be three different types of grammar:
##*[[N-grammar]], which is used to segment the text into sentences (not necessary, because the corpus is already segmented) or to normalize the input text, resolving contractions, abbreviations and other phenomena that can be handled without the dictionary (i.e., by manipulating strings).
##*[[N-grammar]], which is used to segment the text into sentences<ref>This is not necessary, because the corpus is already segmented.</ref> or to normalize the input text, resolving contractions, abbreviations and other phenomena that can be handled without the dictionary (i.e., by manipulating strings).
##*[[T-grammar]], which is used to transform the list structure (i.e., the natural language sentence) into a graph structure (the UNL graph).
##*[[T-grammar]], which is used to transform the list structure (i.e., the natural language sentence) into a graph structure (the UNL graph).
##*[[D-grammar]], which is used to improve the performance of the T-grammar, by preventing the application of wrong rules or by provoking the application of right rules.
##*[[D-grammar]], which is used to improve the performance of the T-grammar, by preventing the application of wrong rules or by provoking the application of right rules.

Revision as of 18:18, 4 August 2014

CLEA1500 certifies that you have reached a standard of knowledge concerning the development of natural language grammars for UNLization and may officially participate in the UNL Research. The certificate consists of a set of 100 sentences that must be UNLized with IAN.

Goal

In CLEA1500, you are expected to provide the lingware (dictionary and grammars) necessary to UNLize a translated version of UCA1.

Instructions

  1. Training corpus
    1. Prepare the training corpus
      • If your native language is not English: translate (manually) the 100 sentences of the corpus UCA1 from English into your native language. Be as close as possible to the original, and provide one single translation for each sentence. This will be your input document file, and your goal will be to provide (automatically, through IAN) the UNL graphs for each sentence. See an example of input file here.
      • If your native language is English: use the corpus UCA2 as your input document file.
    2. Save the translated text (without the English original) in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>NL FILES.
  2. Dictionary
    1. Create a dictionary for your training corpus. Your dictionary must contain all and only the word forms appearing in your training corpus, and must comply with the Dictionary Specs.[1]
    2. Save the NL-UNL dictionary in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>DICTIONARIES.
  3. Grammars
    1. Create the NL-UNL (analysis) grammars necessary to UNLize the natural language sentences of the translated corpus. These grammars are the most difficult (and the actual goal) of the whole analysis task. There can be three different types of grammar:
      • N-grammar, which is used to segment the text into sentences[2] or to normalize the input text, resolving contractions, abbreviations and other phenomena that can be handled without the dictionary (i.e., by manipulating strings).
      • T-grammar, which is used to transform the list structure (i.e., the natural language sentence) into a graph structure (the UNL graph).
      • D-grammar, which is used to improve the performance of the T-grammar, by preventing the application of wrong rules or by provoking the application of right rules.
    2. Save the NL-UNL grammars in a plain text (.txt) file with UTF-8 encoding and upload them to the corresponding tabs in UNLWEB>UNLDEV>IAN.
    3. Test them and do the necessary changes until you get good results.Cite error: Closing </ref> missing for <ref> tag

!T-Grammar[3] !D-Grammar !Output |- |align=center|English |UC-A1 in English |ENG-UNL Dictionary
Default Dictionary |Standardization Grammar
ENG-UNL T-Grammar
Default T-Grammar |ENG-UNL D-Grammar |ENG>UNL |}

Before starting the activity, and in order to fully understand what is expected to be done, it is important for you to be acquainted with the following documentation:

It is also interesting to make a test drive with IAN.

Notes

  1. ↑ Instead of creating a whole new dictionary from scratch, you may try localizing the English dictionary available at eng_unl_dic.txt. Note that "localization" is not the same as "translation". You may need other features (in English, for instance, nouns do not have gender or case) or other entries. In any case, the resulting dictionary should fit your translated version of the corpus (i.e., all the entries appearing in your translated version of the corpus should appear in the dictionary). For further information on localization, see Localization. For information on the dictionary structure, see Dictonary Specs. For an explanation of the structure of the English dictionary, see English Dictionary. In case you need additional features, use only the tags available at the tagset.
  2. ↑ This is not necessary, because the corpus is already segmented.
  3. ↑ Three t-grammars are necessary for each language: the Standardization grammar, the language-specific grammar, and the Default grammar. The standardization grammar and the default grammar are language-independent. The grammars must be loaded in this order: 1) standardization, 2) language-specific, and 3) default.
Retrieved from "https://unlweb.net.br/wiki/index.php?title=CLEA1500&oldid=16529"