|
|
| (4 intermediate revisions by one user not shown) |
| Line 1: |
Line 1: |
| − | The UC-A1 is an experimental corpus used to prepare the initial versions of the grammar for sentence-based [[UNLization]] and [[NLization]], using [[IAN]] and [[EUGENE]], respectively. It comprises a list of 100 structures in UNL, and is supposed to cover very basic linguistic phenomena. | + | The UC-A1 is an experimental corpus used to prepare the initial versions of the grammar for sentence-based [[UNLization]] and [[NLization]], using [[IAN]] and [[EUGENE]], respectively. It comprises a list of 50 structures in UNL, and is supposed to cover very basic linguistic phenomena. |
| | | | |
| | == The corpus == | | == The corpus == |
| − | *UC-A1 in one single file
| + | The corpus UCA1 was extracted from a simplified and translated version of "The Hare and the Tortoise", by Aesop. |
| − | **[http://www.unlweb.net/resources/corpus/UCA1/UCA1_eng.txt UC-A1 in English], to be (manually) translated to your target language in order to be used as the input for the UNLization process (with [[IAN]])
| + | *[http://www.unlweb.net/resources/UCA1/uca1_eng.txt UC-A1 in English], to be translated (manually) in order to be used as the input for the UNLization process (with [[IAN]]) |
| − | **[http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UC-A1 in UNL], to be used, "as is" (i.e., without any change), as the input for the NLization process (with [[EUGENE]])
| + | *[http://www.unlweb.net/resources/UCA1/uca1_unl.txt UC-A1 in UNL], to be used "as is", as the input for the NLization process (with [[EUGENE]]) |
| − | *UC-A1 in different files (the same as above, but divided according to the different classes)
| + | |
| − | {|+CorpusA1 border=1 align=center cellpadding=5
| + | |
| − | !Type
| + | |
| − | !English*
| + | |
| − | !UNL**
| + | |
| − | !Number of sentences
| + | |
| − | |-
| + | |
| − | |Temporary entries
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_TEMP_eng.txt UCA1_TEMP_eng.txt]***
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_TEMP_unl.txt UCA1_TEMP_unl.txt]
| + | |
| − | |align=center|5
| + | |
| − | |-
| + | |
| − | |Determiners
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_DET_eng.txt UCA1_DET_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_DET_unl.txt UCA1_DET_unl.txt]
| + | |
| − | |align=center|10
| + | |
| − | |-
| + | |
| − | |Possessive determiners
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_SPR_eng.txt UCA1_SPR_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_SPR_unl.txt UCA1_SPR_unl.txt]
| + | |
| − | |align=center|10
| + | |
| − | |-
| + | |
| − | |Prepositions
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_PRE_eng.txt UCA1_PRE_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_PRE_unl.txt UCA1_PRE_unl.txt]
| + | |
| − | |align=center|20
| + | |
| − | |-
| + | |
| − | |Conjunctions
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_CJC_eng.txt UCA1_CJC_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_CJC_unl.txt UCA1_CJC_unl.txt]
| + | |
| − | |align=center|10
| + | |
| − | |-
| + | |
| − | |Noun phrase structure
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_NP_eng.txt UCA1_NP_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_NP_unl.txt UCA1_NP_unl.txt]
| + | |
| − | |align=center|20
| + | |
| − | |-
| + | |
| − | |Verb forms
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_VER_eng.txt UCA1_VER_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_VER_unl.txt UCA1_VER_unl.txt]
| + | |
| − | |align=center|10
| + | |
| − | |-
| + | |
| − | |Sentence structures
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_S_eng.txt UCA1_S_eng.txt]
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_S_unl.txt UCA1_S_unl.txt]
| + | |
| − | |align=center|15
| + | |
| − | |}
| + | |
| − | <nowiki>*</nowiki>To be manually translated to your target language in order to be used as the input for UNLization (IAN)<br />
| + | |
| − | <nowiki>**</nowiki>To be used as the input for NLization (EUGENE)<br />
| + | |
| − | <nowiki>***</nowiki>Do not translate (or transliterate) the temporary entries<br />
| + | |
| − | | + | |
| − | == Goals ==
| + | |
| − | #To provide the dictionary and grammars necessary to '''UNLize''' your translated version of UC-A1 (with [[IAN]])
| + | |
| − | #To provide the dictionary and grammars necessary to '''NLize''', to your target language, the UNL version of UC-A1 (with [[EUGENE]])
| + | |
| − | | + | |
| − | == Methodology ==
| + | |
| − | In order to prepare the dictionaries and grammars to deal with the UC-A1, follow the steps below:
| + | |
| − | === UNLization ===
| + | |
| − | #Corpus
| + | |
| − | ##Translate (manually) the sentences of the [http://www.unlweb.net/resources/corpus/UCA1/UCA1_eng.txt UC-A1] from English into your native language. Be as close as possible to the original, and provide one single translation for each sentence. This will be your input document file, and your goal will be to provide (automatically, through [[IAN]]) the UNL graphs for each sentence
| + | |
| − | ##Save the translated text (without the English original) in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>NL FILES.
| + | |
| − | #Dictionary
| + | |
| − | ##Localize the dictionary available at [http://www.unlweb.net/resources/dic/UCA1/eng_unl_dic.txt eng_unl_dic.txt]. Note that "localization" is not the same as "translation". You may need other features (in English, for instance, nouns do not have gender or case) or other entries. In any case, the resulting dictionary should fit your translated version of the corpus (i.e., all the entries appearing in your translated version of the corpus should appear in the dictionary). For further information on localization, see [[Localization]]. For information on the dictionary structure, see [[Dictionary Specs]]. For an explanation of the structure of the English dictionary, see [[English_dictionary|English Dictionary]]. In case you need additional features, use only the tags available at the [[tagset]].
| + | |
| − | ##Save the NL-UNL dictionary in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>DICTIONARIES.
| + | |
| − | ##Upload the [http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary] to UNLWEB>UNLDEV>IAN>DICTIONARIES.
| + | |
| − | #Grammar
| + | |
| − | ##Create the NL-UNL (analysis) transformation grammar necessary to analyze, in UNL, the natural language sentences of the translated corpus. This grammar is the most difficult (and the actual goal) of the whole analysis task. In order to prepare the grammar, study the [[Grammar Specs]]. Next, take a look at the structure of the [[English_grammar|English Grammar]] for a detailed example. Note that the English Grammar is actually made of three different sub-grammars: the [[Standardization grammar]], the English Grammar itself, and the [[Default grammar]]. In many cases, it is simpler just to localize the English grammar to your own locale rather than creating a whole grammar from the scratch. See the instructions at [[Localization]]. The Standardization Grammar and the Default Grammar are supposed to be language-independent, and you may simply use them "as is", i.e., without any change. The grammar is normally developed inside IAN, because we have to test it every time. But you may also prepare a draft grammar, save it in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>IAN>T-RULES.
| + | |
| − | ##Create the NL-UNL (analysis) disambiguation grammar in order to improve the results of the transformation and to control the process of tokenization. Again, this grammar is normally developed inside IAN (at the D-RULES tab), because we have to test it every time. Consult the [[English_grammar|English Grammar]] for a detailed example of a disambiguation grammar.
| + | |
| − | ##Test the grammar against the corpus and provide the necessary changes.
| + | |
| − | | + | |
| − | === NLization ===
| + | |
| − | #Corpus
| + | |
| − | ##Upload the file [http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UCA1_unl.txt] to UNLWEB>UNLDEV>EUGENE>UNL DOCUMENT. Do not do any change to this file.
| + | |
| − | #Dictionary
| + | |
| − | ##Localize the dictionary available at [http://www.unlweb.net/resources/corpus/UCA1/unl_eng_dic.txt unl_eng_dic.txt] to your locale. Note, once again, that localization is not the same as translation. The localized version must reflect the word list of your translated corpus. In any case, use only the tags available at the [[tagset]]. For further information on the dictionary structure, see [[Dictionary Specs]].
| + | |
| − | ##Save the UNL-NL dictionary in a plain text (.txt) file with UTF-8 encoding and upload it to UNLWEB>UNLDEV>PROJECTS>EUGENE>DICTIONARIES.
| + | |
| − | ##Upload the [http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary] to UNLWEB>UNLDEV>GENERATION>DICTIONARIES.
| + | |
| − | #Grammar
| + | |
| − | ##Provide the UNL-NL (generation) T-grammar and D-grammar necessary to generate natural language sentences from the UNL corpus. Follow the same instructions available for the analysis grammar.
| + | |
| − | | + | |
| − | == Assessment ==
| + | |
| − | The actual outputs must be evaluated against the expected outputs using the [[F-Measure]], which can be automatically calcuated at UNLWEB>UNLARIUM>GRAMMAR>[LOCALE]>F-MEASURE
| + | |
| − | *UNLization
| + | |
| − | **Actual output: the output provided by IAN, in your language, with the resources that you have provided, for the translated version of UC-A1
| + | |
| − | **Expected output: [http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UC-A1 in UNL]
| + | |
| − | *NLization
| + | |
| − | **Actual output: the output provided by EUGENE, in your language, with the resources that you have provided, for the input file [http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UC-A1 in UNL]
| + | |
| − | **Expected output: the human-translated version of UC-A1 used as the input for the UNLization
| + | |
| − | | + | |
| − | == Samples and Examples ==
| + | |
| − | The following resources have been used to deal with UC-A1 and may be used as a sample of what is expected to be provided
| + | |
| − | {|border=1 cellpadding=5 align=center
| + | |
| − | |+UNLization
| + | |
| − | !Language
| + | |
| − | !Corpus
| + | |
| − | !Dictionary<ref>Two dictionaries are necessary for each language: the language-specific dictionary, and the [[Default Dictionary]], which contains language-independent entries, such as punctuation signs and regular expressions. The default dictionary must be loaded after the language-specific dictionary.</ref>
| + | |
| − | !T-Grammar<ref>Three t-grammars are necessary for each language: the [[Standardization grammar]], the language-specific grammar, and the [[Default grammar]]. The standardization grammar and the default grammar are language-independent. The grammars must be loaded in this order: 1) standardization, 2) language-specific, and 3) default. </ref>
| + | |
| − | !D-Grammar
| + | |
| − | !Output
| + | |
| − | |-
| + | |
| − | |align=center|English
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_eng.txt UC-A1 in English]
| + | |
| − | |[http://www.unlweb.net/resources/dic/UCA1/eng_unl_dic.txt ENG-UNL Dictionary]<br />[http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/s-grammar.txt Standardization Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/UCA1/eng_unl_tgrammar.txt ENG-UNL T-Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/nl_unl_tgrammar.txt Default T-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/UCA1/eng_unl_dgrammar.txt ENG-UNL D-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/output/UCA1/eng_unl.txt ENG>UNL]
| + | |
| − | |-
| + | |
| − | |align=center|French
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_fra.txt UC-A1 in French]
| + | |
| − | |[http://www.unlweb.net/resources/dic/UCA1/fra_unl_dic.txt FRA-UNL Dictionary]<br />[http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/s-grammar.txt Standardization Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/UCA1/fra_unl_tgrammar.txt FRA-UNL T-Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/nl_unl_tgrammar.txt Default T-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/UCA1/fra_unl_dgrammar.txt FRA-UNL D-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/output/UCA1/fra_unl.txt FRA>UNL]
| + | |
| − | |}
| + | |
| − | <br />
| + | |
| − | {|border=1 cellpadding=5 align=center
| + | |
| − | |+NLization
| + | |
| − | !Language
| + | |
| − | !Corpus
| + | |
| − | !Dictionary<ref>Two dictionaries are necessary for each language: the language-specific dictionary, and the [[Default Dictionary]], which contains language-independent entries, such as punctuation signs and regular expressions. The default dictionary must be loaded after the language-specific dictionary.</ref>
| + | |
| − | !T-Grammar<ref>Three t-grammars are necessary for each language: the [[Standardization grammar]], the language-specific grammar and the [[Default grammar]]. The standardization grammar and the default grammar are language-independent. The grammars must be loaded in this order: 1) standardization, 2) language-specific, 3) default.</ref>
| + | |
| − | !D-Grammar
| + | |
| − | !Output
| + | |
| − | |-
| + | |
| − | |align=center|English
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UC-A1 in UNL]
| + | |
| − | |[http://www.unlweb.net/resources/dic/UCA1/unl_eng_dic.txt UNL-ENG Dictionary]<br />[http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/s-grammar.txt Standardization Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/UCA1/unl_eng_tgrammar.txt UNL-ENG T-Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/unl_nl_tgrammar.txt Default T-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/UCA1/unl_eng_dgrammar.txt UNL-ENG D-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/output/UCA1/unl_eng.txt UNL>ENG]
| + | |
| − | |-
| + | |
| − | |align=center|French
| + | |
| − | |[http://www.unlweb.net/resources/corpus/UCA1/UCA1_unl.txt UC-A1 in UNL]
| + | |
| − | |[http://www.unlweb.net/resources/dic/UCA1/unl_fra_dic.txt UNL-FRA Dictionary]<br />[http://www.unlweb.net/resources/dic/default_dic.txt Default Dictionary]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/s-grammar.txt Standardization Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/UCA1/unl_fra_tgrammar.txt UNL-FRA T-Grammar]<br />
| + | |
| − | [http://www.unlweb.net/resources/grammar/unl_nl_tgrammar.txt Default T-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/grammar/UCA1/unl_fra_dgrammar.txt UNL-FRA D-Grammar]
| + | |
| − | |[http://www.unlweb.net/resources/output/UCA1/unl_fra.txt UNL>FRA]
| + | |
| − | |}<br />
| + | |
| − | | + | |
| − | == Recommended Readings ==
| + | |
| − | Before starting the activity, and in order to fully understand what is expected to be done, it is important for you to be acquainted with the following documentation:
| + | |
| − | *[[Tagset]], because you are expected to use only the tags included in the tagset
| + | |
| − | *[[UNL Dictionary Specs]], which is essential to understand the dictionary structure
| + | |
| − | *[[UNL Grammar Specs]], which is essential to understand the grammar structure
| + | |
| − | *[[X-bar]]
| + | |
| − | *[[Default Dictionary]]
| + | |
| − | *[[English dictionary]]
| + | |
| − | *[[Default grammar]]
| + | |
| − | *[[English grammar]]
| + | |
| − | It is also interesting to make a test drive with [[IAN]] and [[EUGENE]].
| + | |
| − | | + | |
| − | == Notes ==
| + | |
| − | <references/>
| + | |
The UC-A1 is an experimental corpus used to prepare the initial versions of the grammar for sentence-based UNLization and NLization, using IAN and EUGENE, respectively. It comprises a list of 50 structures in UNL, and is supposed to cover very basic linguistic phenomena.
The corpus UCA1 was extracted from a simplified and translated version of "The Hare and the Tortoise", by Aesop.
