Grammar Specs

Revision as of 20:11, 16 August 2013 (view source) Martins (Talk | contribs) (→Basic Concepts) ← Older edit		Revision as of 17:54, 19 August 2013 (view source) Martins (Talk | contribs) Newer edit →
Line 16:		Line 16:
	;[[Hyper-Relation]]		;[[Hyper-Relation]]
	:A hyper-relation is a relation between relations.		:A hyper-relation is a relation between relations.
		+
		+	== [[Nodes]]s ==
		+	{{:Node}}s

---

Revision as of 17:54, 19 August 2013

The following Grammar Specs are used for writing rules for the UNDL Foundation tools (IAN, EUGENE, SEAN, NORMA, etc.).

Contents 1 Basic Symbols 2 Basic Concepts 3 Nodess 4 Basic Symbols 5 Basic Concepts 6 Notation 7 Elements 8 Example 9 Indexation 10 Properties 11 "Strings" or [headwords]? 12 Transformations over nodes 12.1 Altering elements of nodes 12.2 Deleting nodes 12.3 Creating nodes 12.4 Duplicating (cloning) nodes 12.5 Splitting nodes 12.6 Merging nodes (&) 12.7 Retrieving entries in the dictionary after tokenization (?) 12.8 Triggering rules (!) 13 Notes

Basic Symbols

Basic symbols used in the UNL framework

Symbol	Definition	Example
( )	node	(%a)
" "	string	"went"
[ ]	natural language entry (headword)	[go]
[[ ]]	UW	[[to go(icl>to move)]]
//	regular expression	/a{2,3}/ = aa,aaa
rel(x;y)	relation	agt(kill;Peter)
^	not	^a = not a
{ | }	or	{a|b} = a or b
%	index for nodes, attributes and values	%x
:	scope ID	:01
#	index for sub-NLWs	#01
=	attribute-value assignment	POS=NOU
!	rule trigger	!PLR
&	merge operator	%x&%y
?	dictionary lookup operator	?[a]

Basic Concepts

Node

A node is the most elementary unit in the graph. It is the result of the tokenization process, and corresponds to the notion of "lexical item". At the surface level, a natural language sentence is considered a list of nodes, and a UNL graph a set of relations between nodes.

Relation

In order to form a natural language sentence or a UNL graph, nodes are inter-related by relations.

Hyper-Node

A hyper-node is a sub-graph, i.e., a node containing relations between nodes.

Hyper-Relation

A hyper-relation is a relation between relations.

Nodess

A node is the most elementary unit in the grammar. It is the result of the tokenization process, and corresponds to the notion of "lexical item". At the surface level, a natural language sentence is considered a list of nodes, and a UNL graph a set of relations between nodes.

Basic Symbols

Basic symbols used in the UNL framework

Symbol	Definition	Example
( )	node	(%a)
" "	string	"went"
[ ]	natural language entry (headword)	[go]
[[ ]]	UW	[[to go(icl>to move)]]
//	regular expression	/a{2,3}/ = aa,aaa
rel(x;y)	relation	agt(kill;Peter)
^	not	^a = not a
{ | }	or	{a|b} = a or b
%	index for nodes, attributes and values	%x
:	scope ID	:01
#	index for sub-NLWs	#01
=	attribute-value assignment	POS=NOU
!	rule trigger	!PLR
&	merge operator	%x&%y
?	dictionary lookup operator	?[a]

Basic Concepts

Node

A node is the most elementary unit in the graph. It is the result of the tokenization process, and corresponds to the notion of "lexical item". At the surface level, a natural language sentence is considered a list of nodes, and a UNL graph a set of relations between nodes.

Relation

In order to form a natural language sentence or a UNL graph, nodes are inter-related by relations. In the UNL framework, there are three different types of relations: the linear (list) relation, syntactic relations and semantic relations.

Hyper-Node

A hyper-node is a sub-graph, i.e., a scope: a node containing relations between nodes.

Hyper-Relation

A hyper-relation is a relation between relations.

Notation

Nodes are represented between (parentheses).
Each node may have several different elements, which are listed inside parentheses, isolated by comma.
Examples of nodes:

• ("a")
• ([a])
• ([[a]])
• (NOU)
• (POS=NOU)
• ("a",[a],[[a]],LEX=N,POS=NOU,GEN=MCL,NUM=SNG)

Nodes are related by relations. In a relation, different nodes are isolated by ";"

• rel("a";"b") (a relation rel between the nodes ("a") and ("b"))

Elements

Any node is a structure containing the following necessary elements:

• a string, to be represented between "quotes", which expresses the actual state of the node;
• a headword, to be represented between [square brackets], which expresses the original value of the node in the dictionary;
• a UW, to be represented between [[double square brackets]], which expresses the UW value of the node;
• a feature or set of features, which express the features of the node;
• an Index, preceded by the symbol %, which is used to reference the node.

The elements of a node can be:

• native, if defined in the dictionary; or
• non-native, if assigned by transformation rules.

Example

Consider the input string "an apple" and the dictionary^[1] below:

[an]{111}""(LEX=D,POS=ART)<eng,0,0>;

[ ]{3333}""(LEX=O,POS=PUT,BLK)<eng,0,)>;

[apple]{222}"apple(icl>fruit)"(LEX=N,POS=NOU)<eng,0,0>;

In the tokenization process, the input string is segmented into nodes according to the dictionary. This means that the input string above is analyzed as a list of three nodes:

("an",[an],[[]],LEX=D,POS=ART)

(" ",[ ],[[]],LEX=O,POS=PUT,BLK)

("apple",[apple],[[apple(icl>fruit)]],LEX=N,POS=NOU)

Each node consists of:

• a string, between quotes ("an", " ", "apple");
• a headword, between brackets ([an],[ ],[apple]);
• a UW, between double brackets ([[]],[[]],[[apple(icl>fruit)]]);
• a set of features (LEX=D, POS=ART, LEX=O, ..., BLK)

These elements are said to be native because they are inherited from the dictionary.
During the processing, we may change any of these elements with T-rules. The resulting new elements are said to be non-native because they are assigned by rules.
Consider, for instance, the example below:

• INITIAL STATE: ("an",[an],[[]],LEX=D,POS=ART)
• RULE APPLIED: ("an"):=("a");^[2]
• FINAL STATE: ("a",[an],[[]],LEX=D,POS=ART)

Note, in the above, that the node changed its string value from "an" to "a". As this was the only change intended, the rule referred only to the string value of the node ("an").
In addition to changing the string, we could have changed any element of the node:

• (ART):=(-ART); (delete the feature ART from the nodes having the feature ART)
• (ART,^NDEF):=(+NDEF); (add the feature NDEF to the nodes having the feature ART and not having the feature NDEF)
• ("an",ART,^NDEF):=("a",-ART,+NDEF); (set the string to "a", remove the feature ART and add the feature NDEF to the nodes having the feature ART and not having the feature NDEF whose string is "an").
  

Indexation

main article: Indexation

In most cases, we have to assign an index to the node. This happens when we want to perform operations over nodes instead of elements of nodes.
Consider, for instance, the need for reversing the order of the input string "an apple" in order to generate "apple an". If we write a rule as:

• ("an")(" ")("apple"):=("apple")(" ")("an");

We would have the following output:

("apple",[an],[[]],LEX=D,POS=ART)

(" ",[ ],[[]],LEX=O,POS=PUT,BLK)

("an",[apple],[[apple(icl>fruit)]],LEX=N,POS=NOU)

Note, in the above, that we have simply replaced the string "an" by "apple", and "apple" by "an", preserving all the other features, which is not the intended behavior (after the rule, "apple" is ART and "an" is NOU).
In order to manipulate entire nodes (and not only some elements), we have to create indexes such as:

• ("an",%index1)(" ",%index2)("apple",%index3):=(%index3)(%index2)(%index1);

In this case, the output would be the expected one:

("apple",[apple],[[apple(icl>fruit)]],LEX=N,POS=NOU)

(" ",[ ],[[]],LEX=O,POS=PUT,BLK)

("an",[an],[[]],LEX=D,POS=ART)

Indexes, which are introduced by the symbol %, are always temporary (they are valid only within rules using them) and are used for co-indexing nodes. For further information on indexes, see indexation.

Properties

1. Nodes are enclosed between (parentheses)

   ("a") is a node

   "a" is not a node

2. Inside relations, parentheses are not duplicated, except in case of hyper-nodes

   rel("a";"b") (relation rel between the nodes ("a") and ("b"))

   rel(("a");("b"))

3. The elements of a node are separated by comma

   ("a",[a],[[a]],A,B,A=C,%a)

4. The order of elements inside a node is not relevant.

   ("a",[a],[[a]],A,B,A=C,%a) is the same as ([[a]],B,A,"a",[a],A=C,%a)

5. Nodes may have one single string, headword, UW and index, but may have as many features as necessary

   ("a","b") (a node may not contain more than one string)

   ([a],[b]) (a node may not contain more than one headword)

   ([[a]],[[b]]) (a node may not contain more than one UW)

   (%a,%b) (a node may not contain more than one index)

   (A,B,C,D,...,Z) (a node may contain as many features as necessary)

6. A node may be referred by any of its elements, but only the index make it unique

   ("a") refers to all nodes where actual string = "a"

   ([a]) refers to all nodes where headword = [a]

   ([[a]]) refers to all nodes where UW = [[a]]

   (A) refers to all nodes having the feature A

   ("a",[a],[[a]],A) refers to all nodes having the feature A where string = "a" and headword = [a] and UW = [[a]]

   (%a) refers to the specific node with the index %a

7. Nodes are automatically indexed according to a position-based system if no explicit index is provided (see Indexation)

   ("a")("b") is actually ("a",%01)("b",%02)

8. Regular expressions may be used to make reference to any element of the node, except the index

   ("/a{2,3}/") refers to all nodes where string is a sequence of 2 to 3 characters "a"

   ([/a{2,3}/]) refers to all nodes where headword is a sequence of 2 to 3 characters "a"

   ([[/a{2,3}/]]) refers to all nodes where UW is a sequence of 2 to 3 characters "a"

   (/a{2,3}/) refers to all nodes having a feature that is a sequence of 2 to 3 characters "a"

9. Nodes may contain disjoint features enclosed between {braces} and separated by vertical bar

   ({A|B}) refers to all nodes having the feature A OR B

10. Node features may be expressed as simple attributes, or attribute-value pairs:

    (MCL) - feature as an attribute: refers to all nodes having the feature MCL

    (GEN=MCL) - feature as an attribute-value pair, which is the same as (GEN,MCL): refers to all nodes having the features GEN and MCL.

11. Attribute-value pairs may be used to create co-reference between different nodes (as in agreement):

    (%x,GEN)(%y,GEN=%x) - the value of the attribute GEN of the node %x is the same as the attribute GEN of the node %y (see Indexation)

"Strings" or [headwords]?

• "Double quotes" are always used to represent strings: "a" will match only the string "a"
• [Simple square brackets] are always used to represent natural language entries (headwords) in the dictionary

In the initial state, i.e., right after tokenization, the "string" and the [headword] of a node are the same. During the processing, however, they can become different.
Consider, for instance, the case of "an apple" above. Note that, after applying the rule:

("an"):=("a");

The resulting node is:

("a",[an],[[]],LEX=D,POS=ART)

Note, in the above, that the string is now "a" whereas the headword is still [an]. We could have also changed the headword as in:

("an",[an]):=("a",[a]);

But it is normally interesting to preserve the headword in order to keep the track of the dictionary entries being used (regardless of their string changes).
In this case, if we want to make reference to the actual string value of the node, we use "quotes"; if we want to make reference to the string as it was retrieved in the dictionary, we use [brackets].

Transformations over nodes

Nodes are altered, replaced, created and deleted by T-rules:

Altering elements of nodes

Any changes to the elements of nodes must be stated in the right side of rules. Changes affect only the elements explicitly indicated in rules:

("a",[a],[[a]],A):=("b"); (only the string is affected by the rule; all other elements are preserved. The resulting node is ("b",[a],[[a]],A) )

"Strings"

Nodes may have one single string. This value is set through the operator "+", reset through the operator "-", or modified through A-rules. The operator "+" may be omitted. Changes to strings do not affect any other element (headwords, UWs and features)

• (""):=(+"a"); (the string of the node is set from "" to "a")
• (""):=("a"); (the same as above)
• ():=("a"); (the same as above)
• ("a"):=(+"b"); (the string of the node is set from "a" to "b")
• ("a"):=("b"); (the same as above)
• ("a"):=(-"a"); (the string of the node is reset, i.e., it changes from "a" to "")
• ("a"):=(""); (the same as above)
• ("a"):=("x"<0,0>"y"); (the string of the node is modified from "a" to "xay")

[headwords]

Nodes may have one single headword. This value is set through the operator "+" and reset through the operator "-". The operator "+" may be omitted. Headwords may not be modified through A-rules. Changes to headwords do not affect any other element (strings, UWs and features)

• ([]):=(+[a]); (the headword of the node is set from [] to [a])
• ([]):=([a]); (the same as above)
• ():=([a]); (the same as above)
• ([a]):=(+[b]); (the headword of the node is set from [a] to [b])
• ([a]):=([b]); (the same as above)
• ([a]):=(-[a]); (the headword of the node is reset, i.e., it changes from [a] to [])
• ([a]):=([]); (the same as above)
• ([a]):=("x"<0,0>"y"); (it is not possible to modify headwords through A-rules.

[[UWs]]

Nodes may have one single UW. This value is set through the operator "+" and reset through the operator "-". The operator "+" may be omitted. UWs may not be modified through A-rules. Changes to UWs do not affect any other element (strings, headwords and features)

• ([[]]):=(+[[a]]); (the UW of the node is set from [[]] to [[a]])
• ([[]]):=([[a]]); (the same as above)
• ():=([[a]]); (the same as above)
• ([[a]]):=(+[[b]]); (the UW of the node is set from [[a]] to [[b]])
• ([[a]]):=([[b]]); (the same as above)
• ([[a]]):=(-[[a]]); (the UW of the node is reset, i.e., it changes from [[a]] to [[]])
• ([[a]]):=([[]]); (the same as above)
• ([a]):=("x"<0,0>"y"); (it is not possible to modify UWs through A-rules.

Features

Nodes may have as many features as necessary. Features may come isolated in a list (POS,NOU,GEN,MCL,NUM,SNG) or as pairs of attribute=value (POS=NOU,GEN=MCL,NUM=SNG). Changes to features do not affect any other element (strings, headwords and UWs)

• Adding features to nodes

  Features are added through the operator + (add). The operator "+" may be omitted.

  • ():=(+B); (add the feature B to the node)
  • ():=(B); (the same as above)

  Rules are recursive: the feature will be added to the node while the condition is true.

  • ():=(+B); (the resulting node is (B,B,B,...), i.e., this is an infinite loop)
  • (^B):=(+B); (the resulting node is (B), i.e., the feature B is added only if the node does not contain it yet)

  The operator + (add) does not create attribute=value pairs automatically (it simply adds features to the nodes)

  • (%x,^POS,^NOU):=(%x,+NOU); (the resulting node is (NOU) and not (POS=NOU) because POS has not been added)
  • (%x,^POS,^NOU):=(%x,+POS,+NOU); (the resulting node is (POS,NOU) and not (POS=NOU) because there was no assignment POS=NOU)
  • (%x,POS,^NOU):=(%x,+NOU); (the resulting node is (POS,NOU) because there was no assignment POS=NOU)
  • (%x,^POS,^NOU):=(%x,+POS=NOU); (the resulting node is (POS=NOU))
  • (%x,POS,^NOU):=(%x,+POS=NOU); (the resulting node is (POS, POS=NOU) because the feature POS has been duplicated)

• Deleting features from nodes

  Features are deleted through the operator - (delete).

  ():=(-B); (delete the feature B from the node)

  Rules are recursive: the feature will be deleted from the node while the condition is true.

  ():=(-B); (the rule will delete all instances of the feature B from the node, i.e., the node (B,B,B,B,B) will become ()

  The operator "-" may also be used to reset attributes:

  • (%x,POS,NOU):=(%x,-NOU); (the resulting node is (POS) because the feature POS was not deleted)
  • (%x,POS,NOU):=(%x,-POS,-NOU); (the resulting node is () because both features POS and NOU were deleted)
  • (%x,POS=NOU):=(%x,-NOU); (the resulting node is (POS) because only the value of the attribute POS was deleted)
  • (%x,POS=NOU):=(%x,-POS); (the resulting node is () because the attribute POS was deleted with all its values)

• Copying features

  Features can be copied from one to another node through indexes

  • (%x,GEN)(%y,^GEN):=(%x)(%y,GEN=%x); (the value of the attribute GEN is copied from the node %x to %y);

Indexes

Indexes are used to make reference to the whole node instead of its elements. Any change in the index means a completely new node, and no element is preserved.

• (%x,"a"):=(%x,"b"); (the string of the node %x is set from "a" to "b"; all the other elements of the node %x are preserved)
• (%x,"a"):=(%y,"b"); (the whole node %x is replaced by a new node %y whose string is "b"; no element from %x is copied to %y)

Deleting nodes

In linear rules, nodes are deleted if they are not repeated (co-indexed) in the right side:

• (%x)(%y):=(%x); (the node %y will be deleted)

In other rules, nodes are deleted if they are not repeated (co-indexed) in the right side and are not part of any other relation:

• rel(%x;%y):=rel(%x); (the node %y will be deleted if, and only if, it is not part of any other relation)

Creating nodes

Nodes are created through the use of new indexes in the right side:

• ("a",%x)("b",%y):=(%x)(%y)("c",%z); (the node %z will be created)
• ("a",%x)("b",%y):=(%x)("c",%z); (the node %z will be created, and %y will be deleted)

Duplicating (cloning) nodes

Nodes may be duplicated by repeating indexes on the right side along with the command #CLONE:

• ("a",^CLONED,%x):=(%x,+CLONED)(%x,+CLONED,#CLONE);

  ("a") becomes ("a")("a")
  

In order to avoid infinite recursion, it is important to change the condition on the right side (in the example above, the feature +CLONED, assigned to all instances of the clone, prevents the rule from applying indefinitely)
Clones contain the same elements of the original nodes, unless they are explicitly altered during the cloning:

• ("a",[a],[[a]],A,^CLONED,%x):=(%x,+CLONED)(%x,+CLONED,#CLONE);

  ("a",[a],[[a]],A) becomes ("a",[a],[[a]],A,CLONED)("a",[a],[[a]],A,CLONED)

• (A,^CLONED,%x):=(%x,-A,+B,+CLONED)(%x,-A,+C,+CLONED,#CLONE);

  (A) becomes (B,CLONED)(C,CLONED)

Splitting nodes

One node may be split into two or more nodes through the use of splitting rules. Consider, for instance, the cases below:

Splitting rules deal only with strings and apply only to nodes with the feature TEMP.

Original node: ("abc",TEMP)

Split rule: ("abc"):=("ab")("c");

Resulting nodes: ("ab",TEMP)("c",TEMP);

However, if the original node was ("abc"), without TEMP, the rule would not have been applied (i.e., it is necessary to assign the feature TEMP to the node before splitting it)

Splitting rules are conservative: the elements of the original node, except the string, will be preserved unless explicitly altered.

Original node: ("abc",[abc],[[abc]],A,B,C,TEMP)

Split rule: ("abc"):=("ab")("c");

Resulting nodes: ("ab",[abc],[[abc]],A,B,C,TEMP)("c",[abc],[[abc]],A,B,C,TEMP) (i.e., the elements of the original node will be copied to the new nodes)

However, if the rule was: ("abc"):=("ab",-A,-B,-C,-TEMP,+AB)("c",-TEMP);

The result would be: ("ab",[abc],[[abc]],AB)("c",[abc],[[abc]],A,B,C)

Merging nodes (&)

Two or more nodes may be merged by the command &:

• (%x)(%y)(%z):=(%x&%y&%z);

In the example above("a")("b")("c") becomes ("abc")

Merge operations concatenate headwords and UWs, and join features

("hw1",[[uw1]],F1,%x)("hw2",[[uw2]],F2,%y)("hw3",[[uw3]],F3,%z):=(%x&%y&%z);
The resulting node is ("hw1hw2hw3",[[uw1uw2uw3]],F1,F2,F3)

Compare the difference

• (%x)(%y):=(%z); (the nodes %x and %y are replaced by %z, and their features are lost unless explicitly included in %z)
• (%x)(%y):=(%x&%y); (the nodes %x and %y are merged)

Retrieving entries in the dictionary after tokenization (?)

During transformation (i.e., after tokenization), dictionary entries may be accessed from transformation rules by the command "?"

• (?[headword]) retrieves the first entry in the dictionary with the headword "headword"
• (?[[uw]]) retrieves the first entry in the dictionary with the UW "uw"
• (?[headword],?[[uw]],?feature) retrieves the first entry in the dictionary with the headword "headword", the UW "uw" and the feature "feature"

Regular expressions, variables and disjunction may also be used in dictionary search

• (?[/abcd./]) retrieves the first entry in the dictionary whose headword has 5 characters and begins with "abcd" (this works only in natural language generation)
• (?[[/abcd./]]) retrieves the first entry in the dictionary whose UW has 5 characters and begins with "abcd" (this works only in natural language analysis)

Obligatory parameters

Due to the indexation algorithm, the headword is obligatory in IAN and the UW is obligatory in EUGENE:

• (?[headword]) will work only in IAN
• (?[[uw]]) will work only in EUGENE
• (?feature) will not work in IAN or EUGENE

Variables

In order to avoid repetition, dictionary look-up may use the values of indexed nodes in the left side

• (?[%x]) retrieves the first entry in the dictionary with the same headword of the node %x
• (?[[%x]]) retrieves the first entry in the dictionary with the same UW of the node %x
• (?[%x],ATT=%x) retrieves the first entry in the dictionary with the same headword of the node %x and whose attribute ATT has the same value of the attribute ATT of the node %x

Example

Dictionary search is used mainly in natural language generation

• (N,NUM,GEN,@def,%noun):=(?[[]],?ART,?DEF,?NUM=%noun,?GEN=%noun)(%noun,-@def);

In case of node %noun with the features noun (N), number (NUM) and gender (GEN), and with the attribute @def (definite), search the first entry in the dictionary associated with the UW "" (empty UW) with the features ART and DEF, and whose attributes NUM and GEN have the same values of the ones of the node %noun, and insert it in front of the noun. Remove @def from the noun in order to avoid an infinite loop.

Triggering rules (!)

Inflectional rules are triggered in the grammar by the command "!"<ATTRIBUTE>.
Given the dictionary entry:

• [foot] "foot" (POS=NOU, NUM(PLR:="oo":"ee")) <eng,0,0>;

The rule NUM(PLR:="oo":"ee") is triggered by !NUM
For instance:

• (NUM=PLR,^inflected):=(!NUM,+inflected); or
• (PLR,^inflected):=(!NUM,+inflected); or
• (NUM,^inflected):=(!NUM,+inflected);

In the first case (NUM=PLR), the system verifies if the attribute "NUM" is set and if it has the value "PLR". In the second and in the third case, the system simply verifies if the word has any feature (attribute or value) equal to "PLR" or "NUM".
It's important to stress that, as the features of the dictionary are defined by the user, there is no way of pre-assigning attribute-value pairs. In that sense, it's not possible to infer that "PLR" will be a value of the attribute "NUM" except through an assignment of the form "NUM=PLR" (i.e., given only "PLR" or "NUM", is not possible to state "NUM=PLR").

Notes

1. ↑ For the structure of the dictionary, please consult dictionary.
2. ↑ This is a T-rule and means: replace the string value from "an" to "a". For further information on T-rules, please consult T-rule.

s