F-measure
From UNL Wiki
(Difference between revisions)
(Created page with "In the UNL System, the F-measure (or F1-score) is the measure of a grammar's accuracy. It considers both the precision and the recall of the grammar to compute the score, acco...") |
|||
| Line 9: | Line 9: | ||
*In [[UNLization]], when the output is a graph (i.e., all the nodes are interlinked) made only of UW's (i.e., without natural language words) | *In [[UNLization]], when the output is a graph (i.e., all the nodes are interlinked) made only of UW's (i.e., without natural language words) | ||
*In [[NLization]], when the output is a list of natural language words (i.e., without any UW). | *In [[NLization]], when the output is a list of natural language words (i.e., without any UW). | ||
| − | A result is considered '''"correct"''' when the Levensthein distance between the actual result and the expected result was less than 30% of the length of the expected result. | + | A result is considered '''"correct"''' when the Levensthein distance between the actual result and the expected result was less than 30% of the length of the expected result. The Levenshtein distance is defined as the minimal number of characters you have to replace, insert or delete to transform a string (the actual output) into another one (the expected output). |
Revision as of 18:48, 18 September 2012
In the UNL System, the F-measure (or F1-score) is the measure of a grammar's accuracy. It considers both the precision and the recall of the grammar to compute the score, according to the formula
F-measure = 2 x ( (precision x recall) / (precision + recall) )
In the above:
- precision is the number of correct results divided by the number of all returned results
- recall is the number of correct results divided by the number of results that should have been returned
A result is considered "returned" in the following cases:
- In UNLization, when the output is a graph (i.e., all the nodes are interlinked) made only of UW's (i.e., without natural language words)
- In NLization, when the output is a list of natural language words (i.e., without any UW).
A result is considered "correct" when the Levensthein distance between the actual result and the expected result was less than 30% of the length of the expected result. The Levenshtein distance is defined as the minimal number of characters you have to replace, insert or delete to transform a string (the actual output) into another one (the expected output).
