LETA Wins AMR Generation Trophy at SemEval 2017

LETA, as part of Team RIGOTRIO, has won the AMR GENERATION 2017 trophy at SemEval 2017, Task 9.

AMR GENERATION 2017 trophy

SemEval-2017, the International Workshop on Semantic Evaluation, was held August 3-4, 2017 at the ACL 2017 conference in Vancouver, Canada. SUMMA consortium partners University of Sheffield and LETA participated in the SemEval-2017 Task 9 on Meaning Represention Parsing and Generation. New to this competition was Subtask 2: AMR-to-English Generation, where participants had to reconstruct the valid English sentences from their formal Abstract Meaning Representation (AMR) graphs. Subtask 1: Parsing Biomedical Data was a rerun of the AMR parsing task from the last year competition, where LETA also achieved the top result. AMR parsing and generation are vital components of the semantic processing pipeline within the SUMMA project.

Team RIGOTRIO (LETA and University of Latvia) achieved the convincing top result in the Subtask 2 AMR-to-English Generation and was awarded the trophy (pictured) as per following Trueskill human evaluation ranking:

 

  1. RIGOTRIO (University of Latvia, IMCS; LETA): 1.07
  2. CMU (Carnegie Mellon University): 0.85
  3. FORGe (Universitat Pompeu Fabra; ICREA): 0.45
  4. ISI (University of Southern California, ISI): -1.19
  5. Sheffield (University of Sheffield): -2.20