ELRA - HLT Evaluation Workshop
Program
DAY 1:
1st December 2005
Introduction:
Bente Maegaard & Khalid Choukri: Celebrating
ELRA’s 10th anniversary, 1995-2005: workshop dedicated to human language
technology (HLT) evaluation...; 14pp. [PDF from PPT, 61KB]
Invited
speaker:
Mats Ljungqvist: Human language technology initiatives: EU
research programmes FP6 and FP7;
23pp. [PDF from PPT, 186KB]
Michael
Rosner: Towards HLT evaluation at the University
of Malta; 19pp. [PDF from PPT, 128KB]
Maghi King: Evaluation in human language
technology; 32pp. [PDF from PPT, 107KB]
Keith J. Miller: Evaluation principles and
objectives: ISLE, EAGLES (a play in three acts); 48pp. [PDF from PPT,
901KB]
Carol Peters: Evaluating information
retrieval systems: focusing on systems for cross language information retrieval;
47pp. [PDF from PPT, 561KB]
Christian Fluhr: Criticisms
and proposed evolutions of the information retrieval evaluation paradigm;
7pp. [PDF from PPT, 32KB]
Patrick Paroubek: Evaluation of text
analysis core technologies. Two successful examples: evaluating POS taggers and
parsers for French; 35pp. [PDF from PPT, 288KB]
Anja Belz & Ehud Reiter: Evaluation in NLG; 35pp. [PDF from PPT, 2032KB]
Tony
Hartley: MT evaluation – the little we know;
11pp. [PDF from PPT, 74KB]
Andrei Popescu-Belis: Some
questions from the discussant [of paper by Tony Hartley on MT evaluation];
8pp. [PDF from PPT, 48KB]
Nick Campbell: Speech
synthesis evaluation; 14pp. [PDF from PPT, 57KB]
Harald Höge: Evaluation of speech synthesis
technology; 2pp. [PDF from PPT, 16KB]
Edouard Geoffrois: Speech recognition evaluation: practices and
issues; 20pp. [PDF from PPT, 148KB]
DAY 2: 2nd December 2005
Gianni Lazzari: Speech to speech
translation; 110pp. [PDF of PPT presentation, 1091KB]
Jean Carletta: Some problems in
evaluating multimodal systems; 27pp. [PDF from PPT, 436KB]
Gregor Thurmair: Automatic
means of MT evaluation; 40pp. [PDF from PPT, 2974KB]
Evaluation
initiatives:
CELCT -- Amedeo Cappelli: CELCT: Center
for the Evaluation of Language and Communication Technologies; 13pp. [PDF
from PPT, 58KB]
CST -- Bente Maegaard: Evaluation and
validation activities at CST; 6pp. [PDF from PPT, 93KB]
ELRA -- Khalid Choukri: An evaluation infrastructure for HLT in Europe
– part 1; 8pp. [PDF from PPT, 34KB]
DFKI -- Anthony Jameson: The DFKI evaluation center for language technology
systems: from metric optimization to useful systems; 3pp. [PDF from PPT,
708KB]
French Ministry of Research -- Joseph Mariani: From basic research to innovative applications
through evaluation.; 15pp. [PDF from PPT, 95KB]
ISSCO -- M.King: ISSCO
and evaluation 1988-2005; 8pp. [PDF from PPT, 94KB]
Christopher Cieri: Evaluation initiatives: US activities in HLT
evaluation; 7pp. [PDF from PPT, 237KB]
Khalid Choukri: An
evaluation infrastructure for HLT in Europe – part 2; 23pp. [PDF from PPT,
100KB]
Christopher Cieri: HLT evaluation: role of data centers; 6pp. [PDF
from PPT, 195KB]
Wrap
up and further actions
Panel
[not available]