Machine Translation Archive

Index to evaluation measures

Publications 1970-1989

For other periods go to: Publications since 2010; publications 2005-2009; publications 2000-2004; publications 1990-1999; publications before 1970

 

Error detection and correction

(1988) Fujio Nishida, Shinobu Takamatsu, Tadaaki Tani, & Tsunehisa Doi: Feedback of correcting information in postediting to a machine translation system. Coling Budapest: Proceedings of the 12th International Conference on Computational Linguistics, 22-27 August 1988, John von Neumann Society for Computing Sciences, Budapest, Hungary; vol.2, pp.476-481. [PDF, 691KB]

(1984) Anne-Marie Laurian: Machine translation: what type of post-editing on what type of documents for what type of users. Coling84: 10th International Conference on Computational Linguistics & 22nd Annual Meeting of the Association for Computational Linguistics, 2-6 July 1984, Stanford University, California. Proceedings; pp.236-238. [PDF, 216KB]

(1984) Peter J.Wheeler: Changes and improvements to the European Commisssion’s Systran system 1976/84. International Conference on Methodology and Techniques of Machine Translation: processing from words to language… Cranfield, 13-15 February 1984, Natural Language Translation Specialist Group of the British Computer Society; 12pp. [PDF, 48KB] [Printed in: Ian D.K.Kelly (ed.) Progress in machine translation: natural language and personal computers (Wilmslow, UK: Sigma Press, 1989); pp.69-82.]

(1981) J. Albert Bachrach: Introductory remarks [to Session 3: The posteditors’ experience.] Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 99-100. [PDF, 11KB]

(1981) Roy Green: The MT errors which cause most trouble to posteditors. Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 101-104. [PDF, 15KB]

(1981) Barbara Kostrewski & Peter Taylor: Discussion on Session 3: The posteditors’ experience.  Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); [part 1] pp. 111-112. [PDF, 12KB]; [part 2] p.127 [PDF, 8KB]; [part 3] pp.135-136 [PDF, 10KB]

Evaluation measures and metrics

(1989) Shun Ishizaki & Hitoshi Isahara: Machine translation technology: evaluation and selected sentences on which it is based. IFTT’89: Harmonizing human beings and computers in translation. International Forum for Translation Technology, April 26-28, 1989, Oiso Prince Hotel. (Japan Electronic Industry Development Association.); pp.19-21. [PDF, 40KB]

(1989) Veronica Lawson: Practice makes less imperfect: users’ needs and their influence on machine translation development. Georgetown University Round Table on Languages and Linguistics 1989: “Language teaching, testing, and technology: lessons from the past with a view toward the future”, James E.Alatis, ed. (Washington, D.C.: Georgetown University Press, 1989); pp.51-58. [PDF, 54KB]

(1989) Rita McCardell: [review of] Machine translation: linguistic characteristics of MT systems and general methodology of evaluation [by] John Lehrberger and Laurent Bourbeau (John Benjamins Publishing Co., 1988).  Computational Linguistics 15 (3), pp. 194-195 [PDF, 253KB]

(1989) National Research Council: Report of a symposium on Japanese to English machine translation, held at the National Academy of Sciences in Washington, D.C. on December 7, 1989. (Washington, D.C.: National Academy Press, 1990.) xiii, 36pp. [PDF, 239KB]

(1987) Hirosato Nomura: Technical prospects of machine translation. Presentation at MT Summit, September 1987, not published in proceedings; 5pp. [PDF, 178KB]

(1981) J.C.Sager: Types of translation and text forms in the environment of machine translation (MT). Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 11-19. [PDF, 47KB]

(1979) Georges van Slype: Critical study of methods for evaluating the quality of machine translation. Final report. Brussels: Bureau Marcel van Dijk [for] European Commission. [PDF, 887KB]

(1977) A.J. Petit: Objective evaluation criteria for machine translation. Overcoming the language barrier: third European Congress on Information Systems and Networks, Luxembourg, 3-6 May 1977, organised by the Commission of the European Communities (München: Verlag Dokumentation, 1977); vol.1, pp.557-567. [PDF, 108KB]

Evaluations of systems

 (1989) Claude Bédard: GTS: new and affordable machine translation. Electric Word 15, September-October 1989; pp.53-54. [PDF, 626KB]

(1989) Muriel Vasconcellos: Long-term data for an MT policy. Literary and Linguistic Computing 4 (3), 1989; pp.203-213. [PDF, 1313KB]

(1988) Alan K. Melby: Lexical transfer: between a source rock and a hard target. Coling Budapest: Proceedings of the 12th International Conference on Computational Linguistics, 22-27 August 1988, John von Neumann Society for Computing Sciences, Budapest, Hungary; vol.2, pp.411-413. [PDF, 271KB]

 (1987) Eurotra: the day of reckoning. Language Technology 1, May-June 1987; pp.22-25. [PDF, 355KB]

 (1986) Student surveys MT technology. Language Monthly 36, September 1986; p.8. [PDF, 30KB]

(1985) Jonathan Slocum, Winfield S. Bennett, Lesley Whiffin, & Edda Norcross: An evaluation of METAL: the LRC machine translation system.  Second conference of the European Chapter of the Association for Computational Linguistics: Proceedings of the conference, 27-29 March 1985, University of Geneva, Geneva, Switzerland; pp. 62-69. [PDF, 812KB]

 (1984) Elliott Macklovitch: Recent Canadian experience in machine translation. International Conference on Methodology and Techniques of Machine Translation: processing from words to language… Cranfield, 13-15 February 1984, Natural Language Translation Specialist Group of the British Computer Society; 11pp. [PDF, 37KB]. [Printed in: Ian D.K.Kelly (ed.) Progress in machine translation: natural language and personal computers (Wilmslow, UK: Sigma Press, 1989); pp.59-67.]

(1984) Robert Ordish: Personal impressions of a machine translation system. The Incorporated Linguist vol.23, no.3, Summer 1984; pp.136-145. [PDF, 68KB]

 (1982) Ulla Magnusson-Murray: Computer-assisted translation at ITT: a talk given…15th April 1982. British Computer Society Natural Language Translation Specialist Group, Newsletter no 12, April 1982; pp.6-14. [PDF, 231KB]

(1979) Georges Van Slype: Systran: evaluation of the 1978 version of the Systran English-French automatic system of the Commission of the European Communities. The Incorporated Linguist, vol.18, no.3, Summer 1979, pp.86-89. [PDF, 106KB]

(1978) Peter J. Arthern: Machine translation and computerized terminology systems: a translator’s viewpoint. Translating and the Computer: proceedings of a seminar, London, 14th November, 1978; ed. Barbara M.Snell (Amsterdam/New York/Oxford: North-Holland Publishing Company, 1979); pp.77-108. [PDF, 253KB]

(1978) F. Knowles: Error analysis of Systran output – a suggested criterion for the ‘internal’ evaluation of translation quality and a possible corrective for system design. Translating and the Computer: proceedings of a seminar, London, 14th November, 1978; ed. Barbara M.Snell (Amsterdam/New York/Oxford: North-Holland Publishing Company, 1979); pp. 109-133. [PDF, 222KB]

(1977) Jacques Chaumier: Assessment of SYSTRAN. Overcoming the language barrier: third European Congress on Information Systems and Networks, Luxembourg, 3-6 May 1977, organised by the Commission of the European Communities (München: Verlag Dokumentation, 1978); vol.2, pp.74-82. [PDF, 135KB]

(1971) H.Wallace Sinaiko: Translation by computer. Science 174 (1971), 1182 & 1184. [PDF, 22KB]

Examples of MT output

(1989) Ch.Boitet: Current projects at GETA on or about machine translation. In: J.A.Campbell and J.Cuena (eds.) Perspectives in artificial intelligence, vol.II: machine translation, NLP, databases and computer-aided instruction (Chichester: Ellis Horwood, 1989); pp.48-73. [PDF, 198KB]

(1989) Victor Sadler: Translating with a simulated bilingual knowledge bank (BKB). [Utrecht]: Buroo vor Systeemontwikkeling, April 1989; 15pp. [PDF, 399KB]

(1989) Muriel Vasconcellos: Cohesion and coherence in the presentation of machine translation products. Georgetown University Round Table on Languages and Linguistics 1989: “Language teaching, testing, and technology: lessons from the past with a view toward the future”, James E.Alatis, ed. (Washington, D.C.: Georgetown University Press, 1989); pp.89-105. [PDF, 159KB]

(1988) P.J.Pym: Pre-editing and the use of simplified writing for MT: an engineer’s experience of operating an MT system.  Translating and the Computer 10:The translation environment 10 years on, ed. Pamela Mayorcas. Proceedings of a conference… 10-11 November 1988, CBI Conference Centre, Centre Point, London (London: Aslib, 1990); pp.80-96. [PDF, 146KB]

(1987) Ian M.Pigott: Systran machine translation processing as an example of natural language understanding. Terminologie et Traduction, 3.1987; pp.21-31. [PDF, 36KB]

(1986) Christian Boitet: Current machine translation systems developed with GETA’s methodology and software tools.  Translating and the Computer 8: a profession on the move, ed. Catriona Picken. Proceedings of a conference… 13-14 November 1986, CBI Conference Centre, Centre Point, London (London: Aslib, 1987); pp.114-132. [PDF, 169KB]

(1986) Andrew D.C.Evans: Systran – the translator’s viewpoint. (World Systran Conference, 11-14 February 1986; Commisssion des Communautés Européennes, Luxembourg). Terminologie et Traduction, 1, 1986; pp.17-23. [PDF, 57KB]

(1986) F.W.A.Habermann: Provision and use of raw machine translation. (World Systran Conference, 11-14 February 1986; Commisssion des Communautés Européennes, Luxembourg). Terminologie et Traduction, 1, 1986; pp.29-42. [PDF, 121KB]

(1986) Makoto Nagao, Jun’ichi Tsujii & Jun’ichi Nakamura: Science and Technology Agency’s Mu machine translation project. Future Generations Computer Systems 2 (1986); pp.125-139. [PDF, 494KB]

(1986) Muriel Vasconcellos: Application of MT in an integrated environment: update on the Pan American Health Organization. American Translators Association Conference – 1986: Proceedings of the 27th annual conference of the American Translators Association, Cleveland, Ohio, October 16-19, 1986, ed. Karl Kummer (Medford, NJ: Learned Information, Inc., 1986); pp.129-133. [PDF, 2954KB]

(1986) Muriel Vasconcellos: Post-editing on-screen: machine translation from Spanish into English.  Translating and the Computer 8: a profession on the move, ed. Catriona Picken. Proceedings of a conference… 13-14 November 1986, CBI Conference Centre, Centre Point, London (London: Aslib, 1987); pp.133-146. [PDF, 93KB]

(1985) F.W.A.Habermann: MT with Systran at Karlsruhe Nuclear Research Center.  Terminologie et Traduction, 2, 1985; pp.1-8. [PDF, 83KB]

(1984) F.W.A.Habermann: Application of SYSTRAN for translation of nuclear technology texts at the Nuclear Center of Karlsruhe [abstract and slides]. International Conference on Methodology and Techniques of Machine Translation: processing from words to language… Cranfield, 13-15 February 1984, Natural Language Translation Specialist Group of the British Computer Society; 14pp. [PDF, 124KB]

(1984) F.W.A. Habermann: Systran user. In: Product Forum. Translation and Communication: Translating and the Computer 6. Proceedings of a conference … 20-21 November 1984, The Waldorf Hotel, London; pp.123-124.

(1984) Robert Ordish: Personal impressions of a machine translation system. The Incorporated Linguist vol.23, no.3, Summer 1984; pp.136-145. [PDF, 68KB]

(1984) Allen B.Tucker, Jr.: A perspective on machine translation: theory and practice. Communications of the ACM 27 (4); pp.322-329. [note]

(1983) Klaus E.Tschira: Looking back at a year of German-English MT with Logos. Tools for the Trade: Translating and the Computer 5. Proceedings of a conference… 10-11 November 1983, London Press Centre, ed. Veronica Lawson. (London: Aslib, 1985); pp.215-235 [PDF, 200KB]

(1983) Muriel Vasconcellos: Management of the machine translation environment: interaction of functions at the Pan American Health Organization.  Tools for the Trade: Translating and the Computer 5. Proceedings of a conference… 10-11 November 1983, London Press Centre, ed. Veronica Lawson. (London: Aslib, 1985); pp.115-129 [PDF, 90KB]

(1983) Elizabeth Wagner: Rapid post-editing of Systran. Tools for the Trade: Translating and the Computer 5. Proceedings of a conference… 10-11 November 1983, London Press Centre, ed. Veronica Lawson. (London: Aslib, 1985); pp.199-213 [PDF, 98KB]

(1983) P.A.Walker: A Commission of the European Communities user looks at machine translation. Tools for the Trade: Translating and the Computer 5. Proceedings of a conference… 10-11 November 1983, London Press Centre, ed. Veronica Lawson. (London: Aslib, 1985); pp.145-163 [PDF, 105KB]

(1981) Ian M.Pigott: How does Systran translate? British Computer Society Natural Language Translation Specialist Group, Newsletter no 10, February 1981; pp.20-42. [PDF, 198KB]

(1981) Ian M. Pigott: The importance of feedback from translators in the development of high-quality machine translation. Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 61-73. [PDF, 301KB]

(1981) Stanley P. Sereda: Practical experience of machine translation. Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 119-127. [PDF, 78KB]

(1981) Barbara Snell: Has the human translator a future?  British Computer Society Natural Language Translation Specialist Group, Newsletter no 10, February 1981; pp.4-19. [PDF, 351KB]

(1981) Raw SYSTRAN output, (E-F, F-E, E-I), post-edited by Commission translators, autumn 1981; 7pp. [PDF, 227KB]

(1980) Peter J. Arthern: Machine translation – a translator’s viewpoint.  Lebende Sprachen, vol.25, no.1, pp.9-14. [PDF, 65KB]

(1980) Ch.Boitet & N.Nedobejkine: Russian-French at GETA: outline of the method and detailed example. Coling 80: proceedings of the 8th International Conference on Computational Linguistics, Sept.30 – Oct.4, 1980, Tokyo; pp. 437-446. [PDF, 650KB]

(1978) Peter J. Arthern: Machine translation and computerized terminology systems: a translator’s viewpoint. Translating and the Computer: proceedings of a seminar, London, 14th November, 1978; ed. Barbara M.Snell (Amsterdam/New York/Oxford: North-Holland Publishing Company, 1979); pp.77-108. [PDF, 253KB]

(1978) F. Knowles: Error analysis of Systran output – a suggested criterion for the ‘internal’ evaluation of translation quality and a possible corrective for system design. Translating and the Computer: proceedings of a seminar, London, 14th November, 1978; ed. Barbara M.Snell (Amsterdam/New York/Oxford: North-Holland Publishing Company, 1979); pp. 109-133. [PDF, 222KB]

(1978) Shiu-Chang Loh, Hing-Sum Hung, & Luan Kong: A dual language translator. Advances in computer-aided literary and linguistic research: Proceedings of the Fifth International Symposium on Computers in Literary and Linguistic Research, University of Aston, Birmingham, 3-7 April 1978 (AMLC 1979); pp.107-115. [PDF,

 (1978) Alastair McKinnon & Per Bondesen: Computer translation: a different approach. Revue de l’Organisation Internationale pour l’Etude des Langues Ancienne par Ordinateur (Liège), 1978 (2); 37-46. [PDF, 51KB]

(1978) Margaret Masterman: The essential skills to be acquired for machine translation. Translating and the Computer: proceedings of a seminar, London, 14th November, 1978; ed. Barbara M.Snell (Amsterdam/New York/Oxford: North-Holland Publishing Company, 1979); pp.159-180. [PDF, 347KB]

(1975) William N.Locke: Machine translation. In: Allen Kent, Harold Lancour, Jay E.Daisy (eds) Encyclopedia of Library and Information Science, vol.16. New York: Marcel Dekker Inc., 1975; pp.414-444. [PDF, 273KB]

(1974) A. Hood Roberts & Michael Zarechnak: Mechanical translation. Current Trends in Linguistics, vol.12 (The Hague: Mouton); pp. 2825-2870. [PDF, 637KB]

(1973) T.D.Crawford: A review of the Cardiff machine translation project. Computational and mathematical linguistics: proceedings of the International Conference on Computational Linguistics, Pisa, 27/VIII – 1/IX 1973, editors: A.Zampolli, N.Calzolari (Firenze: Olschki, 1980) (Biblioteca dell’Archivum Romanicum, serie II: Linguistica, 37); vol.2, pp.761-771. [PDF, 214KB]

(1970) Sergei Perschke: Machine translation: a necessary component of an international information system [with discussion]. Handling of Nuclear Information: proceedings of the symposium on the handling of nuclear information organized by the International Atomic Energy Agency, and held in Vienna, 16-20 February 1970. Vienna: IAEA, 1970; pp. 291-304. [PDF, 76KB]

Product reviews

(1988) Claude Bédard: How to demo a machine translator: you trust your mother but you cut the cards. Language Technology 7 May-June 1988; pp.26-27. [PDF, 1717KB]

(1988) Mark Olsen: INK text tools: bilingual glossary management for commercial and academic translators. Language Technology 6 March-April 1988; pp.22-25. [PDF, 2125KB]

Quality control (see also Interactive methods)

(1981) Dale A. Bostad: Quality control procedures in modification of the Air Force Russian-English MT system. Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 129-133. [PDF, 27KB]

(1981) Francesco Rossi: The impact of posteditors’ feedback on the quality of MT. Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 113-117. [PDF, 27KB]

Translatability

(1984) Anne-Marie Laurian: Machine translation: what type of post-editing on what type of documents for what type of users. Coling84: 10th International Conference on Computational Linguistics & 22nd Annual Meeting of the Association for Computational Linguistics, 2-6 July 1984, Stanford University, California. Proceedings; pp.236-238. [PDF, 216KB]

(1983) Merle D.Tenney: Machine translation, machine-aided translation, and machine-impeded translation.  Tools for the Trade: Translating and the Computer 5. Proceedings of a conference… 10-11 November 1983, London Press Centre, ed. Veronica Lawson. (London: Aslib, 1985); pp.105-113 [PDF, 38KB]

(1981) J.C.Sager: Types of translation and text forms in the environment of machine translation (MT). Practical experience of machine translation. Proceedings of a conference, London, 5-6 November 1981. Ed.Veronica Lawson (Amsterdam, New York, Oxford: North-Holland Publishing Company, 1982); pp, 11-19. [PDF, 47KB]

(1971) Norman M.Martin: Philosophy of language and the feasibility of MT: a position paper. In: Feasibility study on fully automatic high quality translation. New York: Rome Air Development Center, Griffiss Air Force Base, December 1971; pp.163-174. [PDF, 32KB]

Translationese

(1970) Th.R.Hofmann & Brian Harris: Pidgin translation. Meta 15 (2), June 1970; 71-87. [PDF, 159KB]

User evaluation and feedback

(1984) F.W.A.Habermann: Application of SYSTRAN for translation of nuclear technology texts at the Nuclear Center of Karlsruhe [abstract and slides]. International Conference on Methodology and Techniques of Machine Translation: processing from words to language… Cranfield, 13-15 February 1984, Natural Language Translation Specialist Group of the British Computer Society; 14pp. [PDF, 124KB]