Machine Translation Archive

Index to evaluation measures

Publications before 1970

For other periods go to: Publications since 2010; publications 2005-2009; publications 2000-2004; publications 1990-1999; publications 1970-1989

[To return to home page click here]

Back translation

(1955) Balth van der Pol: An iterative translation test [summary]. In: Information theory: papers read at a symposium on ‘Information theory’ held at the Royal Institution, London, September 12th to 16th, 1955; edited by Colin Cherry, London: Butterworths Scientific Publications, 1956; pp.397-398. [PDF, 23KB]

Error detection and correction

(1969) A.J.Szanser: Automatic error-correction in natural languages. Coling 1969: International Conference on Computational Linguistics, 1-4 September 1969, Sångy-Säby, Sweden; Preprint no. 7; 9pp. [PDF, 350KB]

(1959) D.G.Owen: [contribution to] Discussion [of] V.E.Giuliano and A.G.Oettinger: Research on automatic translation at the Harvard Computation Laboratory. Information processing: International Conference on Information Processing, Unesco, Paris, 15-20 June 1959 (Paris: Unesco, 1960); p. 183. [PDF]

(1954) V.H.Yngve: Language as an error correcting code. Quarterly Progress Report of the Research Laboratory of Electronics, M.I.T., April, 1954; pp.73-74. [PDF, 21KB]

Evaluation measures and metrics [see also Test corpora]

(1966) John B. Carroll: An experiment in evaluating the quality of translations. Mechanical Translation and Computational Linguistics, vol. 9, nos.3 and 4, September and December 1966; pp.55-66 [PDF, 264KB]

(1966) John B.Carroll: An experiment in evaluating the quality of translations. Appendix 10 in: Language and machines: computers in translation and linguistics. A report by the Automatic Language Processing Advisory Committee, Division of Behavioral Sciences, National Academy of Sciences, National Research Council. (Washington, D.C.: National Academy of Sciences – National Research Council, 1966); pp.67-75. [PDF, 144KB]

(1965) Sheila M. Pfafflin: Evaluation of machine translation by reading comprehension tests and subjective judgments.  Mechanical Translation and Computational Linguistics, vol. 8, no.2, February 1965; pp.2-8 [PDF, 179KB]

 (1960) Session 4: Methodology [contributions by A.F.R.Brown, L.Dostert, H.P.Edmundson, V.Giuliano, R.M.Hayes, D.G.Hays, H.H.Josselson, G.W.King, S.M.Lamb, A.W.Lukjanow, A.G.Oettinger, E.Reifler, R.See, D.R.Swanson, S.N.Werbow, V.H.Yngve, and M.Zarechnak]. Proceedings of the National Symposium on Machine Translation, held at the University of California, Los Angeles, February 2-5, 1960; ed. H.P.Edmundson. (Englewood Cliffs, NJ: Prentice-Hall, 1961); pp.195-218. [PDF, 108KB]

(1960) Session 4: Methodology. Questions and discussion [by G.W.Brown, L.Dostert, H.P.Edmundson, P.L.Garvin, D.G.Hays, H.H.Josselson, G.W.King, S.M.Lamb, A.G.Oettinger, E.Reifler, I.Rhodes, R.See, M.E.Sherry, D.R.Swanson, A.Wallace]. Proceedings of the National Symposium on Machine Translation, held at the University of California, Los Angeles, February 2-5, 1960; ed. H.P.Edmundson. (Englewood Cliffs, NJ: Prentice-Hall, 1961); pp.219-224. [PDF, 49KB]

(1956) George A.Miller & J.G.Beebe-Center: Some psychological methods for evaluating the quality of translations. Mechanical Translation 3 (3), December 1956; pp. 73-80 [PDF, 151KB]

Evaluations of systems [see also User experiences]

(1969) A.J.Szanser: Machine translation at the NPL. Physics Bulletin 20 (1969), 92-94. [PDF, 49KB]

(1967) Bozena Henisz-Dostert: Experimental machine translation. In: William M.Austin (ed) Papers in linguistics in honor of Léon Dostert (The Hague: Paris, 1967); pp.57-91. [PDF, 500KB]

(1967) J.McDaniel, W.L.Price, A.J.M.Szanser, & D.M.Yates: An evaluation of the usefulness of machine translations produced at the National Physical Laboratory, Teddington, with a summary of the translation methods. Coling 1967: Conference Internationale sur la Traitement Automatique des Langues, Grenoble, France, August 1967; 18pp. [PDF, 752KB]

(1967) David B. Orr & Victor H. Small: Comprehensibility of machine-aided translations of Russian scientific documents. Mechanical Translation and Computational Linguistics, vol. 10, nos.1 and 2, March and June 1967; pp.1-10 [PDF, 191KB]

(1967) A.J.Szanser: Machine translation—the evaluation of an experiment. The Incorporated Linguist 6 (4), October 1967; pp.90-95. [PDF, 131KB]

(1966) John B. Carroll: An experiment in evaluating the quality of translations. Mechanical Translation and Computational Linguistics, vol. 9, nos.3 and 4, September and December 1966; pp.55-66 [PDF, 264KB]

Examples of MT output

(1968) Sergei Perschke: Machine translation—the second phase of development. Endeavour, vol.27; pp.97-100. [PDF, 76KB]

(1968) L.W.Tosh: Initial results of syntactic translation at the Linguistics Research Center. Linguistics 42 (1968); pp.96-116. [note]

(1967) Paul L.Garvin: The Georgetown-IBM experiment of 1954: an evaluation in retrospect. In: William M.Austin (ed) Papers in linguistics in honor of Léon Dostert (The Hague: Paris), pp.46-56. [PDF, 200KB]

(1967) Bozena Henisz-Dostert: Experimental machine translation. In: William M.Austin (ed) Papers in linguistics in honor of Léon Dostert (The Hague: Paris), pp.57-91. [PDF, 500KB]

(1967) J.McDaniel, A.M.Day, W.L.Price, A.J.Szanser, S.Whelan, & D.M.Yates: Machine translation at the National Physical Laboratory, Teddington, Middlesex, England. In: A.D.Booth (ed.) Machine translation. (Amsterdam: North-Holland Publishing Company, 1967); pp.229-266.

(1967) Margaret Masterman: Mechanical pidgin translation: an estimate of the research value of ‘word-for-word’ translation into a pidgin language, rather than into the full normal form of an output language.  In: A.D.Booth (ed.) Machine translation. (Amsterdam: North-Holland Publishing Company, 1967); pp.1952-27.

(1967) Erwin Reifler: Chinese-English machine translation: its lexicographic and linguistic problems [with extensive bibliography]. In: A.D.Booth (ed.) Machine translation. (Amsterdam: North-Holland Publishing Company, 1967); pp.317-428.

(1967) A.J.Szanser: Machine translation—the evaluation of an experiment. The Incorporated Linguist 6 (4), October 1967; pp.90-95. [PDF, 131KB]

(1965) W.P.Lehmann: An experiment in machine translation. The Graduate Journal 7 (1); pp.111-131. [PDF, 103KB]

(1963) Leon E.Dostert & R.R.Macdonald: Georgetown University Machine Translation research project. General report 1952-1963. Occasional Papers on Machine Translation no.30, June 1963, prepared by R.R.Macdonald; xxi, 236pp. [PDF, 1087KB]

(1963) Léon Dostert: Machine translation and automatic language data processing. In: Paul W.Howerton and David C.Weeks: Vistas in information handling, vol.I: The augmentation of man’s intellect by machine (Washington, D.C.: Spartan Books, 1963); pp.92-110. [PDF, 185KB]

(1960) Neil Macdonald: The photoscopic language translator.  Computers and Automation, August 1960; pp.6-8. [PDF, 35KB]

 (1960) Margaret Masterman & Martin Kay: Mechanical pidgin translation: an estimate of the research value of “word-for-word” translation into a pidgin language, rather than into the full normal form of an output language. (ML133). Cambridge: CLRU, July 13th 1960; [incomplete copy] [PDF, 416KB]

(1960) D.Yu.Panov : Automatic translation. Translated from the Russian by R.Kisch, edited by A.J.Mitchell. New York, London, Oxford, Paris: Pergamon Press, 1960. iii, 73pp. [PDF, 665KB]

(1960) M.Zarechnak & A.F.R.Brown: Current research at Georgetown University. Proceedings of the National Symposium on Machine Translation, held at the University of California, Los Angeles, February 2-5, 1960; ed. H.P.Edmundson. (Englewood Cliffs, NJ: Prentice-Hall, 1961); pp.63-87. [PDF, 161KB]

(1960) [Automatic translation – of Russian article in Izvestia, August 14 1960] [Handout from IBM.] 6pp. [PDF, 58KB]

(1960) Appendix: The simulated machine translations. [In] Linguistic and engineering studies in the automatic translation of scientific Russian into English. Seattle: University of Washington Press, 1960; pp.183-199 [omitting further examples, pp,200-348. [PDF, 206KB]

 (1959) Margaret Masterman: A possible counter-example to F.A.H.Q.M.T.? In: Essays on and in machine translation [ML91, appendix III] (Cambridge Language Research Unit), 1959; 5 pp. [PDF, 16KB]

 (1959) Lew R.Micklesen & Robert E.Wall: Logical programming research in the University of Washington machine translation project.  In: Allen Kent (ed.) Information retrieval and machine translation. Based on the International Conference for Standards on a Common Language for Machine Searching and Translation, Cleveland, Ohio, September 6-12, 1959 (New York/London: Interscience Publishers, 1961); part 2, pp.853-866. [PDF, 140KB]

(1957) I.K.Belskaja: Machine translation of languages. Research vol.10, part 10, October 1957; pp.383-389. [PDF, 193KB]

(1956) I.S.Mukhin: An experiment on the machine translation of languages carried out on the BESM. Session on the computer in a non-arithmetic role, at the Convention on Digital-Computer Techniques, 12th April 1956. Proceedings of the Institute of Electrical Engineers, vol.103, part B: Supplement no.3 (1956); pp.463-472. [PDF, 176KB]

 (1955) Kenneth E.Harper: A preliminary study of Russian. In: William N.Locke & A.Donald Booth (eds.): Machine translation of languages, fourteen essays. The Technology Press of the Massachusetts Institute of Technology; New York: John Wiley, 1955; pp. 66-85.

(1955) Anthony G.Oettinger: The design of an automatic Russian-English technical dictionary. In: William N.Locke & A.Donald Booth (eds.): Machine translation of languages, fourteen essays. The Technology Press of the Massachusetts Institute of Technology; New York: John Wiley, 1955; pp. 47-65.

(1955) R.H.Richens & A.D.Booth: Some methods of mechanized translation. In: William N.Locke & A.Donald Booth (eds.): Machine translation of languages, fourteen essays. The Technology Press of the Massachusetts Institute of Technology; New York: John Wiley, 1955; pp. 24-46.

(1952) R.H.Richens & A.D.Booth: Some methods of mechanized translation. Presented at the Conference on Mechanical Translation, June 1952, Massachusetts Institute of Technology; 15pp. [PDF, 178KB]

Quality improvement techniques (see also Interactive methods)

(1961) Vincent E. Giuliano: A formula finder for the automatic synthesis of translation algorithms.  Mechanical Translation, vol. 6, November 1961; pp.11-24. [PDF, 343KB]

Reading and comprehension

(1967) David B. Orr & Victor H. Small: Comprehensibility of machine-aided translations of Russian scientific documents. Mechanical Translation and Computational Linguistics, vol. 10, nos.1 and 2, March and June 1967; pp.1-10 [PDF, 191KB]

(1965) Sheila M. Pfafflin: Evaluation of machine translation by reading comprehension tests and subjective judgments.  Mechanical Translation and Computational Linguistics, vol. 8, no.2, February 1965; pp.2-8 [PDF, 179KB]

Translatability

(1967) John M. Dolan: A note on Quine’s theory of radical translation. Mechanical Translation and Computational Linguistics, vol. 10, nos.1 and 2, March and June 1967; pp.11-17 [PDF, 220KB]

(1959) F.H.Ellis: Human translateability test. In: Essays on and in machine translation [ML91, appendix I] (Cambridge Language Research Unit), 1959;  2 pp. [PDF, 10KB]

(1959) Margaret Masterman: Fictitious sentences in language. In: Essays on and in machine translation [ML91] (Cambridge Language Research Unit), 25th May 1959;  33pp. [PDF, 97KB]

Translationese

(1960) Margaret Masterman & Martin Kay: Mechanical pidgin translation: an estimate of the research value of “word-for-word” translation into a pidgin language, rather than into the full normal form of an output language. (ML133). Cambridge: CLRU, July 13th 1960; [incomplete copy] [PDF, 416KB]