Microsoft claims MT system “achieved human parity” on test data

Source: Techcrunch
Story flagged by: Ambrose Li

“A team of Microsoft researchers announced on Wednesday they’ve created the first machine translation system that’s capable of translating news articles from Chinese to English with the same accuracy as a person. The company says it’s tested the system repeatedly on a sample of around 2,000 sentences from various online newspapers, comparing the result to a person’s translation in the process – and even hiring outside bilingual language consultants to further verify the machine’s accuracy.”

Source: Techcrunch

Comments about this article


Microsoft claims MT system “achieved human parity” on test data
Michael Marcoux
Michael Marcoux  Identity Verified
United States
Local time: 12:15
Russian to English
+ ...
Don't speak Chinese, but... Mar 21, 2018

it actually looks pretty good. Welp, time to find a new career.

 
Lincoln Hui
Lincoln Hui  Identity Verified
Hong Kong
Local time: 00:15
Member
Chinese to English
+ ...
Strengths and weaknesses Mar 21, 2018

I tested a few strings and the system does have its strengths. The key weaknesses are as follows:

- It doesn't "know" that what it produces is correct. It's capable of producing several options (in this case, two), one of which is likely to be quite accurate, but it can't tell on its own which one to use.

- It looks like it's designed to parse text written in a specific way, i.e. by trained journalists. News reports have certain style conventions and it's not hard to imagin
... See more
I tested a few strings and the system does have its strengths. The key weaknesses are as follows:

- It doesn't "know" that what it produces is correct. It's capable of producing several options (in this case, two), one of which is likely to be quite accurate, but it can't tell on its own which one to use.

- It looks like it's designed to parse text written in a specific way, i.e. by trained journalists. News reports have certain style conventions and it's not hard to imagine that a machine with access to a vast amount of data can identify and pick up relevant information. Evidence to this is that the machine performed extremely poorly when I threw a couple of interview quotes at it. They were not in a style that the machine was trained in, and the confused machine was not even able to try and fake a fluent output. I also tried several legal text excerpts, relatively simple ones (terms of service) that pose no significant difficulty for a well-educated reader without formal legal training. The machine was able to produce a more-or-less acceptable output for the more verbose sample, but the other two threw it off completely.

The second weakness can be mitigated with more data, but the first has no solution in the foreseeable future. Accumulating more human selection data can increase the confidence of the choice, but only to a certain extent. It can't deliver total confidence without a human reviewer, and at that point you have a competent translator in the loop.

Translators are cheap, translation mistakes are expensive. The translation market could transform and it could shrink, but it has a long way to go before it becomes irrelevant.
Collapse


 

Sign in to add a comment

To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Jared Tabor[Call to this topic]

You can also contact site staff by submitting a support request »
This discussion can also be accessed via the ProZ.com forum pages.


Translation news
Stay informed on what is happening in the industry, by sharing and discussing translation industry news stories.

All of ProZ.com
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search