incidents/AIR-2020-0029

Model Extraction and Adversarial Attacks on Translation APIs

Attack on AI1 Research / proof of conceptModel theft / extractionAdversarial evasion

Researchers replicated commercial machine translation services via public APIs and transferred adversarial inputs to trigger targeted errors. The demonstration proved that black-box models are vulnerable to extraction and evasion.

Details

type: Attack on AI
An AI model, agent or AI-powered product was the thing exploited, abused or leaked.
categories: Model theft / extractionprimary
Weights stolen, or model behaviour replicated through extraction or distillation attacks.
Adversarial evasion
Inputs crafted to make a model misclassify or miss detections (perturbations, evasion of AI-based defences).
owasp llm top 10: LLM10:2025 Unbounded Consumption
mitre atlas: AML.T0015 Evade AI Model
AML.T0043 Craft Adversarial Data
AML.T0048.004 External Harms: AI Intellectual Property Theft
Both rows are derived from the type and categories (OWASP 2025 edition, ATLAS release 2026.09), not classified per record. How
severity: 1 · Research / proof of concept
Demonstrated by researchers; no known real-world exploitation.
targets: Google, Microsoft
products: Google Translate, Bing Translator, Systran Translate
incident date: 30 Apr 2020
first reported: 18 Nov 2020
origin: MITRE ATLAS
Derived from MITRE ATLAS (Apache 2.0, © MITRE).
classifier confidence: 0.95 · published at or above 0.80; see the methodology
tags: model-extraction · adversarial-evasion · translation-api
record: created 2026-10-18 06:55Z · updated 2026-10-18 06:55Z · json

Sources · 4

2020-11-18
Attack on Machine Translation Services
MITRE ATLAS · atlas.mitre.org/studies/AML.CS0005
2020-11-18
Wallace, Eric, et al. "Imitation Attacks and Defenses for Black-box Machine Translation Systems" EMNLP 2020
arXiv · arxiv.org/abs/2004.15015
2020-11-18
Project Page, "Imitation Attacks and Defenses for Black-box Machine Translation Systems"
ericswallace.com · ericswallace.com/imitation
2020-11-18
Google under fire for mistranslating Chinese amid Hong Kong protests
thehill.com · thehill.com/policy/international/asia-pacific/449164-google-under-fire-for-mistr

Titles and links belong to their publishers. The summary above is written by GCD AI RADAR in its own words; data is CC BY-SA 4.0.

Related incidents

2020-10-23
Microsoft Edge AI Product Evasion Red Team Exercise
1 Researchon AIEvasion1 src
2020-10-23
Microsoft Internal Azure AI Service Red Team Exercise
1 Researchon AIEvasion1 src
2020-12-15
Botnet DGA Detection Evasion Using Domain Mutation
1 Researchon AIEvasion3 src
2020-12-15
Evasion of Deep Learning Detector for Malware C&C Traffic
1 Researchon AIEvasion2 src
2020-12-03
Bypassing Cylance AI Malware Detection via Universal String
1 Researchon AIEvasion3 src
2020-12-03
Camera Hijack and Facial Recognition Evasion for Tax Fraud
4 Real losson AIEvasion2 src