GCD AI RADAR
incidents/AIR-2025-0052

ChatGPT Package Hallucination Supply Chain Risk

AI-enabled attack1 Research / proof of conceptSupply chainOther

Researchers highlighted that LLMs can hallucinate non-existent package names, which attackers could register to distribute malicious code. Developers following the generated advice risk downloading compromised packages.

Details

type
AI-enabled attack
Attackers used AI as a meaningful part of a cyberattack against someone else.
categories
  • Supply chainprimary
    Malicious or backdoored models, poisoned packages, unsafe deserialization and other compromised AI dependencies.
  • Other
    In scope, but none of the categories above fits. Reviewed periodically for new categories.
owasp llm top 10
no entry for these categories
mitre atlas
no technique for these categories
Both rows are derived from the type and categories (OWASP 2025 edition, ATLAS release 2026.09), not classified per record. How
severity
1 · Research / proof of concept
Demonstrated by researchers; no known real-world exploitation.
targets
OpenAI, Framework / tooling
products
ChatGPT
incident date
Jun 2024 (approx.)
first reported
14 Mar 2025
origin
MITRE ATLAS
Derived from MITRE ATLAS (Apache 2.0, © MITRE).
classifier confidence
0.90 · published at or above 0.80; see the methodology
tags
hallucination · supply-chain · package-squatting
record
created 2026-10-18 06:55Z · updated 2026-10-18 06:55Z · json

Sources · 5

  1. 2025-03-14
    ChatGPT Package Hallucination
    MITRE ATLAS · atlas.mitre.org/studies/AML.CS0022
  2. 2025-03-14
    Vulcan18's "Can you trust ChatGPT's package recommendations?"
    vulcan.io · vulcan.io/blog/ai-hallucinations-package-risk
  3. 2025-03-14
    Lasso Security Research: Diving into AI Package Hallucinations
    lasso.security · lasso.security/blog/ai-package-hallucinations
  4. 2025-03-14
    AIID Incident 731: Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers
    AI Incident Database · incidentdatabase.ai/cite/731
  5. 2025-03-14
    Slopsquatting: When AI Agents Hallucinate Malicious Packages
    Trend Micro · trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatti

Titles and links belong to their publishers. The summary above is written by GCD AI RADAR in its own words; data is CC BY-SA 4.0.

Related incidents