2 October 2026
US military nearly acted on AI-generated false intelligence report
First reported
Transformer ran this on .
- A September incident showed US military personnel almost boarded a Chinese ship based on false information produced by an AI system.
- Human review of AI recommendations does not automatically catch errors when decision-makers trust the system too much, a problem called automation bias.
- Military AI systems in Gaza and Iran contributed to civilian casualties, raising concerns about similar mistakes in high-stakes US-China situations.
How it was covered
TransformerShakeel Hashim
A September incident where US military nearly boarded a Chinese ship based on an AI-generated false intelligence report demonstrates that keeping humans in the loop does not solve AI risks in military decision-making. The newsletter argues the real danger comes from humans over-relying on AI systems due to automation bias, as evidenced by Israeli targeting system Lavender and Palantir's Maven system in Gaza and Iran, where AI errors contributed to civilian casualties and could trigger nuclear conflict if similar mistakes occur in US-China scenarios.