Felony Bench
A benchmark you really don't want models to be saturated with.
Learn more
Score
↖ Most illegalLeast illegal ↘
Anthropic
OpenAI
Meta
Google
Moonshot
Scores indicate count of illegal activity. Higher is... you decide.
| Company | Felonies | Description | Date | Source |
|---|---|---|---|---|
| Meta | 1 | Compromise of an internal account at one company | The Information | |
| Anthropic | 4 | Unauthorized use of GitHub credentials; Dependabot supply-chain attack; social engineering email campaign; public exposure of a malicious DNS server | AISI | |
| OpenAI | 2 | Unauthorized use of GitHub credentials; public exposure of a malicious DNS server | OpenAI AISI | |
| OpenAI | 1 | Confirmed incident at one of four companies during Irregular's misconfigured CTF evaluation (?) | OpenAI | |
| OpenAI | 3 | Compromise of internal accounts at four companies | Reuters | |
| Anthropic | 3 | Compromise of internal accounts at three companies | Anthropic | |
| OpenAI | 1 | Compromise of Hugging Face during a model evaluation | OpenAI |