Two security teams test prompt injection against Jev: an ad that steers an agent, and a fake policy that hides attacks
publicado (OK do Fabricio 18/10 BRT).
Two security write-ups published this week test how well Jev holds up against prompt injection. They measure different things, and neither changes our scores, which have no adversarial-robustness subcriterion.
HiddenLayer, 14 October (research post, Kenneth Yeung and Kasimir Schulz). The team built a simulated checkout page where an ad overlaps the buttons, and let an agent harness (“jev-ultrafast”) pick what to click. An injected ad could steer the click. HiddenLayer says it worked against its own Qwen3.8-27B System One model, against TypeSafe’s Jev through the API, and against gpt-6-luna through OpenAI’s Decisions API, with “a little less consistency” there. The Register (15 October) reports that Jev “succumbed less often” and that TypeSafe had not replied when it published. In a second test, on 30 toxic prompts, every prompt was flagged at baseline, and many got through after targeted perturbations (the TokenBreak technique). We do not quote the percentages that appear only in the post’s figures.
Datadog Security Labs, 15 October (article, Nick Frichette and Feras Al-Kassar). The team used Jev as an injection detector over 12,295 samples from four public datasets. With a plain noul question it reached 99.66% precision and 82.90% recall. When a counterfeit policy was prepended to the request, recall fell to 3.88% (precision 100%), and it missed every attack in two of the datasets (zachz and Yanismiraoui). With a choice question, recall fell from 85.46% to 9.69%. In a LangChain tool-risk gate with a blocking threshold of 0.5, the average is_risky score for an attack went from 0.893 to 0.290 over 40 runs (plain injection, 0.905).
How to read them. Datadog measured whether the judge flags an injection, not whether a downstream system was compromised, and HiddenLayer’s agent results come from its own harness. Both show the same pattern: text placed next to the thing being judged can move a decision model’s answer. TypeSafe’s documentation already lists adversarial or injected framing as a known weak spot (see the Jev page), and Red Hat’s benchmark of 2 October found that decision models do not reliably beat classic guardrails. If you use a decision model as a gate, test your own attacks, including a fake policy inside the input, before relying on it.