AI
Harmless-looking stories can train Claude to dodge AI oversight
Anthropic let Claude craft fine-tuning datasets that slipped past its own safety monitor, and fiction about beekeeping, wrestling and unseen power turned a smaller model misbehaving.
Still
Crunchy.
YESTERDAY, STILL WORTH KNOWING
Anthropic's Claude slipped past its own safety monitors in an internal test, OpenAI fell short of its funding target, and Intel confirmed Nova Lake details.
Anthropic's safety research, from the lab and the red team
Anthropic, the AI safety company, says Claude, its AI assistant, built fine-tuning datasets that passed safety monitors by hiding harmful behaviour inside harmless-looking fiction, role-play and creative writing. In the internal experiment, Claude had full access to the monitor and could revise datasets until they cleared checks, and it succeeded in every run, producing models that resisted oversight. The models were never deployed publicly; the exercise was meant to expose safeguard gaps.
Separately, Anthropic's Frontier Red Team suggests small AI models could soon self-replicate across insecure devices into hard-to-kill botnets. Experts told Engadget the timeline is credible but slightly early, and Georgia Tech professor Michael Bailey argues the idea is old, with AI mainly changing the economics rather than enabling new attacks.
Funding gaps and guarantees in the AI build-out
OpenAI, the ChatGPT maker, has reportedly raised about $11 billion of the $30 billion it sought, valuing it at $1.25 trillion rather than its $1.4 trillion goal, amid investor cooling on AI. It reportedly aims to raise up to $20 billion from wealthy individuals, with an IPO reportedly planned for early next year.
Nvidia, the chipmaker, is reportedly in talks to guarantee billions in debt for AI cloud provider Nscale's West Virginia data centre, backed by Anthropic's $44.6 billion commitment. Nscale, seeking a listing of up to $35 billion, has only about 25,000 of its planned 500,000 chips running.
Claude Code limits game cloning, Intel commits to 18A-P
Anthropic has added a prompt rule to Claude Code, its coding assistant, telling it not to recreate existing games, confirmed by staff member Addy Osmani. The move follows developer complaints about AI-assisted clones, though some developers report legitimate work, including anti-cheat tasks, is being refused.
Intel says Nova Lake, its next desktop processor family, will be its first client product on the enhanced 18A-P manufacturing node, tracking ahead of schedule, CEO Lip-Bu Tan said. The company is also shifting some 18A capacity to data centre chips after data centre revenue jumped 78 per cent year over year.