Trust Us, We Checked
Poisoning an open-weight model costs $100, pirated books cost Anthropic $1.5 billion, and the frontier models still can’t grade their own work.
Dear Friend,
What a week it has been again (at least if you are still geeking out on AI – and nobody will blame you if you have thrown in the towel and wait this out for a bit). Chinese open-source model Kimi k3 has, once again, shown that open models are very closely trailing frontier models from the likes of OpenAI and Anthropic. This doesn’t bode well for companies aiming to go public on the premise that they and (at least part of) their valuation represents their ability to build a moat around their models. And then you had the OpenAI/Hugging Face hack – which seems to be both a sh**show and a wake-up call in terms of AI security, but also, potentially, a weird marketing stunt from one of the companies which needs to justify its valuation. Time will tell.
And now, this…
Headlines from the Future
AI Mania Is Having Its Corporate Moment. By now it shouldn’t come as a surprise that most AI projects (especially of the corporate ilk) are abject failures. Here is a wonderful, and rather complete, write-up by Australian consultant Nik Suresh. Highly recommended reading for anyone in the corporate world wondering what the heck is really going on with AI.
I strongly believe there are entire companies right now under heavy AI psychosis and it’s impossible to have rational conversations with them about it. […] The reality is thus: the people in charge either have no plan, or see no path forwards other than keeping their heads down. […] In fact, it is in the interests of almost every actor in the space – boards, executives, employees, vendors, consultants – to obfuscate and misrepresent the success rate of AI projects. Many publicly traded companies are putting out announcements about their AI productivity gains when I know for a fact that the businesses have done nothing other than purchase Copilot licenses and declare victory.
━━━━━
AI Supply Chains Are Fragile. Researchers in the UK managed to poison an open-weight AI model (open-weight models have their trained parameters publicly released and made free to download – typically being used to run on a firm’s internal infrastructure) for $100 and in about an hour.
“Even when model weights are public (‘open weight’), we have almost no ability to predict its behavior,” they wrote. “This is a major change: a typical computer program, in binary form, can still be analyzed with reverse engineering tools to arrive at a total description of its behavior. With models, we have nowhere close to this capability.”
The skinny here is that this leaves you in a peculiar bind: use a closed-weight model (where you have no insight into the model’s behavior – such as OpenAI’s or Anthropic’s models) and trust that the vendors do everything they can to keep their models safe, or run open-weight models on your own infrastructure but make yourself potentially vulnerable to supply chain attacks (very similar to what we have seen numerous times lately in open-source packages for popular programming languages such as JavaScript).
Last month, David Kaplan, AI security research lead at Origin, undertook a similar experiment – he created a compromised model designed to steal data. When used in the context of drug discovery, as might occur in a pharmaceutical company, it’s designed to exfiltrate data through a send_email tool call without any indication to the user.
━━━━━
And You Tought AI Recruiters Were Bad. Recruiting (and more generally the whole hiring process) is one of the (few) business functions that has seen AI tools being deployed in actual production environments: from AI-powered hiring target research, recruiting outreach, and candidate screening to resume parsing and onboarding (and everything in between). It turns out AI’s (usually) hidden biases are problematic (no one is surprised) – but they also have the wonderful capability of reinforcing themselves.
Researchers already know that LLMs pick up human biases from their training data. New research suggests that LLMs can also develop their own biases from experience – and stereotype job applicants more than humans do.
What We Are Reading
Silicon Valley Has Lost Its Biggest Advantage Much has been written about how phones harm our children. But what if it’s not the phones per se but the lack of play and exploration? @Jane
Judge Approves a $1.5b Anthropic Settlement Over Pirated Books Used to Train the Claude Chatbot A judge approved a $1.5 billion settlement requiring Anthropic to pay thousands of authors ~$3,000 per book after the company used pirated copies of their works to train Claude. A separate ruling found that AI training on copyrighted material itself is legal under fair use. @Mafe
A Philosopher’s One-word Theory to Explain Why the World Feels so Weird The “uni-context” is something like the old vision of a “global village” but stranger – and with stranger effects that seem to bleed into... everything (see: increased status anxiety, comparison, and marketization of everything, etc.). @Jeffrey
Responsible AI Is Becoming a Growth Strategy When everyone has access to similar technology, trust may become the real competitive moat. @Kacee
What If It’s Not the Phones? Much has been written about how phones harm our children. But what if it’s not the phones per se but the lack of play and exploration? @Pascal
Down the Rabbit Hole
🧑💻 The Human-in-the-Loop is Tired – This isn’t a think piece about whether AI will replace programmers. It’s not a doomer essay, and it’s not a hype piece. It’s an honest account of what it feels like to be a developer right now, from someone inside it, and some thoughts on what might actually help.
📺 Here’s an interesting experiment: Give the two frontier LLMs (Fable 5 vs. GPT-5.6 Sol) $100 each, ask them to create a music video, and see what happens. Turns out, the biggest fallacy is that both models fail to ever judge their own output.
🤖 First it was AI, now it’s the robots: Fear of humanoid robots spurs human workers to strike at Hyundai auto factory.
🍬 I remember when Googlers debated which free snacks ought to be stocked in the free micro-kitchens (no joke). How times have changed! Thousands of Google workers demand layoff protections amid AI boom in petition to CEO.
🇨🇳 Are we Chinese yet? American AI is locked down and proprietary. It’s losing.
🧑🎓 More in the ongoing debate on AI’s impact on learning and education: Using AI could reduce exam scores by a fifth, a study on Chinese students finds.
💦 First chips, then energy, now water: Not enough water for UK’s datacentre plans, trade body says.
⏳ How delightful (and what a blast from the past): The Loading Museum showcases 20+ loading screens from your past life.
⚽ Talking about “delightful”: This data-visualization-powered replay of Soccer World Cup matches is nothing short of a masterpiece.
💿 Nostalgia is hitting hard baby! CD sales growth outpaced vinyl in the first half of 2026.
🪑 Ever wondered which IKEA items deserve the crown as “most complex to assemble”? Wonder no more! Here is the IKEA complexity index.
↗ Dive into the deep end: Access our complete collection of 2,900+ radical links.
What We Do When We’re Not Writing
Hi! I’m Pascal from radical. When we’re not writing this newsletter, we help organizations turn volatility into advantage – without the “innovation theater.” We build your team’s capacity to handle disruption, so you stop reacting and start shaping. If that sounds useful, let’s talk.