The AI Litmus Test

Is Your Organization Ready for the Future?

Here’s a reality check for you: When ChatGPT Pro dropped six weeks ago, when DeepSeek’s R1 model launched this weekend, or when Google made the latest version of their Gemini 1.0 Ultra model available, did your organization have a dedicated team ready to put these models through their paces? Did you have benchmarks ready? A framework to evaluate their impact on your business?

No? You’re not alone. And that might prove to be a massive problem.

We’re witnessing quite possibly an unprecedented moment in the history of technology. Matt Webb recently pointed out that the development acceleration we’re seeing with AI – turning 4-day tasks into 20-minute sprints – is equivalent to a decade of Moore’s Law improvements. It is practically happening overnight.

(*) Granted, this is true for some tasks, not all. But still…

It’s not about AI being magical, sentient, super intelligent, or whatever Sam Altman’s latest blurb is. It’s simpler and more profound: Imagine if the iPhone had dropped in 1997 instead of 2007. That’s the scale of disruption we’re dealing with today.

But here’s where most organizations are getting it wrong: They’re treating AI advancement like a spectator sport. Reading the news. Watching demos. Maybe playing around with ChatGPT in their spare time. I don’t think that this is going to cut it.

As a regular reader of the radical Briefing, you know how much we like Wharton professor Ethan Mollick. In one of his recent dispatches, he made a similar point: You need your own benchmarks. Your own testing frameworks. Your own understanding of how these models will impact your specific business context.

And while you’re waiting for the perfect moment to take AI seriously, your competitors might already be building their muscle memory. They’re developing institutional knowledge. They’re creating frameworks and benchmarks that will give them years of advantage.

Here’s a set of concrete ideas for what you can do today:

1. Assign dedicated people (including non-technical folks) to test every significant new AI model.

2. Develop internal benchmarks specific to your business cases.

3. Create a regular cadence of updating your AI strategy based on new capabilities.

4. Build cross-functional teams that understand both the technology and its business implications.

With the pace of AI advancement, gut feelings and guesswork are luxuries you can’t afford. You need a systematic process that transforms the daily flood of AI developments into concrete, actionable insights.

As the (in AI-terms “old”) adage goes: Today is the worst AI will every be.

@Pascal