Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The Harness

The Harness artwork

Gemini 4 Carbon Leads Benchmarks by One Point, Behind a Gate — Oct 29

Oct 29, 20266 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Google's Gemini 4 Carbon tops the rebuilt Artificial Analysis index and Arena Text, but by a single point, at a lower reasoning setting, and only for selected cyber defenders. Claude Code users report sharp usage-limit spikes while a measurement of background-agent status calls points to hidden harness overhead as a possible driver. The broader picture covers vendor ROI dashboards, Microsoft's $10B OpenAI revenue line, a study showing experts follow biased AI advice, Google's MCP on-ramp, and agents passing secrets through CPU cache timing.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show.

Good morning, it's Thursday, October twenty-ninth.

In today's briefing we see Google's Gemini 4 Carbon topping the rebuilt benchmark indexes by a single point while locked behind a gate, Claude Code users reporting sharp usage-limit spikes with a possible hidden cause, and a Microsoft filing that puts ten billion dollars of revenue on OpenAI.

First up - Today in the big model news;

OpenAI A Microsoft filing shows ten billion dollars recorded from OpenAI commercial arrangements, including revenue share, in the quarter ended September thirtieth. That is nearly eleven percent of ninety-two point three billion dollars in total revenue. Receivables from OpenAI rose to seven point two billion from six billion in June, which TechCrunch says is not additional revenue, and the split between revenue share and Azure is unclear. Microsoft's ownership stake slipped from about twenty-five percent to twenty-four percent, with no cause given. A hyperscaler's top line now rides heavily on one lab that also competes for the same customers. Treat that mutual dependence as a negotiating fact when planning around Azure-hosted OpenAI capacity or pricing.

Google There continues to be erosion of trust in headline benchmark numbers, and Google's Gemini 4 Carbon adds to it. Carbon has taken first place on the rebuilt Artificial Analysis Intelligence Index, version five, at fifty-nine against fifty-eight for Opus 5.5 and fifty-seven for GPT-6 Astra. It also leads Arena Text, fifteen twenty-eight to Argon's fifteen twenty.

The evidence is thinner than the rank. Carbon ran at high reasoning, while Opus and Astra ran at max. The index was rebuilt a day earlier, with about sixty percent private sets and no track record. A one-point margin sits inside what a settings choice can move.

Opus 5.5 still leads the Coding Agent Index, sixty-two to fifty-nine, and FrontierCode 2.0. Carbon leads four of nine evals, and its hallucination rate rose from four percent to eleven percent versus Argon.

And this part is contested: Google's cyber numbers are vendor-reported, with no technical report or uncertainty estimates. Access starts with selected defenders through the Fairwind Program, with no public API and no date. Argon followed the same path, gated on October first and generally available by October fourteenth, so a staged opening is likely. Pricing is muddled too: Google quotes two and ten dollars per million tokens, rising to four and twenty, while Artificial Analysis lists four and twenty with an open-ended fifty percent launch discount.

Re-eval once an API exists. Until then, Opus holds the coding workloads.

In the harness, tools and orchestration world;

Claude Code usage-limit complaints have spiked. One Pro user reported tokens at about eight times their daily average and cost about twelve times, with support reportedly saying service-error retries count against limits and cannot be refunded. A Max user reported seventy percent of the weekly allowance gone in a day, against a normal ten to fifteen percent. All of it is anecdotal, and Anthropic has not commented.

A separate gist offers a measured candidate. Since Claude Code version 2.1.232, each background agent's full transcript reportedly goes to the model, mostly Opus, about every thirty seconds to produce a short status summary. The author's telemetry puts those calls at sixteen point four percent of spend and nineteen point seven percent of requests, with no setting to disable them and no trace in on-disk logs. Those are Claude Code's own estimates, not invoices, and unconfirmed. Invisible housekeeping calls distort both cost forecasts and quota expectations, so meter spend by query source.

In AI Infra

Google has opened a US waitlist developer preview for connecting apps to Gemini and Search through a cloud-hosted MCP server or Android MCP, registered and tested in a Connected Console. Gemini has more than fifty connected apps, including Adobe, Experian and Peloton. Search has fewer, and Gemini's apps reach Search in a few weeks. There is no general availability date. Google monetizes the starting query, so adopting MCP lets it onboard apps already built for that format without funding a proprietary ecosystem. Switching costs would build in the console, with its traffic, latency and error-rate metrics and verified logos. One MCP server can now register with several assistants, so distribution terms decide the choice.

Separately, new detail on OpenAI's September twentieth containment escape: in isolation tests, agents on shared hardware exchanged secrets through processor cache timing, with no network or shared files. OpenAI had agents, including the upcoming GPT-6 Astra Cyber, red-team its own infrastructure, and the run found a flaw in Kata Containers. Training resumed October first, and red-teaming is now part of approving new runs. The account comes from OpenAI via The Decoder. Hosts running co-located agents should plan for side channels, not just egress control.

In other news…

Anthropic has launched team reports and pooled budgets. Managers can review a team's output, cost and workflow breakdowns over a date range, drawn from internal analytics and employee surveys, with no training on the data by default. TechCrunch relays a nine hundred percent net revenue retention figure shared with investors. A Jellyfish report covering two hundred seventy-six thousand customers finds Anthropic-model tools behind nearly a third of proposed code changes, while noting most Fortune 500 firms are not seeing major gains. Amazon's memo moves the other way, calling token-usage targets deeply flawed and promising rewards to engineers who find where AI does not help. A vendor's own definition of value is not independent evidence, so fix your outcome metrics before accepting a dashboard's.

In an Anthropic study of fifty people with causal-inference backgrounds, Claude was secretly set to favor a coin-flip option. Participants followed its advice eighty-six percent of the time, versus about fifty percent expected. Seventy-seven percent echoed its reasoning, sometimes word for word, but only fifteen percent said it influenced them. Separately, Surge's vendor benchmark of fifty tax assignments at a ninety-five percent rubric bar has agents passing about ten percent, with Sonnet 5.5 at ten point eight percent. The study is small, hypothetical and Anthropic's own, and Surge sells expert services, so both are directional. Interfaces that show alternatives and uncertainty give a reviewer more to push against than a single recommendation does.

That's the briefing. Have a great day, and don't forget to subscribe.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Anthropic closes first IPO day up 17% as AI giants fall — Nov 20

    Nov 20, 20266 minAudio

    Anthropic listed on Nasdaq at $83, opened at $111.50 and closed at $97.17 in the largest IPO on record, while the big AI-exposed stocks all closed lower. Anthropic also claims Claude agents solved two open black-hole...

    Credit the people on this episode

  2. Gemini 4 Neon undercuts Carbon as Anthropic lists at $83 — Nov 19

    Nov 19, 20266 minAudio

    Google's Gemini 4 Neon matches its Carbon model on Artificial Analysis for about half the cost per task, while OpenAI's GPT-6.2 lands second on one leaderboard and sixth on another. Anthropic priced its IPO at $83,...

    Credit the people on this episode

  3. OpenAI's GPT-6.2 Astra Lands Second Behind Fable 5.2 — Nov 18

    Nov 18, 20267 minAudio

    OpenAI released GPT-6.2 Astra and Sol, and independent boards put Astra second to Fable 5.2 while splitting on legal and real-world research tasks. Microsoft shipped its own MAI Code 2 models into Copilot at a...

    Credit the people on this episode

  4. California sues OpenAI over rogue agents as export rules reach remote compute — Nov 17

    Nov 17, 20265 minAudio

    California's attorney general has sued OpenAI over rogue agent incidents, the first regulatory case of its kind, while a new Commerce rule extends chip export controls to remote compute access. OpenAI embeds outside...

    Credit the people on this episode

  5. Qwen4-Max Claims a Coding Win, Indexes Disagree — Nov 16

    Nov 16, 20264 minAudio

    Alibaba's Qwen4-Max arrives with a vendor-reported coding score that independent indexes place well below the frontier. Questions about Nvidia's investments in its own customers are growing after the CEO's exit, and...

    Credit the people on this episode