7 topics covered

Listen to today's briefing
0:00--:--

TasteVal Benchmark and Jack Clark's Self-Improvement Forecast

What happened: Anthropic co-founder Jack Clark moved his forecast for recursive self-improvement to as early as 2027, citing a new P-Zero Research benchmark, TasteVal, that measures AI 'research taste'.

Key details:

  • TasteVal tests how well models design experiments and interpret results across eight AI-development tasks; it does not test choosing research directions.
  • Models run experiments on a single Nvidia H100 with a fixed coding agent, with each run capped at 40 GPU-hours of compute or 120 hours of wall-clock time.
  • The best human baseline came from 24 experts who had recently worked at places such as OpenAI, Google DeepMind, Nvidia, and Microsoft Research.
  • The top model, Anthropic's Opus 5.5, reached a compute multiplier of 2.3 (statistical range 1.15 to 4.37), meaning expert-level scores with less than half the compute; a single run cost roughly one-thirtieth of an average expert attempt.
  • Across model generations, research taste roughly doubled every three months since December 2025, with a doubling time range of 1.7 to 5.0 months.
  • Clark previously put the odds of a self-improving system at roughly 30 percent by 2027 and 60 percent by end of 2028 in a May essay; he had previously expected 2028.

Why it matters: The benchmark targets a capability critics had named as a barrier to RSI, but the authors acknowledge results may overstate the pace, the tasks are cheap to verify (easy for labs to optimize for), and the comparison group included no top-tier researchers.

Practical takeaway: Treat the doubling-time figures as highly uncertain given the wide reported ranges, and check the benchmark's limitations before citing the 2027 timeline.

Anthropic Recommends Single Agent Over Sub-Agents

What happened: Anthropic is advising companies to consider consolidating multi-agent setups into a single agent that loads context on demand, and has published a migration kit on GitHub.

Key details:

  • In one architecture comparison, the single-agent setup ran about twice as fast as the multi-agent setup.
  • Anthropic says newer models, including Opus 4.5, Opus 5.5, and Fable 5.1, can manage complex plans in a single loop.
  • Anthropic cut the Claude Code system prompt for newer models by over 80 percent, partly because skills and tools now load only when needed.
  • Sub-agents still make sense when a single agent's context shows measurable performance degradation, when tasks need parallel separate context windows, or when tool selection struggles.
  • Anthropic notes tool selection tends to degrade beyond 30 to 50 tool definitions.
  • In one support-agent test, folding in a sub-agent reduced latency but broke a previously working process.

Why it matters: The guidance pushes against the common pattern of spawning specialized agents per team, and it is consistent with a Vals study that found agent teams faster but not higher quality.

Practical takeaway: Teams running multi-agent architectures can use Anthropic's GitHub migration kit to test whether a single agent with on-demand skills and knowledge-base search performs comparably.

Pennsylvania AI Ad Disclosure Law

What happened: Pennsylvania Governor Josh Shapiro signed the Artificial Intelligence Deceptive Advertising Act into law, requiring disclosure labels on AI-generated or AI-modified ads.

Key details:

  • The law requires any ad, whether still image, video, audio, or text, that 'has been significantly modified or generated by artificial intelligence' to include a label in the same format as the ad itself.
  • It applies regardless of where the ad was created, so out-of-state ads airing in Pennsylvania would be covered.
  • Exemptions include 'parody, satire, commentary or criticism' and content that is 'evidently fictional.'
  • Disclosure is required only for synthetic content that 'creates a false consumer perception' about a product or service.
  • The bill was introduced by State Senator Nick Pisciottano.

Why it matters: The Verge notes the law leaves room for interpretation, since 'significantly modified,' 'evidently fictional,' and 'false' are undefined, and Pennsylvania is one of the first states to require such disclosures.

Practical takeaway: Advertisers and agencies running campaigns that reach Pennsylvania should review AI-generated creative for the required labeling.

OpenAI Discounts Poach Anthropic Enterprise Customers

What happened: According to The Information, OpenAI is winning enterprise customers from Anthropic with steep discounts, and several companies say their AI costs fell sharply after switching.

Key details:

  • Rippling CEO Parker Conrad said his company raised OpenAI's share of its coding workload from 16 to 52 percent, while Anthropic's share fell from 57 to 18 percent.
  • Replit switched after OpenAI offered more than 30 percent off a contract worth over $100 million a year.
  • DataCamp CEO Jonathan Cornelissen said OpenAI's pricing cut its costs by 80 to 90 percent.
  • A US bank with more than $7 billion in assets had its ChatGPT seats doubled to 500 employees at no extra charge, paying under $7 per user instead of the usual $20 or more.
  • An investor memo reportedly said Anthropic's growth slowed from Q2 to Q3.
  • Chinese models from DeepSeek and Kimi are also adding price pressure.

Why it matters: The pricing push suggests OpenAI is trading margin for market share, and it adds competitive pressure on Anthropic's enterprise business.

Practical takeaway: Teams negotiating AI contracts should benchmark current OpenAI and Anthropic pricing, since reported discounts are substantial but based on individual customer accounts.

Nvidia Flora Open Robot Video Labeling Engine

What happened: Nvidia open-sourced Flora, a data engine from its GEAR research lab that labels robot training video with action descriptions, step ratings, and success judgments.

Key details:

  • Nvidia says Flora labels one hour of robot video for about 72 cents, versus roughly $88.50 with GPT-6 Astra; these cost figures are projections based on an assumed $10 per GPU-hour rental price.
  • In a 50-video test, Nvidia reports Flora reached 57.1 percent annotation quality versus 53.7 percent for GPT-6 Astra; Claude Opus 5 scored lower and cost about $42 per hour.
  • Flora consists of three fine-tuned Qwen3.5 models: a 9B model for splitting video into action segments, a 27B captioning model, and a 27B Flora-Rewarder.
  • Flora-Rewarder catches 53.3 percent of failed steps within an episode, versus no competitor above 11.7 percent and none caught by Astra; it falsely flags about 11 percent of normal steps as errors.
  • Nvidia says Flora is already annotating up to 200,000 hours of video per day across 512 GB300 GPUs during training of its GR00T models; models are on Hugging Face.
  • On Nvidia's benchmarks, Flora leads on human recordings but Astra wins on robot videos; Flora sometimes describes actions incorrectly (e.g., a wiping motion described as folding a napkin).

Why it matters: Robot training data is expensive to label with frontier LLMs, and an open, cheaper alternative extends Nvidia's push to cover the full robotics value chain. The cost and quality comparisons are Nvidia's own, and the article notes Astra evaluated descriptions while also being a model under test.

Practical takeaway: Robotics teams wanting cheap, open video annotation can evaluate Flora's Hugging Face models, while validating its quality claims on their own data.

NBER Study: AI Helps Junior Patent Attorneys but Skills Retention Varies

What happened: A National Bureau of Economic Research working paper reports a three-month randomized controlled trial with 133 patent attorneys across eleven US intellectual property law firms, finding AI boosts immediate drafting quality but retained skill depends on seniority.

Key details:

  • Two-thirds of participants had access to 'InFlow,' an unreleased AI assistant from Google Labs; the study was Google-funded.
  • Treated attorneys outperformed controls on drafting tasks at both the 10-day and 90-day marks, across all five scoring dimensions.
  • Gains were much larger for attorneys with fewer than seven years of experience than for seniors.
  • In a redlining task performed without AI, the treatment group's advantage came entirely from senior attorneys; juniors showed no benefit.
  • Blinded patent attorneys at an independent firm scored the work on enforceability, technical accuracy, strategic ambiguity, completeness, and clarity.
  • Limitations include a small sample driven by top attorneys' high billable rates, non-representative firms, no baseline skills measurement, and a three-month window.

Why it matters: The researchers suggest baseline expertise may be a prerequisite for turning AI-assisted practice into durable skill, which bears on how firms train junior staff.

Practical takeaway: Organizations deploying AI assistants for junior staff may want to assess unaided skill performance alongside productivity gains.

OpenMobile-2 Open Smartphone Agent Training Environment

What happened: Researchers from Nanjing University, SenseTime, and other universities released OpenMobile-2, two open-weight smartphone agent models built on Qwen3.5 and Qwen3.6, along with their training environment.

Key details:

  • Models are 9 billion parameters (Qwen3.5-based) and 27 billion parameters (Qwen3.6-based); the training environment covers more than 110 applications, including 35 rebuilt apps such as Taobao, JD, DiDi, Meituan, Slack, and Google Drive.
  • The team generated about 11,600 example workflows and more than 2,000 practice tasks; Google's Gemini 3.1 Pro served as the teacher model for those examples.
  • With data volume held constant, increasing app count raised success on unseen apps from 60.7 to 67.5 percent.
  • The 27B model scores 79.9 on AndroidWorld (up from 67.1 before training), and 50.4 on MobileWorld (up from 35.9).
  • On real Android devices, the larger model rose from 31.9 to 59.6 on SPA-Bench, which the authors call the best score among open models.
  • Direct commands (such as 'add to cart') in more than 50 simulated apps improved task success by up to 10.3 percentage points once models were trained on them; this remains experimental and exists only in simulated apps.

Why it matters: The results suggest app diversity in training environments may matter more than raw data volume for phone agents, which could narrow the gap between open and proprietary smartphone agents.

Practical takeaway: Developers building phone-control agents can reproduce the released environment, data, and instructions to test app-diversity training.