Follow
THE CODEX OBSERVATORY
QUALITY NOTEBOOK · REVIEWED SEP 15, 2026

Is your AI getting worse?

How to investigate Claude Code or Codex quality changes without confusing a bad session, a service failure and a repeatable regression.

START HERE

A worse answer is a signal to investigate.

If Claude or Codex feels “dumber,” first separate service errors, tools that fail, a longer conversation and changes in reasoning settings. A single bad result cannot identify which layer changed. NextReset tracks official incidents and client releases, alongside a limited Codex quality monitor with fixed tasks and recorded answers.

Which symptom are you seeing?

Wrong or incomplete work

Write down what a correct answer must do. Compare the selected model, reasoning setting, client version and conversation context before repeating the task.

Quota drains faster

Check usage records and repeated tool attempts. Consumption alone does not measure intelligence. Account limits and public resets are different mechanisms.

Can quality problems be real?

Yes. In its April 23, 2026 postmortem, Anthropic traced Claude Code quality reports to three separate product changes involving reasoning defaults, cached context and response instructions. It described remedies and affected surfaces; that report did not apply the same finding to the Claude API. This is a documented historical case, not evidence of a current regression. Read Anthropic’s account ↗

Community reports can reveal a pattern worth testing, but repeated copies of one complaint are not independent measurements. A provider postmortem establishes a particular incident. A controlled benchmark establishes performance on its own tasks and configuration. Neither gives a universal score for every user.

A useful before-and-after comparison

  1. Freeze the task. Use a saved prompt, repository revision and expected outcome. Keep a small task set you can grade with tests or a written rubric.
  2. Record the environment. Save the date, product, model selection, reasoning setting, client version, tools and input context. Keep API runs separate from chat or agent products.
  3. Repeat both conditions. One run can be lucky or unlucky. Repeat comparable tasks and report the number attempted, failed, timed out and passed. Do not silently discard errors.
  4. Separate the measurements. Report task success, instruction adherence, tool success, first-response latency and total completion time separately.
  5. Check uncertainty. Compare matched tasks across a stable baseline, include uncertainty intervals and predeclare the size of change that matters. A settings change starts a new comparison group.

A useful report says “6 of 20 repeated tasks failed under this configuration.” It does not turn that sample into “the model became 30% less intelligent.” Keep prompts and grading rules available when privacy and permissions allow.

NEXTRESET QUALITY LAB · CODEX PILOT

Follow the Codex quality notebook

The Codex quality monitor publishes eight fixed short tasks, independent checks, configuration details and daily SVG pelican samples. The selected model runs through the official Codex CLI using a subscription account. The pass rate applies to the measured tasks; it is not a universal intelligence score or a probability of degradation.

See the live page for the latest attempt, baseline progress and any collection gaps. A reference needs at least 14 complete, spaced rounds across seven Pacific dates. Visual preference votes stay separate from task scoring. Claude quality experiments have not started.

For an external benchmark, Marginlab’s Claude Code tracker publishes its own results and methodology. Its data is not a NextReset measurement.