Skip to content

Standard Skill: Validate an Analysis

Validate an Analysis is a Standard Skill: it catches the errors that look right before a number reaches someone who acts on it. The agent loads it before it gives you a rate, share, average or total that answers a question about your data, and when you ask it to check, verify or review a number, a report or an analysis, its own or one you share. To pick it yourself, type / and choose validate-an-analysis.

  • The definitions. Each metric against its Term or Rule in the Semantic Layer, because the commonest error is a correct calculation of the wrong thing.
  • Every denominator. For each rate, share or average, numerator and denominator separately: same population, same period, the right unit, no group missing or added. With the Data Source, it re-computes them with queries. An average of averages is re-computed, weighted.
  • Joins and counting. Row counts before and after each join, because a join can multiply rows and inflate sums without any error.
  • Population and period. Who is missing (survivorship), incomplete periods, the time zone dates are cut in, and segments defined by their outcome.
  • The results for sense. Every total with a matching Benchmark, one key figure re-computed a second way, shares that add up, and the axes and scales of Cards.

Without a Data Source, it checks the reasoning and arithmetic only, and says that nothing was re-computed.

A verdict, Ready, Ready with caveats or Needs fixing, with each issue by severity, the corrected figure with its numerator and denominator for every number it found wrong, and the caveats to state. The agent never gives you a number of its own that this check found wrong: it fixes it first.

When the check shows something missing from the Semantic Layer, such as a Term the analysis had to guess, it suggests a Context Change Request, but never opens one in this Skill.

This Skill is adapted from the validate-data Skill of Anthropic’s Data plugin, under the Apache License 2.0. Its folder holds the licence and a notice of what we changed; ask the agent to show you /skills/standard/validate-an-analysis/NOTICE.md.