Skip to content

The Quality score

The Quality score says how well the Semantic Layer serves one Data Source, from 0 to 100, with the reasons for it. The agent judges it: no formula computes it. It weighs how much of what users query the Data Model covers, whether the Context Entries answer what users ask, and what the Data Source’s Query Log shows people ask. See Assess the Semantic Layer for how it judges.

Because the agent judges, two assessments of an unchanged Semantic Layer can differ by a few points. Each score names the model that judged it, so that you can tell a model change from a real one.

In the Data Studio, open a Data Source: its overview shows the Quality score panel.

  • Everyone who sees the Data Source’s entries sees the last score, its date, the model that judged it and who started it, and Earlier assessments, the trend of the scores before.
  • Users who may accept Context Change Requests on the Data Source also see the Reasons: what the layer covers, what is missing, and which questions fail. They come from the Query Log, which only those users may read, so others see that reasons exist but not what they say.

A Data Source that was never assessed shows Not assessed yet.

You need the Right to accept Context Change Requests on the Data Source.

  1. Select Assess in the Quality score panel.
  2. The Web App opens with a new Conversation of yours, already sent, in which the agent assesses the Data Source. It needs nothing more from you: it asks no questions, proposes no changes, and uses no tool that would need your Approval.
  3. When it ends, the Data Studio shows the new score with its reasons.

While an assessment runs, the panel says Assessment running, with who started it and when, and Open its Conversation. Only one assessment of a Data Source runs at a time.

The Conversation stays among your Conversations as the record of how the agent reached its score. Open it any time to follow its reasoning.

If the Conversation ends without a score, or runs for more than an hour, the assessment counts as failed. The panel says the last assessment didn’t finish, and the previous score stays as it was. Select Assess to try again.

Ask the agent to improve the Semantic Layer of the Data Source. If you may accept Context Change Requests on it, Improve the Semantic Layer starts from the latest assessment’s reasons, and proposes the entries that close them as a Context Change Request.

Each assessment is kept for one year, and deleted with its Data Source. When the user who started one leaves your Tenant, their name is taken out of it. Every assessment started and every score saved is recorded in your Tenant’s Audit Log.