ServiceNow Change Data Quality AI Agent: A UK CAB Gate for Persistent Scores

Why persistent change quality scores matter now

In the Australia release, ServiceNow’s Change Data Quality AI Agent stopped being a one-shot panel trick and became something CAB can actually govern: quality scores now persist in the AI Change Quality Scores [ai_change_quality_score] table.

Before that change, a quality assessment lived in the moment you looked at it. You could not reliably wire it into approval conditions, risk assessment, audit evidence packs, or monthly trend dashboards. Governance was only as good as the last score someone happened to open.

For UK platform, change, and SRE teams running Zurich/Australia estates, that is the difference between “AI helped the change manager once” and “change quality is an operational control with owners, thresholds, and evidence.”

This playbook is a UK CAB production gate for turning persistent scores on without letting autonomous agents rewrite production change fields behind your back.

What the agent actually assesses

The assess quality of a change request agentic workflow rates the change request as a whole and the usual CAB-facing fields:

Ratings use the platform scale: Excellent, Very good, Good, Fair, Poor, Very poor, or Incomplete. If every field is Excellent, the agent does not invent suggestions for the sake of looking busy. The assessment lands in Work notes and in ai_change_quality_score.

Two paths: policy document vs similar changes

On execution the agent calls Get Change Quality Policy Document:

1. Active Change Policy Control found — rate against that policy; suggest only fields the policy defines; store a reference to the Change Policy Control record on the score row.
2. No policy found — fall back to similar closed change requests (semantic index path).

For UK regulated CAB, prefer the policy document path for Normal and Emergency models that auditors care about. Similar-changes is useful for Normal low-risk or when you are still maturing written policy, but it is a weaker evidence trail: the score row’s Change Policy Control field stays empty.

Change Policy Control hygiene

• Create records under Change Policy Control; attach the real policy document; let Ingest policy document extract criteria into Policies and set Active.
• Only one active policy per scope (change model or change type). Creating a new active policy deactivates the previous one for that scope — treat that as a CAB-visible configuration change.
• Deactivating a policy later does not rewrite historical score rows; they keep pointing at the policy version that applied when scored. That is good for audit.
• Reassessing the same change request overwrites the score row. There is no multi-version history in the table — design Platform Analytics and evidence exports accordingly (export before reassessment if you need a freeze).

Supervised vs autonomous: pick for CAB, not for demos

The change quality assessor ships in two modes:

Child agents (policy-document path and similar-changes path) each have versions. Set the active version in AI Agent Studio → Create and manage → View versions. Note: Set Change Field / Set chosen Change Fields tools remain supervised even when the agent version is autonomous — still treat Version 2 as the production default so score recording never depends on field mutation.

If you need autonomous field drafting, keep that on the separate Change request plans AI agent under a different CAB story. Do not conflate “score quality” with “rewrite the RFC.”

Role, channel, and clone traps

Wiring scores into real governance

Persistence only pays off if something consumes the score.

1. Approval and risk conditions

• Add change approval conditions that require a minimum overall score (or rating band) for Normal changes above a risk threshold.
• Keep Emergency paths explicit: either a lower threshold with named emergency approvers, or a mandatory reassessment after the incident bridge closes.
• Document the threshold in the Change Policy Control attachment so the agent and the CAB are reading the same rules.

2. CAB evidence pack

For each CAB item export or screenshot:

• Change number, model, type
• Overall score + rating + explanation
• Per-field scores
• Change Policy Control reference (or “similar changes” if fallback)
• Timestamp of assessment (remember: reassessment overwrites)

3. Platform Analytics trends

Build a line chart of average score by month on ai_change_quality_score. Group by assignment group, change model, or template. Pin it to the change management dashboard. This is how you spot a team whose implementation plans are systematically Incomplete before MIM notices.

4. SRE / platform coupling

Poor backout and test plans are leading indicators for failed changes. Feed low-score cohorts into your weekly change failure review alongside failed changes and PIR themes. The agent does not replace PIR; it makes weak RFCs visible earlier.

UK production gate checklist

Use this as the CAB exit criteria before enabling scoring in production:

1. Australia (or later) Now Assist for ITSM capabilities confirmed on the target instance; plugin/licence owners named.
2. Change Policy Control active for the models in scope; ingestion completed; Active flag verified.
3. Version 2 (autonomous scoring) active for both child agents in PROD; Version 1 only where a named pilot accepts field writes.
4. Otto panel Display enabled; role sn_itsm_aia.sn_aia_chg_quality present on change managers and CAB coordinators (not every fulfiller if you want controlled rollout).
5. Semantic index for similar changes healthy if any model still uses the fallback path.
6. Approval condition or CAB checklist item references the score threshold; exception process written.
7. Platform Analytics widget live; ownership for monthly review assigned (Change Manager + Platform Owner).
8. DPIA / AI use-case register updated: scoring uses change text and optionally similar historical changes — data residency and retention already covered by your instance DPIA, but call out the new table and export practices.
9. Reassessment policy: who may rescore, and when evidence must be exported first.
10. Runbook for “agent suggests Excellent but CAB disagrees” — human override wins; score is an input, not the approver.

What not to do

• Do not treat a high score as automatic CAB approval. Scores measure documentation quality against policy or peers, not business risk acceptance.
• Do not leave Version 1 autonomous-adjacent habits in PROD where field writes surprise implementers mid-freeze.
• Do not clone the workflow and forget the semantic index — silent degradation looks like “AI is useless.”
• Do not put policy text only in Confluence. If the agent cannot ingest it via Change Policy Control, CAB and the agent will diverge.
• Do not expect multi-row history in ai_change_quality_score; design exports if you need a forensic trail.

Bottom line

Persistent change quality scores turn Now Assist from a helpful panel into a control plane input. For UK estates, the safe production pattern is clear: policy documents first, autonomous Version 2 for scoring, humans for field mutation and CAB decisions, and Platform Analytics so trends outlive any single CAB meeting.

Ship the gate before you celebrate the demo.