# Measuring The Impact Of AI In Council Communications

URL: https://www.starien.com/blog/measuring-the-impact-of-ai-in-council-communications
Published: 2026-10-06
Author: Geoffrey Moffett
Topics: measurement, council communications, councils, AI, evaluation

> How to measure the impact of AI in council communications honestly: baselines, time saved, quality, reach, resident outcomes, risks and reporting to leaders and members.

## Quick answer

To measure the impact of AI in council communications, set a baseline for specific tasks before using AI, then track time spent, quality (errors, readability, approvals needed), output and reach, resident outcomes such as fewer repeat queries, and risks or incidents. Report results honestly, including where AI didn't help.

## Key takeaways

- Without a baseline, you can't show any impact.
- Measure specific tasks, not 'AI' in general.
- Quality and risk measures matter as much as time saved.
- Report honestly, including what didn't work.

"Is AI actually helping?" Senior leaders will ask. So will elected members, auditors and possibly the local paper. Measuring the impact of AI in council communications gives you an honest answer, and helps you decide where to use it more, and where to stop.

## Why Measuring The Impact Of AI In Council Communications Matters

- **Justifying investment** in tools and training.
- **Finding what works** so you can spread it.
- **Spotting what doesn't** before it causes problems.
- **Accountability.** Public bodies need to show their decisions are evidence-based.

## Step 1: Pick Specific Tasks

"AI" is too broad to measure. Pick three to five tasks, such as:

- press release first drafts;
- social media posts for the weekly calendar;
- web page plain-English rewrites;
- consultation response theming;
- staff bulletin editing.

## Step 2: Set A Baseline

Before introducing AI for those tasks, record for a few weeks:

- **time taken** per task, from start to approved;
- **approval rounds** needed;
- **errors** caught at approval or after publication;
- **output volume** per week.

Keep it simple: a shared spreadsheet where officers log tasks. Accept that it's approximate, and say so in reports.

## Step 3: Measure With AI

Track the same measures once AI is used. Include the checking and editing time. Being honest about this is important; AI drafts are fast, but checking takes real time, and the true saving is the difference in total time.

## Step 4: Measure Quality

Time saved is meaningless if quality drops. Track:

- **accuracy:** errors caught before and after publishing;
- **readability:** use a readability checker on samples before and after;
- **consistency:** do messages match across channels?
- **tone:** feedback from senior officers and service teams;
- **accessibility:** proportion of images with alt text, videos with corrected captions.

## Step 5: Measure Reach And Resident Outcomes

Where AI helps you publish more, or better, look at outcomes:

- engagement on social content (but don't chase vanity numbers);
- web page visits and time on page for rewritten pages;
- **repeat queries:** do calls or messages about a topic drop after its page is rewritten?
- **AI search accuracy:** are AI assistants answering your top questions correctly? See [AEO for council websites](/blog/aeo-for-council-websites-bin-days-planning-and-leisure-centres/).

Outcome measures take longer and have many causes, so describe them carefully.

## Step 6: Track Risks And Incidents

Measure the downside too:

- data incidents or near-misses involving AI;
- inaccurate AI content published;
- complaints relating to AI use;
- policy breaches spotted.

A rise in reported near-misses after training can actually be good news: people are noticing and reporting.

## Step 7: Ask Staff

A short quarterly survey:

- How confident are you using AI for your work? (1–5)
- Which tasks has it helped with most?
- Where has it caused problems or extra work?
- What would help?

## A Simple Reporting Template

> **AI in communications: quarterly update**
>
> **Tasks in scope:** [list]
> **Time:** [task] average went from [x] to [y] minutes, including checking.
> **Quality:** [errors, readability, approval rounds].
> **Outcomes:** [any resident-facing changes, described carefully].
> **Risks:** [incidents, near-misses, actions taken].
> **What didn't work:** [honest notes].
> **Time reinvested in:** [e.g. more proactive media work, accessibility improvements].
> **Next quarter:** [plans].

## Careful With Savings Claims

Time saved by comms officers rarely becomes a cashable saving. It usually gets reinvested: more proactive stories, better accessibility, faster responses. That's valuable, so say what it was used for rather than converting hours into pounds.

## A Made-Up Example: Three Months Of Measurement

Imagine a comms team picks three tasks: weekly social post drafts, press release first drafts and plain-English rewrites of service pages. For four weeks before using AI, officers log time on each task in a simple spreadsheet. Then they use AI for the same tasks for three months, logging time including checking.

The results are mixed, and that's useful. Social post drafting is noticeably quicker. Press releases are quicker to draft but checking takes longer than expected, so the overall saving is modest. Plain-English rewrites are much quicker, and a readability check shows clearer pages. One AI-assisted social post went out with a wrong date, caught by a resident and corrected within the hour, which leads to an extra step in the checklist.

The quarterly report says all of this plainly, including the error, and explains that time saved was used to rewrite more service pages. Senior leaders get an honest picture, and the team gets support to continue.

## Choosing Measures That Matter To Leaders

Different audiences care about different things:

- **senior leaders:** risk, value and capacity;
- **elected members:** resident experience and accountability;
- **service managers:** fewer repeat queries and clearer information;
- **the comms team:** workload and quality.

Pick a small set that speaks to each, and report them consistently over time so trends become visible.

## Keep It Proportionate

Measurement shouldn't become a burden that eats the time AI saves. Log a few tasks for a few weeks at a time, rather than everything forever. Repeat the exercise when you introduce a new tool or approach, and keep the spreadsheet simple enough that officers actually fill it in.

## Pitfalls To Avoid

- **No baseline.**
- **Forgetting checking time.**
- **Vanity metrics** like post volume.
- **Only reporting good news.** Credibility comes from honesty.
- **Measuring too much.** A few good measures beat a dashboard nobody reads.

## How Starien Can Help Train Council Teams

During six years leading Coleraine BID's digital delivery, we reported regularly to a board on what was working, including a 40%+ increase in engagement. Our [AI Coaching](/ai-coaching/) includes setting baselines and simple measures, so teams can show what changed. Our [Public Relations](/public-relations/) experience helps you tell that story clearly. [Get in touch](/contact/) to talk about training.

## Frequently asked questions

### How do you measure time saved with AI?

Time a set of specific tasks before introducing AI, then time the same tasks with AI, including the checking and editing time. Compare averages over several weeks.

### What quality measures should councils use?

Errors caught at approval, corrections after publication, readability, number of approval rounds and feedback from service teams and residents.

### Should time saved be reported as cost savings?

Be careful. Time saved is often reinvested in other work rather than cashable savings. Describe what the time was used for.
