Practical AI · 3-minute read
The dashboard says production is up. The team says lunch is now a concept. Both can be telling the truth.
AI can make one stage of a job faster while sending more work into the next stage. If you count drafts produced but overlook the people checking them, your productivity report may be congratulating a traffic jam.
Follow the work to the person finishing it
Juniper Lane Creative is a fictional marketing agency. Its writers use AI to prepare more campaign drafts. Two account managers still verify client claims, pricing, permissions and tone before anything leaves the building.
In an invented week, the team submits 60 drafts. Reviewers can finish 40 at the agreed quality standard. Unless the workload or process changes, 20 drafts remain unfinished. That is a queue, not 20 additional completed campaigns.
Start with one recurring job and define ‘finished’ together. For this agency, it means approved and ready for its intended channel. A generated draft does not meet that definition simply because it has stopped typing.
Count the burden as well as the output
Finished work: how many items meet the agreed quality standard?
Human effort: who spends time preparing, reviewing, correcting and maintaining the workflow?
Waiting: how long does work sit between stages?
Rework: how often does an item return, and for what reason?
Workload: are interruptions or after-hours tasks concentrating on particular roles?
Compare similar work over a useful period. Record changes in volume and difficulty, because a week of simple posts is not comparable to a week of complex proposals. Keep one-time setup costs separate from recurring effort, but record both.
Using this approved, task-level workflow log, summarize completed items, human effort by stage, waiting time and rework. Identify where work accumulates and which roles receive extra tasks. Separate observations from possible explanations. Flag missing records and changes in task difficulty. Do not rank individual employees or claim that AI caused a change. Suggest one small process test.
Ask the people behind the numbers
Have the process owner check the totals against original records. Then ask the people doing the work what the log misses. Use the exercise to improve the process, with a clear explanation of what is being measured. Avoid collecting unnecessary personal information.
NIST’s AI RMF Playbook recommends evaluating systems in realistic conditions and incorporating feedback about their actual effects. That supports looking beyond a model’s speed to the surrounding work. [1]
Juniper Lane might test a smaller draft batch, a clearer brief or protected review time. The useful result is more good work finished without quietly handing someone else the bill.
Your turn: Who is doing the extra checking that your AI productivity dashboard does not currently count?
Source [1]: NIST AI RMF Playbook: Measure 2.3. Supports evaluation in realistic conditions and stakeholder feedback. The agency, numbers, workload measures and prompt are original illustrative guidance.
