Measurement
Part of Measuring finance content so that activity, attribution and causal impact never blur
Finance creators key metrics: six rates with numerators, denominators and exclusions, tested on GOV.UK data quality
Six finance-content metrics with exact numerators, denominators and exclusions, checked against GOV.UK data quality dimensions and FCA outcome monitoring.
Finance creators key metrics should answer separate questions about delivery, response, verified events, attribution, correction and understanding. Each question needs its own rate, with the numerator, denominator and exclusions written down before anyone reads the result.
What to take away
- Each metric answers a separate question, so no single ratio proves reader welfare or causal impact.
- Name the numerator, denominator and exclusions for every rate, and record any further exclusion beside the result.
- Check each rate against a data quality dimension such as completeness, uniqueness, timeliness or accuracy.
- Counts of completed processes are not outcomes, so pair every operational rate with an outcome measure.
- Do not generalise a convenience sample to England, and give the exact question, route, sample size and uncertainty.
Set the period and the quality test
The period for every example is the reporting month in the publisher's stated time zone. Exclude staff tests, identified automated traffic and deleted test content through a written rule.
The Government Data Quality Framework gives six dimensions to test each input. Completeness describes the degree to which records are present. Uniqueness is the degree to which there is no duplication. Validity is whether data is in the range and format expected.
Accuracy describes the degree to which data matches reality, and timeliness is whether the data reflects the period it represents and is up to date. Consistency covers values that do not contradict each other for the same entity. Log which dimension each metric is most exposed to.
Eligible delivery rate
Numerator: messages delivered. Denominator: eligible attempts in the month. Exclude drafts and cancelled items. This is an operational measure, most exposed to completeness and uniqueness, because missing or duplicated send records distort it.
It says nothing about whether a person noticed, understood or benefited from the content. A review of outcomes monitoring found some firms reported metrics showing a process had been completed rather than insight into outcomes delivered. Treat delivery the same way.
Link response rate
Numerator: unique valid visits to the reviewed destination from an included tagged link. Denominator: eligible delivered messages containing that link during the month. Exclude scanners, staff visits and records with unreliable delivery data.
If a platform supplies impressions but not delivered messages, create a differently named rate. Do not join unlike denominators into one series. The uniqueness and validity dimensions matter most here.
Observed completion rate
Numerator: verified eligible business events completed within the defined follow-up window. Denominator: eligible started events from the same cohort. Exclude tests, duplicates and records without verification.
An observed event is not proof that the content caused it. Accuracy is the exposed dimension, so state how an event was verified and by whom.
Attributed outcome share
Numerator: verified outcomes assigned to the content by one named rule from the attribution methods page. Denominator: all verified outcomes eligible for assignment in the period.
Changing the rule changes the answer. Keep the observed outcome count alongside the attributed count so model movement is visible. Consistency is the exposed dimension, because the same event must not be credited twice.
Material correction rate
Numerator: items first published in the month that later receive a material factual correction. Denominator: all items first published in that month. Exclude cosmetic spelling changes under a written correction policy.
Add late discoveries back to the original publication cohort and show the extraction date. A low rate can reflect weak detection rather than accurate work, so add root causes and time to containment. Timeliness is the exposed dimension.
Tested understanding rate
Numerator: eligible participants answering the predefined core question correctly without prompting. Denominator: all eligible participants who received that question in the period.
Give the exact question, recruitment route, sample size and uncertainty. In its consumer understanding review, the FCA reports that firms use proportionate tools such as surveys, comprehension checks and before-and-after A/B testing. Do not generalise a convenience sample to England.
An illustrative month
The figures below are invented to show the arithmetic. They are not benchmarks.
| Metric | Numerator | Denominator | Rate |
|---|---|---|---|
| Eligible delivery rate | 1,900 delivered | 2,000 eligible attempts | 95% |
| Link response rate | 190 unique valid visits | 1,900 delivered with link | 10% |
| Observed completion rate | 15 completed | 60 started | 25% |
| Attributed outcome share | 12 assigned | 40 verified outcomes | 30% |
| Material correction rate | 2 corrected | 50 items | 4% |
| Tested understanding rate | 56 correct | 80 participants | 70% |
Interpret the set
No single ratio demonstrates reader welfare or causal impact. Firms must regularly monitor the outcomes retail customers receive, so a finance creator working with a regulated firm should expect the partner to ask what the data shows and what action followed.
Before reporting, retain raw numerator and denominator counts, source version, data delay, missing records and calculation code. If the denominator changed, stop the trend line and start a new series. Common errors are listed under measurement mistakes.
The wider measurement and reporting guide separates activity, attribution and cause. Use the benchmark research page before comparing any rate with another team, and the quality checklist before release.
Before you act
- State the reporting month and the publisher's time zone.
- Exclude staff tests, automated traffic and deleted test content.
- Name the attribution rule and publish its window.
- Keep observed outcome counts alongside attributed counts.
- Record source version, data delay and missing records.
- Stop the trend line at any denominator break.
Common questions
What should I do if a platform gives impressions but not delivered messages?
Create a differently named rate rather than forcing the two together. Do not join unlike denominators into one series, and exclude messages with unreliable delivery data.
How should I handle a material correction found after the reporting month?
Add late discoveries back to the original publication cohort and show the extraction date. Exclude cosmetic spelling changes under the written correction policy.
Which data quality dimension should I check first?
Start with the one the metric is most exposed to, such as completeness for delivery or accuracy for completion. Then record the check beside the result so a reviewer can repeat it.
