Measurement
Part of Measuring finance content so that activity, attribution and causal impact never blur
Finance creators measurement mistakes: three claims a reviewer should challenge
Review a finance-content report in England by challenging three claims: attention as outcome, attribution as cause and tracking as harmless.
A finance-content report can look rigorous and still make a claim its data cannot carry. Three claims deserve a challenge before anyone acts on them: that attention is an outcome, that credit is cause, and that tracking identities is harmless. Each has a question a reviewer can ask.
Rates, denominators, geography and benchmark populations are a separate checking job, set out in the benchmark protocol. The questions here are the ones to raise in a review meeting, with the evidence an acceptable answer should contain.
What to take away
- Views and clicks describe recorded activity, not understanding, suitability, financial benefit or avoided harm.
- Attribution assigns credit by convention; causation needs a comparison or control group.
- Personal data must be limited to the stated purpose, and cookies, pixels and link decoration need a PECR assessment.
- Where a definition changed, stop the comparison.
- Where causal evidence is absent, keep the result labelled observed or attributed.
Challenge the claim that attention is an outcome
Ask what outcome the report defined before it collected any numbers. Views and clicks describe recorded activity. They do not demonstrate understanding, suitability, financial benefit or avoided harm, so a report that stops at activity has not yet measured an outcome.
The FCA's outcomes-monitoring findings describe the same trap in regulated firms. Some firms used operational activity metrics, such as conversion rates or review completion, as a proxy for customer outcomes without defining good or poor outcomes at different stages of the journey.
The lesson transfers to creators. Write down what a good and a poor result look like for the reader at each stage, then choose the metric. The planning mistakes of vague tasks and vanity metrics begin before any data exists.
Challenge the claim that credit is cause
Ask whether the report compared exposed readers with a similar group that was not exposed. A code or last-click rule assigns a verified event by convention. It does not show what would have happened without the content. Only a comparison does that.
HM Treasury's QPIE guide says that monitoring change does not establish cause, and that confident attribution requires a comparison or control group. The ROI method keeps attributed and incremental results apart for exactly this reason.
Where no control group exists, the report should say which method it used and what claim that method can bear. Attribution methods are chosen by how strong a claim the decision needs, and a headline should not claim more than the method can.
Challenge the claim that tracking is harmless
Ask which personal data the report's tool collected and for what stated purpose. More granular tracking can create unnecessary privacy risk without improving the decision, and a tool permitting a field is no reason to collect it.
The ICO's data minimisation guidance says personal data must be adequate, relevant and limited to what is necessary for its purposes. A report that lists identities nobody used in a decision has already shown a gap.
Cookies, pixels and link decoration also need a PECR assessment. The ICO's final storage and access guidance covers consent, exceptions and user objection after the 2025 legislative changes. Data protection for finance creators is the place to document the answers.
Turn the questions into a review table
Put each question in writing and require an answer from the report's author. A table keeps the answers comparable, and a blank cell is a finding in itself. Return the report rather than fill the gap by assumption.
| Claim in the report | Question to ask | An acceptable answer contains | If it is missing |
|---|---|---|---|
| The content worked | Which outcome was defined before collection? | A written definition of good and poor outcomes for the reader | Label the result as activity only |
| The content drove results | Was there a comparison or control group? | The comparison group and how it was chosen | Keep the result labelled observed or attributed |
| The tool needed this data | What purpose does each personal-data field serve? | A stated purpose and a PECR assessment | Remove the field from collection |
| The trend is continuous | Did a definition change during the series? | The date, and the old and new definitions | Stop the comparison |
End every review with a decision
A review is finished when the report's reader knows what to do next. The measurement and reporting guide shows how activity, attribution and causal impact are kept apart in the report itself, and the routine below applies it.
- State the decision the report informs and the evidence that decision needs.
- Run the four questions in the table.
- Stop any comparison where a definition changed.
- Keep causal wording out of results labelled observed or attributed.
- Send the report back with the blank cells listed.
Common questions
Can views and clicks stand in for a customer outcome?
Only as a proxy, and only once the outcome has been defined. Views and clicks describe recorded activity. Without a written definition of a good and a poor result for the reader, the proxy has nothing to be checked against, and the claim should stay labelled as activity.
What does a control group add that last-click attribution cannot?
A last-click rule assigns credit according to a convention. A comparison or control group shows what would have happened without the content. Monitoring change over time does not establish cause on its own, so results without a comparison stay labelled observed or attributed.
Does a tracking tool's default setting decide what to collect?
No. Personal data must be limited to the stated purpose, and cookies, pixels and link decoration need a PECR assessment. More granular tracking can create unnecessary privacy risk without improving the decision, so collect only the fields that a stated decision actually uses.
