How to run a phishing exercise that improves reporting
Share
Published
A useful phishing exercise measures whether staff recognise, report and safely handle a suspicious message. Define the learning objective, obtain internal authorisation, use harmless simulations and interpret results alongside the difficulty of the message.
A low click rate looks reassuring on a training dashboard. It may also mean that the test email was obvious, irrelevant or identical to last month’s exercise. A high click rate may reveal a difficult scenario, a confusing workflow or a reporting button that staff cannot find.
The exercise should help the organisation understand those conditions. A public list of people who clicked contributes little to explaining what needs to change.
Choose one behaviour to practise
Pick a realistic task: recognising an unexpected sign-in prompt, verifying a payment-change request or reporting a questionable attachment. Decide what a good response looks like and make sure staff have the tools to perform it.
If the intended response is to report the message, check the reporting route first. Who receives reports? Is it monitored during the exercise? Can the team distinguish the simulation from a real incident? Staff should not have to guess which private inbox the security team prefers this week.
NIST’s Phish Scale User Guide provides a method for rating how difficult a phishing email is for people to detect. It gives trainers a way to interpret results in context rather than treating every click percentage as comparable.
Set the boundaries before sending
Get approval from the responsible management, security and privacy functions. Record the audience, schedule, message scenario, information collected, access to results and deletion date. Include a stop procedure if a real incident occurs or the exercise causes unexpected disruption.
Use an approved simulation environment. Do not collect real passwords, deliver malware or make a real payment demand. If the learning objective involves a sign-in page, the exercise can record the relevant interaction without storing a person’s actual credential.
Avoid scenarios that exploit a particular employee’s bereavement, health situation or other personal vulnerability. A business-relevant exercise can be credible without using distress as a training tool. Provide an appropriate explanation afterwards and a route for questions.
| Measure | What it can tell you | What it cannot prove |
|---|---|---|
| Messages delivered | The audience actually reached | That every recipient read the message |
| Reports received | Use of the reporting route | That all unreported messages were trusted |
| Time to first useful report | How quickly the response team obtained actionable detail | That the whole organisation would respond equally quickly |
| Simulation interactions | Where the scenario persuaded people to act | A person’s overall security competence |
| Debrief feedback | Confusing cues or workflow barriers | An independently verified reduction in breaches |
A small-team example
In this fictional exercise, 40 staff receive an approved simulation. Eight report it through the designated route. Three interact with the simulated sign-in page. Several others tell the trainer that they deleted the message without reporting it.
The trainer records the delivered-message count and defines each metric. Eight reports out of 40 delivered messages is a 20% reporting rate for that exercise. It is not evidence that the other 80% would fall for a real attack. The three interactions are another observation, interpreted alongside the scenario’s difficulty.
The debrief finds that mobile users cannot see the reporting button without opening a menu. The team improves the reporting instructions and practises the process again. This gives the exercise a concrete operational outcome beyond the headline percentage.
Offer practice before measuring people
Privacy Needle’s Spot the Phishing game and related training games can support discussion during a staff session. Use a fictional example, ask learners to explain their choice and connect it to the organisation’s own reporting route. A game score is a learning aid, not a certification that a participant can detect every real attack.
Combine that discussion with a short incident-reporting practice. The wrong-recipient email guide is useful for showing that staff should also report accidental disclosures, rather than reserving the reporting route for suspicious links.
Close the exercise with a change record
Record the scenario, difficulty assessment, denominator for each metric, lessons, corrective action and owner. Limit individual-level results to people with a legitimate need to see them. Apply the agreed retention period to platform exports as well as the platform itself.
Compare later exercises only when the audience, difficulty and measurement method are sufficiently similar. Our privacy-training measurement guide explains how to track changes in day-to-day behaviour without treating completion certificates as proof of effectiveness.
Common questions
Is a zero-click result proof that staff are safe?
No. It describes one exercise. Message difficulty, delivery, prior familiarity and the audience all affect the result.
Should we publish individual failures?
Use results for proportionate support and improvement. Team reporting can usually focus on aggregate findings and process changes while individual coaching remains private.
How often should we run a simulation?
Choose a schedule that supports learning and follow-up. Repeating tests without fixing the problems they expose can produce more data without a better response.
Featured image: AI-generated conceptual illustration.
