Introducing a phishing simulation: the process in seven steps

What a phishing simulation is — and what it is not

A phishing simulation is a controlled, harmless test email to your own workforce. It does not measure what people know, but how they react in everyday work. The topic carries weight: According to the ENISA Threat Landscape 2025, phishing was the dominant route for initial access to a network in the reporting period July 2024 to June 2025, at around 60 percent — in ENISA's count including vishing, malspam, and malvertising. Nevertheless, three clarifications belong at the beginning. First, a simulation should not be a performance check and is not aimed at individuals — the fact that it would be technically suitable for this is precisely the reason for the co-determination step further below. Second, it is not instruction within the meaning of Section 12 of the Occupational Health and Safety Act (ArbSchG): That provision concerns safety and health protection at work, not information security. Third, IT-Grundschutz does not require it: The module ORP.3 "Awareness Raising and Training" of the IT-Grundschutz Compendium (2023 edition, still the current version) contains no requirement regarding simulations; the word does not appear there. The BSI merely recommends them in a guide on IT security awareness. How a simulation is embedded in a larger security awareness program is described elsewhere — which also includes provider-based effectiveness figures that are not consistent with independent research; more on this below. This article is about introducing the individual instrument, in seven steps:

1. clarify the legal framework
2. define the goal and the measure of success
3. collect the baseline unannounced
4. select and document the lure
5. run the campaign
6. evaluate in aggregate
7. provide follow-up training and set the rhythm

Step 1: clarify the legal framework

This step deliberately comes first because it cannot be remedied afterwards. A simulation processes personal data of employees — who clicked on what and when. Consent is, as a rule, not a sustainable basis here: The Independent Centre for Privacy Protection Schleswig-Holstein pointed out as early as 2018 that the power imbalance in the employment relationship stands in the way of voluntariness and that consent informed in detail would distort the reactions to the test. What is therefore being discussed is the route via a collective agreement under Section 26 (4) in conjunction with subsection 1 of the Federal Data Protection Act (BDSG) and Article 88 GDPR. This includes being open about the fact that the legal situation is in flux: In 2023, in case C-34/21 concerning a Hessian provision, the European Court of Justice ruled that national rules on employee data protection must remain unapplied if, beyond repeating the general legal bases, they do not provide for any specific protective measures under Article 88 (2) GDPR. Because Section 26 (1) sentence 1 BDSG largely corresponds to the state provision examined, supervisory authorities assume that the ruling also affects it; this has not yet been decided for Section 26 BDSG. And there is not yet any guidance from the Data Protection Conference specifically on phishing simulations.

The second strand is co-determination. Section 87 (1) no. 6 of the Works Constitution Act (BetrVG) gives the works council a right of co-determination in the "introduction and use of technical devices designed to monitor the behavior or performance of employees" — insofar as no statutory or collective bargaining provision exists. When a device is "designed" for this purpose is described by the Federal Labour Court in its established case law: What matters is whether it is objectively suitable for collecting or recording information on behavior or performance; a subjective intention of the employer to monitor is irrelevant. The court most recently confirmed this principle in its decision of July 16, 2024, which refers to decisions from 2013 and 2016 for this purpose. In federal authorities, a provision with identical content can be found in Section 80 (1) no. 21 of the Bundespersonalvertretungsgesetz; for federal states and municipalities, the respective state staff representation acts apply.

Steps 2 and 3: set the goal and collect the baseline

Before the first email goes out, the campaign needs a named goal — and one that is not called "low click rate". Viable goals are ones such as: We want to know how quickly suspicious emails are reported, and we want to increase this reporting rate over four runs. For measuring success, the ORP.3 module provides for the standard requirement ORP.3.A8 "Measuring and evaluating learning success": The measurements should take into account both quantitative and qualitative aspects. A single percentage does not do justice to this.

The baseline is the first, unannounced run. It does not serve as an assessment but as a zero measurement: Without it, it is impossible to say later whether a change is due to the measure or to an easier lure. It is important that the works council or staff council and those responsible for data protection are already involved at this point — unannounced means unannounced to the workforce, not to the committees. Two decisions also belong in this step: Will the entire workforce be contacted or a sample, and will the project as such be announced in advance? Both are justifiable as long as they are documented and kept constant across runs — otherwise you end up comparing apples and oranges later.

Step 4: choose and document the lure

The difficulty level of the lure determines the result more than any training beforehand. In the randomized field study at the university hospital of the University of California San Diego, discussed in more detail below, the failure rate varied between 1.8 and 30.8 percent depending on the lure — with the same workforce. So anyone who compares campaigns of different difficulty is measuring the lure, not the workforce. A structured method for classifying difficulty is the Phish Scale of the US NIST: It rates lures on the basis of comprehensible characteristics — how conspicuous the warning signs are and how plausible the scenario is for the respective target group — instead of by gut feeling. In practice, this means: document the difficulty level, keep it constant across runs, or disclose changes. In terms of content, it is worth looking at typical patterns from social engineering; which characteristics employees should recognize is described under Recognizing phishing.

Step 5: run the campaign

Technically, two preparations are needed: a sender that does not get caught in the spam filter, together with a coordinated exception in the protection systems — otherwise the campaign measures the quality of the filter instead of behavior — and a reporting channel that works at the time of the campaign and that everyone knows. For the technical process, the State Office for Information Security in Bavaria has developed installation instructions for the open-source tool "gophish" that follow the three-step approach of knowing, practicing, measuring and explicitly provide for anonymized evaluation; they are available from the State Office's advisory service.

Running the campaign involves three decisions that are often forgotten. First, the duration: A window of a few working days reflects everyday work better than a single cut-off date on which vacations and shift work distort the picture. Second, the landing page after the click: It should explain in one sentence what just happened and do without assigning blame. Third, the communication afterwards: Anyone who hears nothing after the wave experiences the simulation as a trap rather than an exercise. A short, aggregated review sent to all employees concludes the run.

Step 6: evaluate without exposing anyone

The evaluation is the point at which simulations tip over. Results should be reported in aggregate — by department or location, not by person. Individual evaluations are not only sensitive under data protection law, they are also counterproductive: The UK's National Cyber Security Centre states in its guidance on phishing attacks that blaming employees for clicking does not work — and that an organization that rewards the absence of reports is cultivating silence. In practice, this means: no ranking, no feedback by name to managers, no employment law consequences from a click. Anyone who reports gets a confirmation; anyone who clicks gets an explanation.

What should be included in the collective agreement follows from Article 88 (2) GDPR: It requires suitable and specific measures to safeguard the human dignity, legitimate interests, and fundamental rights of employees — expressly also with regard to monitoring systems at the workplace. The Independent Centre for Privacy Protection Schleswig-Holstein concludes from this for phishing simulations that the general requirements continue to apply unchanged — from the principle of data minimization through transparency and technical and organisational measures to the rights of data subjects. In practice, these are three decisions: Who gets access to the campaign's raw data, how long they are retained, and from what point only the aggregated evaluation is available. Anyone who no longer needs the personal reference after the evaluation should no longer keep it.

Step 7: follow-up training and repetition — what the evidence shows

At this point, the article calls for some uncomfortable honesty. The methodologically strongest study published to date — randomized and with a genuine control group — on the effect of phishing training is a field study at the university hospital of the University of California San Diego, presented at the IEEE Symposium on Security and Privacy 2025: over eight months, ten campaigns, more than 19,500 employees. Result: The annual security training showed no statistically significant effect on the probability of clicking; the short training displayed immediately after the click improved it by 1.7 percentage points. More than half of these training sessions were clicked away within ten seconds. A long-term study by ETH Zurich with more than 14,000 employees produced a similar picture. Both data sets come from the US and Switzerland; no comparable German survey exists.

A second finding of the San Diego study is remarkable: With purely static training material, the probability of clicking again the next time increased with each additional completed session. Repeating more often is therefore not an answer to an ineffective format. In any case, nobody prescribes a fixed interval; what makes sense is a rhythm that matches the measurement of success. It does not follow from this that simulations are useless — it follows that they are suitable as a measuring instrument and not as a learning measure. The BSI, too, has a separate section in the ORP.3 module on ineffective awareness activities, listing among other things a lack of management support, unclear goals, and insufficient measurement of success. The conclusion for practice: The simulation provides the situational picture; the actual knowledge transfer takes place through recurring, short learning units — at Bridgly via the training courses on data protection and information security. For public authorities, the framework is described under Security awareness in public authorities.

Why the click rate measures nothing without the difficulty of the lure

Which metrics an awareness program tracks overall is described in the program article. For the individual simulation, there is an additional condition that remains open there: Without a documented difficulty level, none of these figures are comparable across two runs. A click rate that falls from twelve to six percent can mean learning success — or simply an easier lure in the second wave. The range of 1.8 to 30.8 percent with the same workforce shows how large this effect is. A time series therefore only becomes robust when the difficulty level is recorded for each wave and taken into account in the comparison — and when the reporting rate and the time to the first report are collected alongside the click rate, because a single early report can stop an entire wave. How robust such measurements are overall is shown by an independent recalculation from 2025 based on the NIST methodology: It found no significant training effect for either clicks or reports.