Topics in this domain
- Event vs Incident vs Breach
- Developing the Incident Response Plan
- Incident Response Team Roles and Authority
- Incident Classification and Severity Levels
- Incident Response Playbooks and Decision Trees
- Escalation and Communication Plans
- Business Impact Analysis
- RTO, RPO, MTD and Recovery Prioritisation
- Business Continuity Planning and Strategies
- Crisis Management and Continuity Governance
- Disaster Recovery Planning
- Recovery Sites: Hot, Warm and Cold
- Backup Strategies and Cloud DR
- Incident Response Training
- Testing Plans: Tabletop, Functional and Full Interruption
- Evaluating Exercises and Improving Readiness
- Security Event Detection and Monitoring Methods
- SIEM, the SOC and Event Correlation
- Threat Intelligence and Indicators of Compromise
- Incident Triage and Initial Investigation
- Evidence Preservation, Forensics and Chain of Custody
- Containment Strategies: Short-Term and Long-Term
- Making Containment Decisions
- Breach Notification and Regulatory Reporting
- External Communications: Customers, Media and Law Enforcement
- Eradication, Recovery and Return to Normal Operations
- Post-Incident Review and Root Cause Analysis
- Lessons Learned and Incident Trend Analysis
Evaluating Exercises and Improving Readiness
The point of the exercise
Exercises aren’t exams to pass. Their value lies in the gaps they reveal and what changes afterwards. A well-run evaluation turns a two-hour tabletop into concrete improvements, tracked until they’re proven. Evaluation is the third part of the CISM topic “incident management training, testing and evaluation”.
From exercise to improvement
- Observe: neutral observers record times, decisions and problems.
- Hot debrief immediately afterwards: what went well, and what didn’t, while memories are fresh.
- Measure against the objectives: was the RTO met? Were decisions made on time?
- Report: findings, root causes and recommendations.
- Act and retest: owners and dates; plans, playbooks and training updated; failures retested.
Readiness metrics
| Measure | Exercise 1 | Exercise 2 |
|---|---|---|
| Time to declare the incident | 95 min | 30 min |
| Time to reach the executive sponsor | 3 h | 40 min |
| Critical system restored, against an 8-hour RTO | 14 h | 7 h |
| Decisions made without clear authority | 4 | 1 |
Tracking the same measures across exercises shows management whether readiness is actually improving.
A blame-free culture
- Focus on process: why did the process let this happen?
- Encourage honesty: people admit confusion only when it’s safe to.
- Celebrate findings: a gap found in an exercise is a win, not a failure.
- Share results with management, together with the action plan.
Example. In a simulation, the backup administrator couldn’t find the decryption keys for the offsite backups. Rather than blame the administrator, the review found the keys were stored only on the server being restored. The keys moved to an offline vault, and the retest a month later restored within the RTO.
How the exam asks about it
| If the question asks… | The answer is usually… |
|---|---|
| Most important outcome of an exercise | Identify weaknesses and improve the plan |
| What follows fixing the findings | Retest to verify |
| Who should observe exercises | Neutral observers |
| How to show readiness is improving | Metrics across successive exercises |
| A test finds no issues | Question whether it was realistic |
In short
- The value of an exercise lies in what it reveals and what changes afterwards.
- Neutral observers record times, decisions and problems against the pre-agreed objectives.
- A hot debrief right after the exercise captures fresh observations; a written report follows.
- Findings get root causes, owners, dates and retests, tracked like any other risk.
- Readiness metrics (time to detect, escalate, contain, recover) show improvement over successive exercises.
- Results are reported to management, and plans, playbooks and training are updated.
Exam traps
- Blaming individuals for exercise failures discourages honest participation; focus on process.
- Fixing findings without retesting leaves the plan unproven.
- Exercises with easy, unrealistic scenarios create false confidence.
- Results not reported to management can't drive investment.
Slides
Click the slide and use the arrow keys. F toggles full screen.
From exercise to improvement
1 / 8