An After-Action Review is a formal, or informal, review of recent past actions. Sometimes called a post-mortem, although we prefer the term After-Action Review as post-mortem stirs up all sorts of unproductive conversations. Retrospective is another great reference term, especially when you are working with a B2B SaaS Agile development team.
Whatever the name, the objective is to capture and act on lessons learned. It is important to note in the above definition the use of the word “recent.” Reviews are most valuable when performed as soon as possible after the action being reviewed.
Formal or Informal?
The review can be either formal or informal. If it is a review after a training session, or a routine technology change management process, it is common for incident teams to opt for an informal review.
If the review is at the conclusion of a tabletop drill, event, or real-life incident, it is appropriate to make the review formal.
Whether formal or informal, a written report is created and archived for future reference. Current and future incident team members can reference the knowledge base of past lessons learned. This reference activity can become a valuable part of an annual security program review. It will also assist in ensuring all action steps from the lessons learned are effectively implemented.
Three main sections of an After-Action Review
- What went well?
- What can we do better?
- Action items
In the most basic sense, the After-Action Review is designed to determine what worked during the event, what didn’t work, and what needs to be done differently next time.
Dos and Don’ts for Facilitators
DO keep the team engaged and on topic. It is easy to meander down unproductive paths when looking back at an activity. Strong facilitation is key to capturing a useful set of lessons learned.
DON’T let it become personal. Lessons learned are good for the business, the team, and each of the individuals. Reinforce the benefits gained from the session.
DO elicit feedback from different perspectives. Frequently the participants least involved in the event being reviewed will have the most innovative and valuable insights.
DON’T forget to assign owners and timelines to action items. A lack of ownership and accountability is often the downfall to an otherwise effective incident response team.
Real-life examples of B2B SaaS lessons learned
SCENARIO: During a global DNS outage, not only was the SaaS product site down, but so was the CRM. The normal email process could not be used to notify customers.
ACTION: Create a routine feed from the CRM to an always available non-SaaS location and establish alternate customer communication channels.
SCENARIO: While drilling the incident response plan, it was determined it would be more expedient to have all customer communications pre-written and approved by legal in advance of an incident.
ACTION: Draft and have legal counsel review several communications based on possible scenarios such as unscheduled downtime, DNS failure, data corruption, ransomware attack, and potential security breach.
SCENARIO: When the cyber insurance company was contacted during a routine tabletop drill, it was realized the specific requirements for customers’ credit monitoring services was not readily available.
ACTION: Review customer agreements and interview key customers to gather requirements and document in security communications database for reference.
SCENARIO: While reviewing the Incident Response team contact information after a recent event, two of the backup team members contact information had changed, but the Incident Response Plan had not been updated.
ACTION: Assign accountability to incident response team leader to work with human resources to create a connection so all contact data is available and up to date.
SCENARIO: When contacting the cyber insurance carrier after an incident, it was determined (too late) that a mandatory breach notification reporting window had been missed, which jeopardized the claim status.
ACTION: Review cyber policy and interview insurance carrier to create a data gathering checklist and timetable based on incident type.
Photo credit to Kenny Eliason.
If you need help thinking through this or other leadership challenges, let’s have a discussion to see if I can help in some way.