Security Incident Response Planning: 4 Lessons Learned

One of the most important, and yet easily overlooked, elements of keeping an organization secure is creating an incident recovery roadmap--also known as an incident recovery plan. As its name implies, this plan provides a course of action to be taken following a cyber security incident. Having been involved in the creation of several such plans over the years, I wanted to pass along some lessons learned.

1. Every incident is different.

One thing to keep in mind during cyber security incident response planning is that every security incident is different. You wouldn’t respond to a minor malware infection in the same way that you would respond to someone physically breaking into your data center. As such, it is important to create a list of the various types of cyber security incidents that could conceivably occur and develop a plan for coping with each one.

2. The first priority is containment.

Regardless of the type of incident that has occurred, your first priority should be containment. Whether you are dealing with malware or a hacker, you need to make sure that the attack cannot progress any further than it already has.

A containment plan’s contents will vary widely depending on the type of incident and on the systems that have been attacked. In any case, the containment plan should act as a checklist of things that should be done immediately. For example, you might change the passwords for all privileged accounts, and perform an immediate review of all accounts to see if any accounts have been recently created or have had their privileges elevated.

When developing a containment plan, detail is key. For example, don’t just include a checklist item saying that the passwords for privileged accounts need to be changed. Include a list of those accounts. Otherwise, an account might be accidentally overlooked in the heat of the moment.

As you write a containment plan, it's smart to include a contact list of everyone in the IT department, any vendors or support providers that you use, and any key stakeholders. After all, you never know when you may need help, so it is useful to have everyone’s contact information on hand. You don’t want to waste time during an emergency trying to find someone’s phone number.

 3. Containment and recovery are just the beginning.

Containment and recovery are really only the first half of an incidence response. There is an entire laundry list of things that need to be done post-recovery, and all of these should be documented as part of your security incident response planning.

First, take the time to thoroughly document both the incident and your response to the incident. This information could prove to be very useful later on if it is found that the breach was more extensive than you thought or if there is a recurrence. Don’t assume that you will be able to remember all of the details. Write it down.

Another important thing to do following the recovery process is to evaluate your backups. You will need to determine whether backups that were taken just prior to the incident are trustworthy. If those backups are deemed to be clean, then it is going to be important to preserve those backups. You may need them if you later determine that your recovery efforts were not completely successful.

Another crucially important post-recovery task is to monitor the affected system, and any other systems that could conceivably have been affected if the breach were a bit more extensive. I recommend spelling out within your incident recovery roadmap what to monitor, what specifically to look for, and what the course of action should be if any of the things that you are looking for are detected. You might even go so far as to provide instructions for which staffers should be monitoring those systems and what tools they should use.

As you plan out a monitoring strategy, it is exceedingly important to decide up front how long you will continue your monitoring efforts. Everyone tends to be on guard immediately after a security incident, but it is easy to become complacent in the weeks and months that follow. Building a time frame into your monitoring plan will keep that from happening. At the same time, there will also be a point at which you can safely assume that everything is back to normal. As such, your monitoring plan might call for extreme scrutiny for 120 days following a cyber security incident, and then normal security monitoring after the 120-day period has expired.

4. Keep the plan front of mind and up to date.

By far the best advice that I can give regarding security incident response planning, and the final lesson learned, is to keep the plan up to date and to keep everyone involved in the plan. I recommend having a monthly meeting to review the security plan in light of any new systems that have been brought online (or that have been upgraded), any old systems that have been retired or any changes in the perceived threats.

Be sure to involve the entire IT staff in these meetings. This will help everyone to remember that there is a plan that should be referred to if an incident were to occur.