Agile in Security Operations Centers - what's your take?

[robvanos]

Our entire organisation has chosen to shift towards an agile approach. We've been doing agile (to be exact: scrum) in our security operations center for about a year now. I wanted to share some experiences and lessons learned:

1. Moving to a scrum approach is not an easy transition to make. It takes some time and some frustration among your team members. It is important to keep an open culture in which you can express concerns and frustrations and address them.
   
2. Scrum revolves around teams of a maximum of 7 people. A higher number is possible, but will make sessions longer and more complex. It may be difficult to find a way to split your team into smaller teams, while still supporting your SOC services and sharing knowledge within the teams. You should also be careful not to introduce process handovers between teams that introduce additional overhead.
   
3. Scrum is not intended for operational departments. This means you shouldn't be too strict in implementing scrum rules. In short: make agile work for you, not the other way around.
   
4. If you want to make scrum work, you will need a dedicated scrum master. This could be someone from the team. Just realise that this person will be tasked with optimizing the way in which the team works and matures in its agile approach and will have little time left for actual operational security tasks.
   
5. There's overhead in the scrum process. It will likely replace several team meetings, but introduces other meetings. This overhead does serve a purpose: to gain visibility in the team workload and make the activities of the team more transparent.
   
6. If done properly, scrum can help you accelarate the improvement of your SOC. This is because there's a clear understanding of the tasks at hand and a continuous flow of improvements (stories) that can be conducted.
   
7. Not everything is a 'story'. Don't put eveything in your backlog, just the improvements. Operational stuff, such as monitoring duties should not be in there. That's just administration for the sake of administration.
   
8. Kanban may solve some of your problems, but introduces others. There's still sessions, there's still planning and there's still a backlog. The basics are similar.
   
9. Leave enough room in your sprints (if you've adopted scrum) to do ad-hoc stuff that wasn't foreseable.
   
10. Seek to understand and optimize the amount of work a team can perform in any sprint while also responding to ad-hoc requests while having an acceptable workload.
    
11. Backlogs can be used as an excuse not to act immediately. This actually makes you less agile. While this seems to be a contradiction, it wil happen if the team has a high workload.
    
12. Clean you backlog every now and then. If a certain task wasn't given sufficient priority in the last few months, why should it get any priority in the next months?
    

Note: scrumban (a combination of scrum and kanban) is also possible. We're only just looking into this, so no lessons learned here yet.

I'm curious as to the experiences of others regarding implementing agile in security operations. What added value did or didn't it bring to your SOC?

[Amna]

Our entire organisation has chosen to shift towards an agile approach. We've been doing agile (to be exact: scrum) in our security operations center for about a year now. I wanted to share some experiences and lessons learned:

1. Moving to a scrum approach is not an easy transition to make. It takes some time and some frustration among your team members. It is important to keep an open culture in which you can express concerns and frustrations and address them.
   
2. Scrum revolves around teams of a maximum of 7 people. A higher number is possible, but will make sessions longer and more complex. It may be difficult to find a way to split your team into smaller teams, while still supporting your SOC services and sharing knowledge within the teams. You should also be careful not to introduce process handovers between teams that introduce additional overhead.
   
3. Scrum is not intended for operational departments. This means you shouldn't be too strict in implementing scrum rules. In short: make agile work for you, not the other way around.
   
4. If you want to make scrum work, you will need a dedicated scrum master. This could be someone from the team. Just realise that this person will be tasked with optimizing the way in which the team works and matures in its agile approach and will have little time left for actual operational security tasks.
   
5. There's overhead in the scrum process. It will likely replace several team meetings, but introduces other meetings. This overhead does serve a purpose: to gain visibility in the team workload and make the activities of the team more transparent.
   
6. If done properly, scrum can help you accelarate the improvement of your SOC. This is because there's a clear understanding of the tasks at hand and a continuous flow of improvements (stories) that can be conducted.
   
7. Not everything is a 'story'. Don't put eveything in your backlog, just the improvements. Operational stuff, such as monitoring duties should not be in there. That's just administration for the sake of administration.
   
8. Kanban may solve some of your problems, but introduces others. There's still sessions, there's still planning and there's still a backlog. The basics are similar.
   
9. Leave enough room in your sprints (if you've adopted scrum) to do ad-hoc stuff that wasn't foreseable.
   
10. Seek to understand and optimize the amount of work a team can perform in any sprint while also responding to ad-hoc requests while having an acceptable workload.
    
11. Backlogs can be used as an excuse not to act immediately. This actually makes you less agile. While this seems to be a contradiction, it wil happen if the team has a high workload.
    
12. Clean you backlog every now and then. If a certain task wasn't given sufficient priority in the last few months, why should it get any priority in the next months?
    

Note: scrumban (a combination of scrum and kanban) is also possible. We're only just looking into this, so no lessons learned here yet.

I'm curious as to the experiences of others regarding implementing agile in security operations. What added value did or didn't it bring to your SOC?

Hi, I am trying to figure out how to implement Agile for SoC ,  searching for it led me back here, so how did you implement it? what was the outcomes?

[robvanos]

Hi Amna,

Our agile implementation has been quite a journey, where we tried different things to see what worked. Just to give you an idea: during the agile transition, we split the team into 2 sub-teams, and have now made an even further split to further optimise the teams efficiency. We started out by using 2-week scrum sprints, then moved to 3-week sprint to reduce session overhead, introduced a weekyl refinement, moved bck to 2-week sprints because longer sprints had the negative side effect of other teams waiting to long for their requests. The latest change has been moving away from scrum and using a kanban workflow, which we our now learning and making more efficient every day.

So, there have been many changes and adaptations, and I think this is natural. You have to find what works best, and that takes some trial and error. While 2-week sprints worked fine for a longer period of time, the changes to the team also meant that a new way of working was required. One of the biggest disadvantages that we had to deal with is that, especially during refinement, we were refining stories that were relevant to a part of the team, but irrelevant to another part of the team with a different focus. So we initially made a split between everything related to vulnerabilities & assessment and security monitoring & analysis on the other hand. Then, a second division was made to differentiate further between engineering tasks and analysis tasks.

You should keep in mind that implementing agile may require changes to the team as well. We have found a way of working with Kanban that seems appropriate for now and that will be improved in the months to come. But if we encounter another problem that requires us to that the way we work or the organisation of the SOC, we will make that change and move forward from that point. In conclusion, agile should not only apply to the way the work is done, but should be a mindset to make the necessary changes no matter where.

Regards,
Rob.

[moritz]

Hi Amna,

Our agile implementation has been quite a journey, where we tried different things to see what worked. Just to give you an idea: during the agile transition, we split the team into 2 sub-teams, and have now made an even further split to further optimise the teams efficiency. We started out by using 2-week scrum sprints, then moved to 3-week sprint to reduce session overhead, introduced a weekyl refinement, moved bck to 2-week sprints because longer sprints had the negative side effect of other teams waiting to long for their requests. The latest change has been moving away from scrum and using a kanban workflow, which we our now learning and making more efficient every day.

So, there have been many changes and adaptations, and I think this is natural. You have to find what works best, and that takes some trial and error.  While 2-week sprints worked fine for a longer period of time, the changes to the team also meant that a new way of working was required. One of the biggest disadvantages that we had to deal with is that, especially during refinement, we were refining stories that were relevant to a part of the team, but irrelevant to another part of the team with a different focus. So we initially made a split between everything related to vulnerabilities & assessment and security monitoring & analysis on the other hand. Then, a second division was made to differentiate further between engineering tasks and analysis tasks.

You should keep in mind that implementing agile may require changes to the team as well. We have found a way of working with Kanban that seems appropriate for now and that will be improved in the months to come. But if we encounter another problem that requires us to that the way we work or the organisation of the SOC, we will make that change and move forward from that point. In conclusion, agile should not only apply to the way the work is done, but should be a mindset to make the necessary changes no matter where.

Regards,
Rob.

Hi Rob,

I know this post is a bit older, but I still would like to ask a question, because we, a smaller soc, are trying to get into agile work. At the moment we think kanban fits best for us, but we still would like to see and learn the possibilities scrum could offer us. You've written that you changed to a kanban based model. What were your learnings out of this? Was kanban or scrum more suitable for your soc work?

Would be very thankful if u would answer on these questions

Anyway, thanks for sharing your experience.

Regards,
Moritz.

[robvanos]

Hi Moritz,

Over the last years, I have changed quite a bit on the way I use agile in security operations. The basic principles I always use are the following:
1. Make the process work for the team, instead of working the team to match the process
2. The process itself will likely need to change a number of times.
3. Reduce the overhead as much as possible

I currently favor a weekly sprint approach for teams starting with agile. There is a weekly session to look at the backlog in a standard format, go through the previous iteration and define the new iteration. I don't do separate refinement sessions, backlog items are refined with the team during the meeting (this requires the team to be strict and avoid unnecessary discussion). Retro sessions are done periodically and not for each iteration. During daily sesssions, lack of progress an blocking issues are discussed.

So, basically, I have created my own brand of light-weight agile with reduced overhead to maximize th efficiency. The exact implementation depends on the team size, organisational culture, etc. This requires some tuning (see principe #2).

Regards,
Rob.