Picture it: the company boardroom, two weeks ago: Due to “an uncertain economic outlook,” the expanded security budget and new hires for 2023 you asked for have been denied. As the company “tightens the belt,” you may even lose existing budget and some headcount.
You had plans to use those resources to help you shore up security weaknesses and react more adroitly to changes and new environments that seem to appear like clockwork with every two-week development sprint.
So, what’s a CISO to do? Telling your engineers to work harder won’t really cut it. You need a way to complete work more efficiently and do more with less. You need automation. Everyone’s first reaction is that automation doesn’t work everywhere — and that’s correct — but there are plenty of cases where it works.
Identifying What to Automate
Reach for automation when you have tasks that are repetitive — tasks that happen often enough that the time savings of automation can justify the upfront cost of development and maintenance of the solution. Things to consider:
- Frequency: How often does the task occur? Even a short, 30-second task can be a valuable target for automation if you are doing it 100 times per week.
- Standardization: Is the task standardized? Repetitive tasks that have well-documented playbooks or that your staff feel like they can do in their sleep are great candidates for automation, while tasks that require substantial human consideration are not.
- Sub-tasks: Can a portion of a task be broken off and automated? End-to-end automation isn’t always possible, but partial automation can be valuable. However, partial automations are limited by
- Opportunity cost: Does this task have external costs that should be considered? A 5-minute task that interrupts an engineer’s workflow and delays the work by 20 minutes is actually a 25-minute task. If this engineer’s time is better served building vs. completed interrupt-driven tasks, the task is a prime candidate for automation. Be sure to include these factors when you weigh the opportunity cost of automating a task.
If a task isn’t a good candidate for automation but still takes up a bunch of time, can you identify a different, but related, process to automate and reduce the number of times a task must be performed? For example, you might not be able to update vulnerable dependencies automatically in production, but flagging them during development may mean you have far fewer that make it that far.
Example: Automating PCI Scans & Reporting
Let’s consider automation in the context of vulnerability scanning, the bane of many security teams’ existence. At one employer, I needed to run weekly PCI scans of all of our infrastructure. Each week, before I could run that scan, I needed to update the asset inventory by manually compiling lists of IPs and hostnames from our cloud infrastructure providers, and then updating the target list in the scanner’s Web interface. We only did this once a week, but it took about 30 minutes each time.
Both the cloud infrastructure providers and the scanner had an API, so it was relatively simple to build automation that could authenticate to both systems and compile the relevant information. This was an automation win.
Once the vuln scanner reports were produced, we needed to review and act on the findings — another weekly task that took several hours. Because the reports were provided in XML, it was simple enough to have them machine-parsed, deduplicated, and summarized via email with new issues logged as tickets. This was an even bigger automation win.
Where to Start
As with everything in information security, getting started with security automation boils down to prioritization. You can’t possibly tackle everything at once, so identify the list of possibilities; rank them based on how much they matter, both in terms of risk and potential effort savings; and start working your way down the list.
If you’re totally new to automation, start with smaller, easier wins and advance from there.
Take, for example, a security operations scenario you encounter more frequently than you’d like: open S3 buckets. Open S3 buckets seem to happen all the time, despite AWS’ best efforts to warn against them. This can be a good candidate for automation because it is a standard process that happens with high frequency. Here is one way to accomplish this with the AWS command-line interface.
The AWS CLI command aws s3api list-buckets lists all of the available S3 buckets. From that list, take each bucket name and use the command aws s3api get-bucket-policy –bucket YOUR_BUCKET_HERE to get the bucket’s permissions.
This will output the IAM policy that is applied to the bucket. Parse the IAM policy, looking for policies that allow AllUsers (everyone on the Internet), AuthenticatedUsers (everyone with an AWS account, even people not in your organization), or buckets that simply allow the * principal. This way we generate a list of all open buckets.
You can use that same aws s3api get-bucket-policy command with the –policy parameter to upload a new policy file that doesn’t have those permissions so that you can close these gaps. Once you get familiar enough performing these tasks on the command line, you can start to script the repeatable steps into a Python or shell script. Eventually you can automatically kick off and run the entire process while you sleep or do other work.
Automating Can Present Challenges
If you are worried that your automation won’t be ready for the primetime demands of your production environment, start by automating away tasks in development and staging environments, or even sales engineering and data science environments.
Finally, if you want to automate in production, but have concerns about business impacts, focus on automating in response to recent changes pulled from a CI/CD system or something like AWS’ EventBridge. Finding that you need an exception for a newly deployed system in the first minute of its lifecycle (when it is being closely monitored by the teams that just deployed it) is much more palatable than finding it out because it broke a week ago after working for nine months and now customers are complaining.
There are costs associated with building and maintaining this automation. These costs vary depending on the tooling available to you and your team’s skillsets. Some teams prefer to custom-script everything in Python/Ruby/Perl/Bash and orchestrate it through cron jobs and modules in your CI/CD pipeline. Other teams may prefer to shift some of the maintenance costs onto a remediation vendor — limiting their direct involvement to configuring tools that interface with a SIEM/SOAR and using that to kick off low-code/no-code remediation workflows built into another tool.
The choice to use an external vendor can be a good way to lower the ongoing maintenance costs of your solution. This is especially true for interactions with APIs and services whose changes might break your tooling.
The Cumulative Effect of Automation
Each individual task might not seem to amount to much. However, over a whole team’s worth of tasks for an entire organization, the savings start to add up and the cost starts to scale much more favorably.
Automation will never replace a security team — some tasks require human interaction and human decision making. However, as businesses continue to grow and security budgets grow tighter, it can help to empower those humans to do more and be more effective and more productive.
