Solution Spotlight – Incident Response Foundations

Overview

Cloud providers empower businesses with the ability to swiftly experiment with new technology and expand their global presence quickly. While AWS relieves the burden of managing and securing the physical infrastructure, via the AWS Shared Responsibility Model  businesses are still required to properly configure and monitor their cloud hosted applications, data and infrastructure. The cloud advantage of rapid deployment leads to an equally rapid expansion of logs, metrics and vulnerabilities presenting an ever growing challenge to security and operations staff.

Traditionally, these challenges have been addressed  through log aggregation, filtering, and the use of service agents. Open-source solutions like Elastic Stack (formerly ELK) require some level of expertise in both the query language and the management of the underlying technology. In addition, the elastic stack does not offer a native way to invoke remediation workflows. Commercial solutions like Splunk add a rich set of integrations and addons, but get exponentially more expensive as the “monitored” infrastructure grows and its logs and events along with it. This is in addition to the cost of integration plumbing commonly required to interact with commercial SaaS.

When an incident occurs the time to resolution needs to be as short as possible. In the event of an incident, swift resolution is paramount. This solution serves as a foundation for leveraging native AWS services to manage the identification and resolution of security events in the AWS cloud. By leveraging native AWS services you can take advantage of the standards included in the AWS security services while keeping the ability to use third party tools or custom finding workflows. This example solution enables these features in select US regions.

Prescriptive Guidance

Before diving into incident response, it is important to acknowledge that the steps taken in this solution are just the start of a more comprehensive solution. This solution provides a central location for the managing of security incidents leveraging AWS native services with their built in standards and AWS Organizations to simplify the configuration of AWS services for all member accounts in an AWS Organization.

Figure-01

Definitions

• Delegated Security Account – Account in an AWS Organization that is used as a single access point for all security related services and data. This account is enabled as the delegated administrator for the AWS Organization giving it the authority to enable and manger security related services in all AWS member accounts.
• Security Hub Aggregation Region– Region in an account where all SH enabled region findings are forwarded. The forwarding of these events causes a duplicate of the finding to be created in this aggregation region. The finding will still exist in its origin region.
• Finding – The core data object of Security Hub. A finding is created by built in standards or third party partner products when a security issue is discovered or a security check fails.
• Custom Action–  Security Hub resource that serves as an entry point for executing remediation functions on one or more types of finding.

Best Practices / Design Principles

While it is impossible to encompass all the design principals and best practices related to incident response, below are some of the most critical considerations:

Use AWS Organizations to organize and control enterprise accounts
Organize and manage all AWS accounts using AWS Organizations. This greatly reduces the overhead required to manage AWS services at enterprise scale and ensures that any new accounts added to an organization have required services enabled automatically. For reference, please see Vertical Relevance’s Account Foundations for an automated approach to managing AWS accounts in enterprise environments.

Use Hub Accounts
Utilize a single hub account for accessing all security related tools and findings. This gives security personnel a single pane of glass for viewing and taking actions on all incidents within the organization. This also provides a single fanout point for third party and custom tools operating off of the findings produced. In this solution the hub account will be the delegated security account.

Limit access to hub account
Only have individuals responsible for monitoring and responding to security events to have access to the central security account. The information contained within the findings may be sensitive in nature. This becomes even more important when dealing with findings that could contain or have references to PCI or HIPAA information.

Aggregate findings to a single AWS region when possible
Select a region and then aggregating all findings in an account to this single region. This provides an account level single pane of glass for all findings.

Automate remediations
Automate remediations when possible. This provides an auditable, standardized and timely way of working towards resolution of security and operations issues.

Document all workflows
Document all remediation workflows whether they are manual or automated. This helps to ensure that the steps taken to fix an issue are consistent. For non-automated workflows, it provides a blueprint for creating an automated solution.

Leverage native AWS service standards
The built in standards and checks provided by AWS security services should be leveraged whenever possible. The included checks and standards provide a baseline that can be built upon without investment of engineer time. Many natively integrate with other AWS services eliminating the need to create wrappers or manually connect two services together.

Note: At the time of writing, CloudFormation does not support the resources required to implement this solution. Specifically, AWS Organizations, Security Hub, GuardDuty and Inspector 2 lack the resource objects required to enable and configure each service and add existing member accounts. Because of this the blueprint for this solution implements a CloudFormation custom resource to fill the gaps.

Security Hub Setup and Integrations

Security Hub serves as the central repository for all incident findings for this solution.  Although there is generally no order of operations when enabling Security Hub supported AWS services, this solution enables Security Hub first.  This involves configuration changes in the organization and Security Hub Admin account.

Components

1. Security Hub – Region bound central repository for security related findings and automated remediation. Built in integration with other AWS services
2. GuardDuty – Service that can continuously monitor flow logs, cloudtrail, DNS logs, EKS audit logs and S3 data event logs
3. Inspector 2 – Service capable of inspecting AWS compute resources for vulnerabilities.

Prerequisites

• Trusted access must be enabled for the three services used by this solution.  This is done in the organization management account.  Enabling trusted access ensures the service roles for the organization exist.
  • Security Hub
  • GuardDuty
  • Inspector 2
• Determine the organization member account to use as the delegated security account.

How it works

This solution implements AWS Security Hub as a single pane of glass for security issues in an AWS Organization. The delegated security account will be the central location for all security engineers to view and take action on findings in the AWS organization. In addition this solution enables two sample integrated services, GuardDuty and Inspector 2. The enabling of Security Hub and these services involves the configuring of the services through the delegated security account and configuring them to control all existing member accounts and autojoin any new member accounts in the future.

Figure-02

The API calls used to enable this solution are made in two accounts, the organization management account and the account that is made the organizations’ delegated administrator for the three AWS security services.  Although a delegated admin account is set once for each service, the account must be set as the service administrator for each service in each region needed.  Once the delegated security account is made the administrator for a region, that service is automatically started in the delegated security account. 

Security Hub Enablement
Enabling security hub integrated services prior to enabling security hub is generally supported by AWS. Being this solution places security hub at the nexus for all incident response findings, the CloudFormation template used to deploy it makes security hub a dependency for GuardDuty and Inspector2 ensuring that it is the first service enabled.

Organization Management Account Workflow Steps

1. Once the prerequisites have been met, the delegated security account can be made the delegated administrator account for AWS Security Hub.  Making the account a delegated administrator gives it permission to manage the service for all member accounts through the roles and permissions granted it through AWS Organizations. 
2. The next step is to make the account the organization administrator for Security Hub.  This needs to be performed for each region used by member accounts.  In this example it will be the us-east-1 and us-east-2.  Once the delegated security account is made the organization administrator for a region, the Security Hub service will automatically be enabled in the delegated security account.  The remaining API calls to enable this solution are made in the delegated security account.

Delegated Security Account Workflow Steps

1. Preexisting member accounts must be added as “managed by” to the AWS service.  This is done within the delegated security account for each region in this solution.  In addition, auto enable is set for all solution services so that any new organization member accounts are controlled by the delegated security account. 
2. Once an account is set as managed, the delegated security account will enable Security Hub in the member account automatically.  The settings configured in the delegated security account for Security Hub are automatically applied to the member accounts. This includes finding aggregation.

GuardDuty and Inspector 2 Enablement

The workflow for enabling GuardDuty and Inspector at the organization level are like those used to enable Security Hub.  Both services have built in integration with Security Hub so by default events and incidents they detect will become Security Hub fundings.  As with Security Hub, both services are region bound so the enablement workflow must be run for each region needed. 

Organization Management Account Workflow Steps

1. Once the prerequisites have been met, the delegated security account can be made the delegated administrator account for AWS GuardDuty and Inspector2.  Making the account a delegated administrator gives it permission to manage the services for all member accounts similar to Security Hub. 
2. Next, the account is made the organization administrator for GuardDuty and Inspector2.  Like Security Hub, this needs to be performed for each region used by the solution.  Once the delegated security account is made the organization administrator for a region, both services will automatically be enabled in the delegated security account.  The remaining API calls to enable this solution are made in the delegated security account.

Delegated Security Account Workflow Steps

1. Preexisting member accounts must be added as “managed by” to both AWS services.  This is done within the delegated security account for each region in this solution.  In addition, auto enable is set for all solution services so that any new organization member accounts are controlled by the delegated security account. 
2. Once an account is set as managed, the delegated security account will enable GuardDuty and Inspector2 in the member account(s) automatically.

 See the links in the blueprint section below for solution templates and custom resource code that can be used to implement this solution.

Blueprint

This blueprint is composed of a CloudFormation template that is able to take the accompanying python modules and use them as CloudFormation custom resources capable of enabling and configuring Security Hub, GuardDuty and Inspector v2 for an AWS Organization.

• Python Modules – Python is used to implement the enabling and configuration of the services in this solution.  It leverages the boto3 library for all AWS API interaction and is enabled to run as a custom resource or through python at the command line.
• CloudFormation – CloudFormation is used to invoke the python code which takes care of enabling and configuring the AWS services for the solution. Using a CloudFormation stack to invoke the custom resource provides an IAC record of the services enabled for any whom have the ability to view CloudFormation stacks in the management account.
• Support Tooling – A base class and installation python script are provided to help with the deployment and implementation of additional Security Hub supported services. install.py will package the custom resource, push it to an S3 bucket, and create a new cloudformation stack based on the parameters provided. org_accounts/manager provides a base class with methods that are compatible with the methods of most Organizations enabled AWS service APIs through the boto3 library.

GitHub Repo:  Incident Response Solution GitHub

Benefits

• Automates the configuration of an entire organization to use Security Hub, GuardDuty and Inspector through a simple CloudFormation stack.
• Centralized location for security related findings and remediation across all accounts in an AWS Organization.
• Provides the foundation for implementing a comprehensive remediation stack using native AWS services.

End Result

This solution lays the groundwork for an AWS Organization to take advantage of native AWS services for the detection and management of security incidents. It supplements the available CloudFormation resources with a custom resource that can be enhanced for the enabling of additional Security Hub supported services at the organization level.