Enforcing Public Block Access on S3 Bucket automatically

Overview

Here is a quick guide on how to implement a lambda function which would scan all of your S3 Buckets and enforce Public Block Access ACLs automatically if they’re missing. Once completed, an email would be triggered to indicate which S3 Bucket was modified. This configuration is using Terraform 0.12 version to deploy this project.

This documentation is based on this following GitHub repository, any questions, don’t hesitate to raise it.

I also keep my blog updated with new articles with my consulting company, you can check at https://cdcloudlogix.com/blog for more information :)

cdcloudlogix/terraform_s3_public_notifications

I. Terraform Providers

Start by configuring your main AWS account as a Terraform provider as follow:

Once completed, add your Terraform backend:

Don’t forget to modify the S3 bucket name as well as the key name based on your situation.

II. Modules

This project is using a unique module:

• s3-scan-public

This module would install the following resources:

• IAM roles and policies
• CloudWatch Events
• Lambda
• SNS Notification

III. Lambda function

This project is using a bash script for installing / testing and zipping our lambda function:

Before applying Terraform plan, the zip file would have to be present on the local file system.

Prepare a virtual environment for python before running this on your terminal:

$ virtualenv -p python3 venv
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/oli/Documents/venv/bin/python3.7
Also creating executable in /Users/oli/Documents/venv/bin/python
Installing setuptools, pip, wheel...
done.$ source venv/bin/activate

IV. SSM Parameters

To avoid exposing email addresses in a GitHub repository, this project is making the use of the following SSM parameters:

• s3-public-exception-list: List of S3 Buckets excluded from Scan(StringList type)
• default-source-email: Default name used as a source (String type)
• default-destination-emails: List of email recipients (StringList type)

Regarding the configuration of each of these SSM parameters, have a look below on these screenshots:

SSM Parameters configuration

V. Deploy

Package your lambda script first (Truncated output):

21:12 $ ./python_packages.sh
Requirement already satisfied: pytest in /Users/oli/Documents/terraform_s3_public_notifications/venv/lib/python3.7/site-packages 
[...]
~/Documents/terraform_s3_public_notifications/modules/s3-scan-public ~/Documents/terraform_s3_public_notifications/modules
-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 9.22/10, +0.78)
~/Documents/terraform_s3_public_notifications/modules
~/Documents/terraform_s3_public_notifications/modules/s3-scan-public ~/Documents/terraform_s3_public_notifications/modules
  adding: s3_public.py (deflated 70%)
~/Documents/terraform_s3_public_notifications/modules

Then execute terraform init command (Truncated output):

21:13 $ terraform init
Initializing modules...
Initializing the backend...
Initializing provider plugins...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
[...]

Finally, apply Terraform (Truncated output):

(venv) ✔ ~/Documents/terraform_s3_public_notifications [master|✔]
21:51 $ terraform apply
module.s3-public-lambda.data.aws_ssm_parameter.s3_public_emails: Refreshing state...
module.s3-public-lambda.data.aws_caller_identity.current: Refreshing state...
module.s3-public-lambda.data.aws_ssm_parameter.display_name: Refreshing state...
module.s3-public-lambda.data.template_file.cloudformation_sns_stack: Refreshing state...
[...]
Plan: 15 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
Enter a value: yes
module.s3-public-lambda.aws_iam_policy.s3_public_log_policy: Creating...
module.s3-public-lambda.aws_cloudwatch_event_rule.schedule: Creating...
module.s3-public-lambda.aws_iam_policy.access_ssm_policy: Creating...
module.s3-public-lambda.aws_iam_policy.access_s3_policy: Creating...
module.s3-public-lambda.aws_iam_role.lambda_s3_public_role: Creating...
module.s3-public-lambda.aws_cloudformation_stack.sns_topic: Creating...
[...]

If you’re happy with this configuration, just confirm by typing yes

After deploying, each recipient would receive the following email from AWS, don’t forget to confirm that you wish to subscribe:

Confirmation email of subscription

VI. Details of this repository

The following lambda function will:

• List all your existing buckets
• Compare this list with the exempt list of S3 Buckets present in your SSM parameter
• Check if Public Access Block ACL is applied on each of your bucket
• Apply Public Access Block ACL on your bucket if it wasn’t apply
• Notify by SNS notification

This lambda would be run from Monday to Friday each week:

Terraform doesn’t support SNS Email configuration natively, hence, the reason why we need to use a Cloudformation template for this purpose:

Combine with the following template file:

VII. Testing

You can test your lambda by using the Test button as follow:

AWS Lambda test

This function would start running and setting up the Public Access Block ACLs on each of your existing buckets. Once Completed, you should receive the following email of confirmation:

Email notification