Whitepaper 'FinOps and cost management for Kubernetes'
OptScale is fully available as an open source solution under Apache 2.0 on GitHub
Ebook 'From FinOps to proven cloud cost management & optimization strategies'

The quickest way to get a list of public buckets in AWS to enhance your security

Problem description

Storing objects in AWS S3 public buckets could threaten the security of your company data. Public access is not granted to new AWS buckets and objects by default. However, in most cases public buckets are created by members of your engineering team occasionally or the reason lies in a lack of knowledge and experience of configuring AWS buckets properly. Therefore your critical data can become accessible to anyone possessing the link. The best way to prevent such unsecure cases is to avoid public buckets except special cases, when it is really needed. 

The_quickest_way_to_get_a_list_of_AWS_public_buckets

However it is a tricky task to set up centralized control under buckets with public access settings. The AWS console provides users with an opportunity to identify these buckets by filtering, but we have great doubts that your engineers will execute this procedure on a daily basis to avoid unexpected public buckets. 

A lack of notification system makes it difficult to keep the public bucket lifecycle under control.

Our recommendations will help you manage public access to Amazon S3 resources properly and ensure that all required buckets and objects have their public access blocked.

How to generate a list of bucket names which has public access in a straightforward way

Recommendation:

Find all buckets with public access using aws cli. It’s not necessary to install aws cli into your system. You can use aws cli right from aws console. Here is an instruction: https://aws.amazon.com/cloudshell/

A bucket could be marked as public in two cases:

  1. It has public policy configured.
  2. It has public acls configured.

At the same time, public access shouldn’t be blocked by a public access block, otherwise the bucket is not public.

Free cloud cost optimization. Lifetime

1. Public policy case

1.1 List all of the user’s buckets, and output the name, as text

aws s3api list-buckets --query 'Buckets[*].[Name]' --output text

1.2 Check bucket policy status

aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output text

1.3 Check that public access doesn’t blocked by public access block configuration

aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output text

As a result one line command:

for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text);do if [[ $(aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output text 2>/dev/null) == True ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output text 2>/dev/null) == False ]]; then echo "$bucket"; fi; done

2. Public acls case

2.1  List all of the user’s buckets, and output the name, as text

See point 1.1

2.2 Check bucket acls that contains AllUsers permissions

aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output text

2.3 Check that public acls isn’t blocked by public access block configuration

aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output text

As a result one line command:

for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text);do if [[ $(aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output text 2> /dev/null) ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output text 2>/dev/null) != True ]]; then echo "$bucket"; fi; done

Result

Both cases in one line:

for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text); do if [[ $(aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output text 2> /dev/null) ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output text 2>/dev/null) != True ]]; then echo "$bucket"; elif [[ $(aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output text 2>/dev/null) == True ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output text 2>/dev/null) == False ]]; then echo "$bucket"; fi; done

This command will generate a list of bucket names which has public access.

👆🏻 Dozens of AWS volume snapshots are usually created by your engineering team on a daily basis. But some challenges may occur while deleting instances and companies continue to pay for them. 
💡 Find the ways of identifying and deleting orphaned and unused snapshots to avoid wastage and achieve optimization goals → https://hystax.com/reduce-your-aws-bill-by-cleaning-orphaned-and-unused-disk-snapshots/

Enter your email to be notified about new and relevant content.

Thank you for joining us!

We hope you'll find it usefull

You can unsubscribe from these communications at any time. Privacy Policy

News & Reports

FinOps and MLOps

A full description of OptScale as a FinOps and MLOps open source platform to perform multi-scenario cloud cost optimization and ensure ML/AI profiling and optimization

FinOps, cloud cost optimization and security

Discover our best practices: 

  • How to release Elastic IPs on Amazon EC2
  • Detect incorrectly stopped MS Azure VMs
  • And much more deep insights

From FinOps to proven cloud cost management & optimization strategies

This ebook covers the implementation of basic FinOps principles to shed light on alternative ways of conducting cloud cost optimization