Problem description
Storing objects in AWS S3 public buckets could threaten the security of your company data. Public access is not granted to new AWS buckets and objects by default. However, in most cases public buckets are created by members of your engineering team occasionally or the reason lies in a lack of knowledge and experience of configuring AWS buckets properly. Therefore your critical data can become accessible to anyone possessing the link. The best way to prevent such unsecure cases is to avoid public buckets except special cases, when it is really needed.
However it is a tricky task to set up centralized control under buckets with public access settings. The AWS console provides users with an opportunity to identify these buckets by filtering, but we have great doubts that your engineers will execute this procedure on a daily basis to avoid unexpected public buckets.
A lack of notification system makes it difficult to keep the public bucket lifecycle under control.
Our recommendations will help you manage public access to Amazon S3 resources properly and ensure that all required buckets and objects have their public access blocked.
How to generate a list of bucket names which has public access in a straightforward way
Recommendation:
Find all buckets with public access using aws cli. It’s not necessary to install aws cli into your system. You can use aws cli right from aws console. Here is an instruction: https://aws.amazon.com/cloudshell/
A bucket could be marked as public in two cases:
- It has public policy configured.
- It has public acls configured.
At the same time, public access shouldn’t be blocked by a public access block, otherwise the bucket is not public.
Free cloud cost optimization. Lifetime
1. Public policy case
1.1 List all of the user’s buckets, and output the name, as text
aws s3api list-buckets --query 'Buckets[*].[Name]' --output textMore info could be found here: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/list-buckets.html
1.2 Check bucket policy status
aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output textMore info could be found here:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-bucket-policy-status.html
1.3 Check that public access doesn’t blocked by public access block configuration
aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output textMore info could be found here:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-public-access-block.html
As a result one line command:
for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text);do if [[ $(aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output text 2>/dev/null) == True ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output text 2>/dev/null) == False ]]; then echo "$bucket"; fi; done2. Public acls case
2.1 List all of the user’s buckets, and output the name, as text
See point 1.1
2.2 Check bucket acls that contains AllUsers permissions
aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output textMore info could be found here:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-bucket-acl.html
2.3 Check that public acls isn’t blocked by public access block configuration
aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output textMore info could be found here:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-public-access-block.html
As a result one line command:
for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text);do if [[ $(aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output text 2> /dev/null) ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output text 2>/dev/null) != True ]]; then echo "$bucket"; fi; doneResult
Both cases in one line:
for bucket in $(aws s3api list-buckets --query 'Buckets[*].[Name]' --output text); do if [[ $(aws s3api get-bucket-acl --bucket "$bucket" --query 'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers`]' --output text 2> /dev/null) ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicAcls' --output text 2>/dev/null) != True ]]; then echo "$bucket"; elif [[ $(aws s3api get-bucket-policy-status --bucket "$bucket" --query 'PolicyStatus.IsPublic' --output text 2>/dev/null) == True ]] && [[ $(aws s3api get-public-access-block --bucket "$bucket" --query 'PublicAccessBlockConfiguration.BlockPublicPolicy' --output text 2>/dev/null) == False ]]; then echo "$bucket"; fi; doneThis command will generate a list of bucket names which has public access.
👆🏻 Dozens of AWS volume snapshots are usually created by your engineering team on a daily basis. But some challenges may occur while deleting instances and companies continue to pay for them.
💡 Find the ways of identifying and deleting orphaned and unused snapshots to avoid wastage and achieve optimization goals → https://hystax.com/reduce-your-aws-bill-by-cleaning-orphaned-and-unused-disk-snapshots/
