A collection of bash shell scripts for automating various tasks with Amazon Web Services using the AWS CLI and jq.
The AWS CLI is an open source tool built on top of the AWS SDK for Python (Boto) that provides commands for interacting with AWS services.
Installing the AWS Command Line Interface
Requirements:
- Python 2 version 2.6.5+ or Python 3 version 3.3+
- macOS, Linux, or Unix
If you already have pip and a supported version of Python, you can install the AWS CLI with the following command:
$ pip install awscli --upgrade --user
For general use, the aws configure command is the fastest way to set up your AWS CLI installation.
$ aws configure
The AWS CLI will prompt you for four pieces of information. AWS Access Key ID and AWS Secret Access Key are your account credentials.
The AWS CLI supports named profiles stored in the config and credentials files. You can configure additional profiles by using aws configure with the --profile option or by adding entries to the config and credentials files.
$ aws configure --profile example
jq is a lightweight and flexible command-line JSON processor.
OS X: Use Homebrew to install jq:
$ brew install jq
Linux: jq is in the official Amazon Linux AMI, Debian and Ubuntu repositories.
Amazon Linux AMI, RHEL, CentOS:
$ sudo yum install jq
Debian/Ubuntu:
$ sudo apt-get install jq
- cloudfront-invalidation-status.sh Checks CloudFront Distributions for cache invalidation status to complete
- cloudwatch-create-alarms.sh Create AWS CloudWatch alarms for EC2, RDS, Load Balancer environments
- cloudwatch-logs-retention-policy.sh Set CloudWatch Logs Retention Policy for all log groups in all regions available
- ec2-associate-elastic-ip.sh Reassign a previously allocated Elastic IP to the instance which runs this script
- ec2-classic-import-network-acl.sh Import CIDR IP list to AWS EC2 Classic ACL rules and deny access
- ec2-ebs-create-snapshots.sh Create a snapshot of each EC2 EBS volume that is tagged with the backup flag
- ec2-ebs-delete-snapshots.sh Deletes snapshots for each EC2 EBS volume that is tagged with the backup flag and matches the specified date
- ec2-elb-export-template.sh Export an ELB to a JSON template file for version control, duplication or recreation
- ec2-elb-upload-ssl-cert.sh Upload an SSL Certificate to AWS for use in setting up an ELB
- elastic-beanstalk-set-hostname-within-instance.sh Set the hostname on Elastic Beanstalk servers from within the instance with their EB environment name and public IP address
- elastic-beanstalk-update-hostnames.sh Updates the hostname on Elastic Beanstalk servers with their environment name and IP address
- iam-create-s3-users.sh Create the S3 IAM user, generate IAM keys, add to IAM group, generate user policy
- route53-export-zones.sh Uses cli53 to export the zone file for each Hosted Zone domain in Route 53
- s3-buckets-local-backup.sh Backup all contents of all S3 buckets in AWS account locally
- s3-buckets-security-audit.sh Export S3 bucket ACL, CORS, Policy and Website as JSON for auditing security of all buckets
- s3-buckets-total-file-size.sh Count total size of all data stored in all S3 buckets using s3api
- s3-fix-content-type-metadata.sh Safely fix invalid content-type metadata on AWS S3 bucket website assets for some common filetypes
- s3-openbucketpolicy.sh Set an S3 bucket policy to allow GetObject requests from any IP address
- s3-remove-glacier-objects.sh Delete all Glacier storage type objects in a single S3 bucket
- s3-restrictbucketpolicy.sh Set an S3 bucket policy to only allow GetObject requests from an IP whitelist file named iplist
- s3-setup-buckets.sh Create S3 buckets, set CORS config and tag bucket with client name
- vpc-eni-monitor.sh Generate an HTML page to monitor the number of AWS VPC Elastic Network Interfaces currently in use and upload it to an S3 bucket website
- vpc-sg-import-rules.sh Create an AWS VPC Security Group with rules to allow access to each IP at the port specified
- vpc-sg-import-rules-cloudflare.sh Create VPC Security Group with Cloudflare IP ranges
- vpc-sg-import-rules-cloudfront.sh Create VPC Security Group with CloudFront IP ranges
- vpc-sg-import-rules-pingdom.sh Create VPC Security Group with Pingdom probe server IP ranges
- vpc-sg-update-rules-pingdom.sh Update existing AWS VPC Security Groups with new IP rules to allow access to each Pingdom probe server IP at the port specified
- waf-export-ip-sets.sh Export each AWS WAF IP set match condition to a JSON file for backup
- waf-import-ip-set-facebook.sh Import list of current Facebook crawl server IPs into AWS WAF IP Set - work in progress, currently not possible to execute
- waf-web-acl-pingdom.sh Manage WAF Web ACL to allow current Pingdom probe server IPs by creating or updating AWS WAF IP Addresses Set, Rules and Web ACLs
- convert-iplist-cidr-json-array.sh Converts an IPv4 iplist to CIDR block notation and JSON array format, sorting and de-duplicating IPs
- install-awscli.sh Install and configure AWS CLI
- install-s3cmd.sh Install and setup s3cmd from the GitHub Repo
- terraform-redact-iam-secrets.sh Replaces AWS IAM Secret Keys and IAM SES SMTP Passwords with "REDACTED" in Terraform state files
Have a bug or a feature request? The issue tracker is the preferred channel for bug reports, feature requests and submitting pull requests. If your problem or idea is not addressed yet, please open a new issue.
Shawn Woodford
Code and Documentation Copyright 2012-2018 Shawn Woodford. Code released under the Apache License 2.0.