Skip to content

Commit

Permalink
Update READMEs
Browse files Browse the repository at this point in the history
  • Loading branch information
kyhau committed Mar 19, 2022
1 parent c6971ae commit d205598
Show file tree
Hide file tree
Showing 102 changed files with 457 additions and 321 deletions.
26 changes: 13 additions & 13 deletions .aliases
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ alias aws-ddb-enable-contributor-insights='aws dynamodb update-contributor-insig
alias aws-ddb-search='python ${REPO_HOME}/DynamoDB/search_dynamodb.py'
alias aws-ddb-start-local='. ${REPO_HOME}/DynamoDB/start_dynamodb_local.sh'

alias aws-ec2-instances='python ${REPO_HOME}/EC2-VPC/list_ec2_instances.py'
alias aws-ec2-check-userdata='python ${REPO_HOME}/EC2-VPC/UserData/check_UserData.py'
alias aws-ec2-create-key-pair='. ${REPO_HOME}/EC2-VPC/create_key_pair.sh'
alias aws-ec2-instances='python ${REPO_HOME}/EC2/list_ec2_instances.py'
alias aws-ec2-check-userdata='python ${REPO_HOME}/EC2/UserData/check_UserData.py'
alias aws-ec2-create-key-pair='. ${REPO_HOME}/EC2/create_key_pair.sh'
alias aws-ec2-db-instances-based-on-ports='python ${REPO_HOME}/RDS/list_ec2_db_instances_based_on_ports.py'
alias aws-ec2-unreachable='python ${REPO_HOME}/EC2-VPC/list_ec2_unreachable.py'
alias aws-ec2-security-groups='python ${REPO_HOME}/EC2-VPC/list_security_groups.py'
alias aws-ec2-unreachable='python ${REPO_HOME}/EC2/list_ec2_unreachable.py'
alias aws-ec2-security-groups='python ${REPO_HOME}/VPC/list_security_groups.py'

alias aws-ecr-list-used-images-having-critical-or-high-severity-findings='python ${REPO_HOME}/ECR/list_used_images_having_critical_or_high_severity_findings.py'
alias aws-ecr-get-login='aws ecr get-login --no-include-email'
Expand All @@ -64,7 +64,7 @@ alias aws-iam-lookup-identities='python ${REPO_HOME}/Security/lookup_iam_identit
alias aws-inspector-cvelist-ap-southeast-2='curl https://s3.ap-southeast-2.amazonaws.com/rules-engine.ap-southeast-2/CVEList.txt'
alias aws-inspector-list-results='python ${REPO_HOME}/Inspector/inspector_helper.py'

alias aws-ip-lookup='python ${REPO_HOME}/EC2-VPC/find_subnet_for_ip.py'
alias aws-ip-lookup='python ${REPO_HOME}/VPC/find_subnet_for_ip.py'
alias aws-ip-ranges='curl https://ip-ranges.amazonaws.com/ip-ranges.json'
alias aws-ip-ranges-link='echo https://ip-ranges.amazonaws.com/ip-ranges.json'

Expand Down Expand Up @@ -109,20 +109,20 @@ alias aws-sts-get-caller-identity='aws sts get-caller-identity'
alias aws-ssm-parameters-list-global-infrastructure='. ${REPO_HOME}/SSM/ssm_parameters_list.sh'
alias aws-ssm-rdp='aws ssm start-session --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["3389"],"localPortNumber"=["9999"] --target '

alias aws-subnet-ip-usage='python ${REPO_HOME}/EC2-VPC/available_ip_address_count.py'
alias aws-subnet-ip-usage='python ${REPO_HOME}/VPC/available_ip_address_count.py'

alias aws-tag-list='python ${REPO_HOME}/_multi_services/list_tags.py'

alias aws-trustedadvisor-list-result='python ${REPO_HOME}/TrustedAdvisor/trusted_advisor_check_result.py'

alias aws-vpc-flow-logs='python ${REPO_HOME}/EC2-VPC/vpc_flow_logs.py'
alias aws-vpcs='python ${REPO_HOME}/EC2-VPC/list_vpcs.py'
alias aws-vpc-flow-logs='python ${REPO_HOME}/VPC/vpc_flow_logs.py'
alias aws-vpcs='python ${REPO_HOME}/VPC/list_vpcs.py'

alias aws-vpc-endpoints='python ${REPO_HOME}/EC2-VPC-Endpoint/vpc_endpoints.py'
alias aws-vpc-endpoint-gateway-prefix-list='python ${REPO_HOME}/EC2-VPC-Endpoint/gateway_vpc_endpoint_prefix_lists.py'
alias aws-vpc-endpoint-services='python ${REPO_HOME}/EC2-VPC-Endpoint/vpc_endpoint_services.py'
alias aws-vpc-endpoints='python ${REPO_HOME}/VPC-Endpoint/vpc_endpoints.py'
alias aws-vpc-endpoint-gateway-prefix-list='python ${REPO_HOME}/VPC-Endpoint/gateway_vpc_endpoint_prefix_lists.py'
alias aws-vpc-endpoint-services='python ${REPO_HOME}/VPC-Endpoint/vpc_endpoint_services.py'

alias aws-vpn-connections='python ${REPO_HOME}/EC2-VPC/list_vpn_connections.py'
alias aws-vpn-connections='python ${REPO_HOME}/VPC/list_vpn_connections.py'

alias aws-workspaces-details='python ${REPO_HOME}/Workspaces/list_workspaces_details.py'
alias aws-workspace-restore-rebuild='python ${REPO_HOME}/Workspaces/workspace_restore_rebuild.py'
Expand Down
11 changes: 11 additions & 0 deletions APIGateway/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# API Gateway

## Useful Libs and Tools

| Description | Repo/Link |
| :--- | :--- |
| Amazon API Gateway CORS Configurator | [aws-samples/amazon-api-gateway-cors-configurator](https://github.com/aws-samples/amazon-api-gateway-cors-configurator) |
| [Amazon API Gateway Serverless Developer Portal](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-developer-portal.html) | [awslabs/aws-api-gateway-developer-portal](https://github.com/awslabs/aws-api-gateway-developer-portal) |


---
## Useful Articles and Blogs

1. Private API invocation between two separate AWS accounts

1. [How can I access an API Gateway private REST API in another AWS account using an interface VPC endpoint?](https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-private-cross-account-vpce/), AWS, 2021-10-11
Expand Down
7 changes: 7 additions & 0 deletions AWS-Tools/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# AWS Tools

| Description | Repo/Link |
| :--- | :--- |
| AWS Tools for Microsoft VSTS | [AWS Tools for Microsoft Visual Studio Team Services](https://docs.aws.amazon.com/vsts/latest/userguide/welcome.html) |
| AWS Tools for PowerShell | [AWS Tools for PowerShell](https://aws.amazon.com/powershell/) |
| AWS Tools and sample code provided by AWS Premium Support | [awslabs/aws-support-tools](https://github.com/awslabs/aws-support-tools) |
8 changes: 8 additions & 0 deletions Amplify/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Amplify

## Useful Libs and Tools

| Description | Repo/Link |
| :--- | :--- |
| AWS Amplify CLI | [aws-amplify/amplify-cli](https://github.com/aws-amplify/amplify-cli) |
| AWS Amplify Flutter | [aws-amplify/amplify-flutter](https://github.com/aws-amplify/amplify-flutter) |
2 changes: 1 addition & 1 deletion Useful-tools/service_mesh/README.md → AppMesh.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Service Mesh
# App Mesh

See also https://www.nginx.com/blog/what-is-a-service-mesh/
- A service mesh is a configurable, low‑latency infrastructure layer designed to handle a high volume of network‑based interprocess communication among application infrastructure services using APIs.
Expand Down
6 changes: 3 additions & 3 deletions Athena/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@

# Athena

[Querying AWS CloudTrail Logs](
https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html)

- Option 1: Creating a Table for CloudTrail Logs in the CloudTrail Console
- Option 2: Manually Creating the Table for CloudTrail Logs in Athena
- Option 1: Creating a Table for CloudTrail Logs in the CloudTrail Console
- Option 2: Manually Creating the Table for CloudTrail Logs in Athena

1 change: 1 addition & 0 deletions AutoScaling/README.md → AutoScaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@

1. AWS scaling options - AWS Auto Scaling vs. Amazon EC2 Auto Scaling vs. Auto Scaling for Other Services
- See "Q. How is AWS Auto Scaling different than the scaling capabilities for individual services?" in https://aws.amazon.com/autoscaling/faqs/
2. AWS Auto Scaling Custom Resources - [aws/aws-auto-scaling-custom-resource](https://github.com/aws/aws-auto-scaling-custom-resource)
34 changes: 23 additions & 11 deletions CDK/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,31 @@
# CDK Notes

### My small examples for demo
## My small examples for demo

- https://github.com/kyhau/cdk-examples (CDK v2 and v1)
- https://github.com/kyhau/have-a-smile (CDK v2)

### Useful links
---
## Useful Libs and Tools

- [cdk8s-team/cdk8s](https://github.com/cdk8s-team/cdk8s)
- [cdk8s-team/cdk8s-cli](https://github.com/cdk8s-team/cdk8s-cli)
- [Getting Started Guide](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html)
- [github.com/aws/aws-cdk](https://github.com/aws/aws-cdk)
- [AWS CDK API Reference](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html)
- [jsii](https://github.com/aws/jsii) allows code in any language to naturally interact with JavaScript classes.
It is the technology that enables the AWS CDK to deliver polyglot libraries from a single codebase.
| Description | Repo/Link |
| :--- | :--- |
| AWS CDK (Cloud Development Kit) | [aws/aws-cdk](https://github.com/aws/aws-cdk) |
| AWS CDK Construct Hub - Find reusable components for your cloud applications | [constructs.dev](https://constructs.dev/) |
| [AWS CDK for Kubernetes (cdk8s)](https://cdk8s.io/) | [cdk8s-team/cdk8s](https://github.com/cdk8s-team/cdk8s) |
| AWS cdk8s-cli - a command-line-interface for CDK for Kubernetes | [cdk8s-team/cdk8s-cli](https://github.com/cdk8s-team/cdk8s-cli) |
| [AWS CDK Assume Role Credential Plugin](https://aws.amazon.com/blogs/devops/cdk-credential-plugin/) | [aws-samples/cdk-assume-role-credential-plugin](https://github.com/aws-samples/cdk-assume-role-credential-plugin) |
| [AWS Solutions Constructs](https://docs.aws.amazon.com/solutions/latest/constructs/api-reference.html) | [awslabs/aws-solutions-constructs](https://github.com/awslabs/aws-solutions-constructs) |
| AWS JSII allows code in any language to naturally interact with JavaScript classes | [aws/jsii](https://github.com/aws/jsii) |
| CDK Patterns | [cdk-patterns/serverless](https://github.com/cdk-patterns/serverless) |
| Serverless Stack Toolkit (SST) (extension of AWS CDK) | [serverless-stack/serverless-stack](https://github.com/serverless-stack/serverless-stack) |
| Awesome CDK | [kolomied/awesome-cdk](https://github.com/kolomied/awesome-cdk) |
| kyhau/cdk-examples (CDK v1 and v2) | [kyhau/cdk-examples](https://github.com/kyhau/cdk-examples) |


---
## Useful Articles and Blogs

- [CDK Getting Started Guide](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html)
- [CDK API Reference](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html)
- [CDK Assume Role Credential Plugin](https://aws.amazon.com/blogs/devops/cdk-credential-plugin/)
- [aws-samples/cdk-assume-role-credential-plugin](https://github.com/aws-samples/cdk-assume-role-credential-plugin) |
- [cdk-patterns/serverless](https://github.com/cdk-patterns/serverless) |
58 changes: 39 additions & 19 deletions CloudFormation/README.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,58 @@
## Useful links

* [Understanding AWS CloudFormation !Sub Syntax](https://www.fischco.org/technica/2017/cloud-formation-sub/)
* Passing value to UserData to set EC2 env variable https://stackoverflow.com/questions/54858072/aws-cloudformation-userdata-ec2-environment-variable

## Useful tools

* [AWS CloudFormer](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html)
# CloudFormation Notes

- [Understanding AWS CloudFormation !Sub Syntax](https://www.fischco.org/technica/2017/cloud-formation-sub/)
- Passing value to UserData to set EC2 env variable https://stackoverflow.com/questions/54858072/aws-cloudformation-userdata-ec2-environment-variable

---
## Useful Libs and Tools

| Description | Repo/Link |
| :--- | :--- |
| `cfn` - AWS CloudFormation CLI | [aws-cloudformation/aws-cloudformation-cli](https://github.com/aws-cloudformation/cloudformation-cli) |
| `cfn-guard` - AWS CloudFormation Guard | [aws-cloudformation/cloudformation-guard](https://github.com/aws-cloudformation/cloudformation-guard) |
| AWS CloudFormation Handling Region parity| [aws-samples/aws-cloudformation-region-parity](https://github.com/aws-samples/aws-cloudformation-region-parity) |
| `cfn-lint` - AWS CloudFormation Linter | [aws-cloudformation/cfn-python-lint](https://github.com/aws-cloudformation/cfn-python-lint) |
| AWS CloudFormation Macros | [aws-cloudformation/aws-cloudformation-macros](https://github.com/aws-cloudformation/aws-cloudformation-macros) |
| AWS CloudFormation Registry (public/private extensions/modules) | [User Guide](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/registry.html) |
| AWS CloudFormation Resource Provider Python Plugin | [aws-cloudformation/cloudformation-cli-python-plugin](https://github.com/aws-cloudformation/cloudformation-cli-python-plugin) |
| AWS CloudFormation Resources and Projects | [aws-cloudformation/awesome-cloudformation](https://github.com/aws-cloudformation/awesome-cloudformation) |
| AWS CloudFormation Sample Templates | [awslabs/aws-cloudformation-templates](https://github.com/awslabs/aws-cloudformation-templates) |
| AWS CloudFormation Template Flip (cfn-flip) | [awslabs/aws-cfn-template-flip](https://github.com/awslabs/aws-cfn-template-flip) |
| [AWS predefined CloudWatch metric filters and alarms](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/use-cloudformation-template-to-create-cloudwatch-alarms.html) |[CloudWatch_Alarms_for_CloudTrail_API_Activity.zip](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/samples/CloudWatch_Alarms_for_CloudTrail_API_Activity.zip)|
| AWSUtility::CloudFormation::CommandRunner | [aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner](https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner) |
| CloudMapper | [duo-labs/cloudmapper](https://github.com/duo-labs/cloudmapper) |
| CloudFormer | [CloudFormer for creating templates from existing AWS resources](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html) |
| Former2 generates CloudFormation / Terraform / Troposphere templates from existing AWS resources | [iann0036/former2](https://github.com/iann0036/former2) |

---
## Useful Articles and Blogs

- [AWS CloudFormer](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html)
(beta) - creates AWS CloudFormation templates from existing AWS resources.

* [cfn-flip](https://github.com/awslabs/aws-cfn-template-flip) -
- `cfn-flip` [awslabs/aws-cfn-template-flip](https://github.com/awslabs/aws-cfn-template-flip) -
converts AWS CloudFormation templates between JSON and YAML formats, making use of the YAML format's short function
syntax where possible.
- `pip install cfn_flip`

* [cfn-format](https://github.com/awslabs/aws-cloudformation-template-formatter) -
reads in an existing AWS CloudFormation template and outputs a cleanly-formatted, easy-to-read copy of the same
template adhering to standards as used in AWS documentation.
- cfn-format [awslabs/aws-cloudformation-template-formatter](https://github.com/awslabs/aws-cloudformation-template-formatter) -
reads in an existing AWS CloudFormation template and outputs a cleanly-formatted, easy-to-read copy of the same template adhering to standards as used in AWS documentation.

* [cfn-lint](https://github.com/aws-cloudformation/cfn-python-lint) -
- `cfn-lint` [aws-cloudformation/cfn-python-lint](https://github.com/aws-cloudformation/cfn-python-lint) -
validates CloudFormation yaml/json templates against the CloudFormation spec and additional checks.
- `pip install cfn-lint`

* [cfn-nag](https://github.com/stelligent/cfn_nag) -
- [stelligent/cfn_nag](https://github.com/stelligent/cfn_nag) -
looks for patterns in CloudFormation templates that may indicate insecure infrastructure.

* [cfn-skeleton](https://github.com/awslabs/aws-cloudformation-template-builder) -
consumes the published CloudFormation specification and generates skeleton CloudFormation templates with mandatory
and optional parameters of chosen resource types pre-filled with placeholder values.
- [cfn-skeleton](https://github.com/awslabs/aws-cloudformation-template-builder) -
consumes the published CloudFormation specification and generates skeleton CloudFormation templates with mandatory and optional parameters of chosen resource types pre-filled with placeholder values.

* [sceptre](https://sceptre.cloudreach.com/) -
- [sceptre](https://sceptre.cloudreach.com/) -
manages the creation, update and deletion of stacks while providing meta commands which allow users to retrieve
information about their stacks.

* [taskcat](https://github.com/aws-quickstart/taskcat) -
- [aws-quickstart/taskcat](https://github.com/aws-quickstart/taskcat) -
tests AWS CloudFormation templates. It deploys your AWS CloudFormation template in multiple AWS Regions and
generates a report with a pass/fail grade for each region.
taskcat is implemented as a Python class that you import, instantiate, and run.
Expand Down
2 changes: 2 additions & 0 deletions CloudFront-S3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@

1. If you use an **S3 bucket as the origin**, CloudFront uses the **REST API** interface of S3 to communicate with the origin.

1. E.g. bucket-name.s3.Region.amazon.com

1. S3 REST API is more versatile, allowing the client to pass richer information like AWS Identity, thereby allowing the exchange of information that makes **OAI** possible.

2. If you use the **website endpoint as the origin**, CloudFront uses the **website URL** as the origin.
Expand Down
8 changes: 7 additions & 1 deletion CloudWatch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

- [What alerts should you have for serverless applications?](https://lumigo.io/blog/what-alerts-should-you-have-for-serverless-applications/)


| Description | Repo/Link |
| :--- | :--- |
| Amazon CloudWatch Embedded Metric Format Client Library (Python) | [awslabs/aws-embedded-metrics-python](https://github.com/awslabs/aws-embedded-metrics-python) |


## Some known Issues

1. Delay in ASG Cloudwatch Alarm issue
Expand All @@ -10,6 +16,6 @@

- **"The ALB metric delay is due to an Ingestion delay time of 3 minutes and this delay cannot be reduced at this stage"**

- > CloudWatch being a push based service, the data is pushed from the source service- ELB. Some delay in metrics is expected, which is inherent for any monitoring systems- as they depend on several variables such as delay with the service publishing the metric, propagation delays and ingestion delay within CloudWatch to name a few. I do understand that a consistent 3 or 4 minute delay for ALB metrics is on the higher side. Upon further investigation, **I found out that the ALB metric delay is due to an Ingestion delay time of 3 minutes and this delay cannot be reduced at this stage**.
- > CloudWatch being a push based service, the data is pushed from the source service- ELB. Some delay in metrics is expected, which is inherent for any monitoring systems- as they depend on several variables such as delay with the service publishing the metric, propagation delays and ingestion delay within CloudWatch to name a few. I do understand that a consistent 3 or 4 minute delay for ALB metrics is on the higher side. Upon further investigation, **I found out that the ALB metric delay is due to an Ingestion delay time of 3 minutes and this delay cannot be reduced at this stage**.
Furthermore, please kindly note that the CloudWatch OPS and internal service team are still working on this issue, however, the ETA (Estimated Time of Availability) is still unknown. I sincerely apologize for any inconvenience this has caused on your side."

15 changes: 10 additions & 5 deletions Config/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# Querying AWS resources

See also

| AWS Config Rules Development Kit (RDK) CLI | [awslabs/aws-config-rdk](https://github.com/awslabs/aws-config-rdk) |
| AWS Config Rules Development Kit (RDK) Library (Python) | [awslabs/aws-config-rdklib](https://github.com/awslabs/aws-config-rdklib) |


See also
- [Querying AWS resources](
https://docs.aws.amazon.com/config/latest/developerguide/querying-AWS-resources.html)
- [Resource Types supported](
Expand All @@ -11,7 +16,7 @@ CLI
aws configservice select-resource-config --expression "$(cat ec2.sql)"
```

The SQL SELECT query components are as follows.
The SQL SELECT query components are as follows.
```
SELECT property [, ...]
[ WHERE condition ]
Expand All @@ -25,9 +30,9 @@ SELECT resourceId WHERE resourceType='AWS::EC2::Instance'
```

```
SELECT configuration.complianceType, COUNT(*)
WHERE resourceType = 'AWS::Config::ResourceCompliance'
GROUP BY configuration.complianceType
SELECT configuration.complianceType, COUNT(*)
WHERE resourceType = 'AWS::Config::ResourceCompliance'
GROUP BY configuration.complianceType
```

```
Expand Down
5 changes: 5 additions & 0 deletions DynamoDB/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# DynamoDB

| Description | Repo/Link |
| :--- | :--- |
| AWS DynamoDB Encryption Client (Python) | [aws/aws-dynamodb-encryption-python](https://github.com/aws/aws-dynamodb-encryption-python) |
| Amazon DynamoDB NoSQL Workbench | [NoSQL Workbench for Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html) |

- [On-demand DynamoDB table is being throttled](https://aws.amazon.com/premiumsupport/knowledge-center/on-demand-table-throttling-dynamodb/)

## NoSQL Workbench for DynamoDB
Expand Down
Loading

0 comments on commit d205598

Please sign in to comment.