-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
14 changed files
with
375 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
## AWS Health Tools | ||
|
||
### Description | ||
The samples provided in AWS Health Tools can help you build automation and customized alerts in response to AWS Health events. | ||
|
||
AWS Health provides ongoing visibility into the state of your AWS resources, services, and accounts. The service gives you awareness and remediation guidance for resource performance or availability issues that may affect your applications that run on AWS. AWS Health provides relevant and timely information to help you manage events in progress, as well as be aware of and prepare for planned activities. The service delivers alerts and notifications triggered by changes in the health of AWS resources, so you get near-instant event visibility and guidance to help accelerate troubleshooting. | ||
|
||
More information about AWS Health and Personal Health Dashboard (PHD) is available here: http://docs.aws.amazon.com/health/latest/ug/what-is-aws-health.html | ||
|
||
Setup and usage instructions are present for each tool in its respective directory: | ||
[AWS Health event SMS notifier](sms-notifier/) | ||
[AWS Health event Amazon Simple Notification Service (SNS) Topic Publisher](sns-topic-publisher/) | ||
[AWS Health event Slack notifier](slack-notifier/) | ||
[AWS Health AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED Automated EC2 Instance stop](automated-actions/AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED/) | ||
|
||
### License | ||
AWS Health Tools are licensed under the Apache 2.0 License. | ||
|
23 changes: 23 additions & 0 deletions
23
automated-actions/AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED/IAMPolicy
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Sid": "Stmt1477516473539", | ||
"Action": [ | ||
"logs:CreateLogGroup", | ||
"logs:CreateLogStream", | ||
"logs:PutLogEvents" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "arn:aws:logs:*:*:*" | ||
}, | ||
{ | ||
"Sid": "Stmt1477680111144", | ||
"Action": [ | ||
"ec2:StopInstances" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "*" | ||
} | ||
] | ||
} |
41 changes: 41 additions & 0 deletions
41
automated-actions/AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED/LambdaFunction.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
// Sample Lambda Function to stop EC2 instances when AWS Health AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED events are generated. This is useful for situations where there is data redundancy and automated launch of instnaces (e.g. via Autoscaling). | ||
var AWS = require('aws-sdk'); | ||
|
||
// define configuration | ||
const tagKey ='stage'; | ||
const tagValue ='prod'; | ||
|
||
//main function which gets AWS Health data from Cloudwatch event | ||
exports.handler = (event, context, callback) => { | ||
//extract details from Cloudwatch event | ||
eventName = event.detail.eventTypeCode; | ||
affectedEntities = event.detail.affectedEntities; | ||
region = event.region; | ||
const awsHealthSuccessMessage = `Successfully got details from AWS Health event, ${eventName} and executed automated action.`; | ||
|
||
// setting up a loop that calls the function for the automated action for each of the resources flagged by AWS Health | ||
for ( var i=0; i < affectedEntities.length; i+=1 ) | ||
{ | ||
instanceId = affectedEntities[i].entityValue; | ||
if (affectedEntities[i].tags[[tagKey]] == tagValue){ | ||
stopInstances (instanceId, region); | ||
} | ||
else console.log ('The following instance does not match the configured tag: ', instanceId); | ||
} | ||
callback(null, awsHealthSuccessMessage); //return success | ||
}; | ||
|
||
//This function stops an EC2 Instance | ||
function stopInstances (instanceId, region) { | ||
AWS.config.update({region: region}); | ||
var ec2 = new AWS.EC2(); | ||
console.log ('attempting to stop the following instance: ', instanceId); | ||
var stopInstancesParams = { | ||
InstanceIds: [instanceId], | ||
DryRun: true | ||
}; | ||
ec2.stopInstances(stopInstancesParams, function(err, data) { | ||
if (err) console.log(instanceId, region, err, err.stack); // an error occurred | ||
else console.log("Instance stopped: ", instanceId, region); // successful response | ||
}); | ||
} |
25 changes: 25 additions & 0 deletions
25
automated-actions/AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
## AWS Health AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED | ||
|
||
### Description | ||
EC2 has detected a performance degradation of one or more physical storage drives that backs the instance store volumes of your Amazon EC2 instance. Because of this degradation, some instance store volumes could be unresponsive or exhibit poor performance. | ||
|
||
### Setup and Usage | ||
You can automatically stop or terminate EC2 instances that have degraded instance-store performance using Amazon Cloudwatch events and AWS Lambda using the following instructions: | ||
|
||
1. Create an IAM role for the Lambda function to use. Attach the [IAM policy](IAMPolicy) to the role in the IAM console. | ||
Documentation on how to create an IAM policy is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html | ||
Documentation on how to create an IAM role for Lambda is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console | ||
|
||
2. Create a Lambda JavaScript function by using the [sample](LambdaFunction.js) provided and choose the IAM role created in step 1. The sample Lambda function will stop EC2 instances when AWS Health AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED events are generated. This is useful for situations where there is data redundancy and fault tolerance (for example, when using Auto Scaling). Be sure to set the appropriate tags and region in the configuration section of the Lambda function. | ||
More information about Lambda is available here: http://docs.aws.amazon.com/lambda/latest/dg/getting-started.html | ||
|
||
3. Create a CloudWatch Events rule to trigger the Lambda function created in step 2 matching the AWS_EC2_INSTANCE_STORE_DRIVE_PERFORMANCE_DEGRADED event. | ||
Documentation on how to create an AWS Health CloudWatch Events rule is available here: http://docs.aws.amazon.com/health/latest/ug/cloudwatch-events-health.html | ||
|
||
More information about AWS Health is available here: http://docs.aws.amazon.com/health/latest/ug/what-is-aws-health.html | ||
|
||
Note that this is a just an example of how to set up automation with AWS Health, Amazon CloudWatch Events, and AWS Lambda. We recommend testing the example and tailoring it to your environment before using it in your production environment. | ||
|
||
### License | ||
AWS Health Tools are licensed under the Apache 2.0 License. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"logs:CreateLogGroup", | ||
"logs:CreateLogStream", | ||
"logs:PutLogEvents" | ||
], | ||
"Resource": "*" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
''' | ||
This is a sample function to send AWS Health event messages to a Slack channel. | ||
Follow these steps to configure the webhook in Slack: | ||
1. Navigate to https://<your-team-domain>.slack.com/apps | ||
2. Search for and select "Incoming WebHooks". | ||
3. Select "Add Configuration" and choose the default channel where messages will be sent. Then click "Add Incoming WebHooks Integration". | ||
4. Copy the webhook URL from the setup instructions and use it in the configuration section bellow | ||
You can also use KMS to encrypt the webhook URL as shown here: https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/ | ||
''' | ||
|
||
from __future__ import print_function | ||
|
||
import boto3 | ||
import json | ||
import logging | ||
import os | ||
|
||
from urllib2 import Request, urlopen, URLError, HTTPError | ||
|
||
#configuration | ||
|
||
# The Slack channel to send a message to stored in the slackChannel environment variable | ||
SLACK_CHANNEL = "#awshealth" | ||
# Add the webhook URL from Slack below | ||
HOOK_URL = "https://hooks.slack.com/services/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" | ||
# Setting up logging | ||
logger = logging.getLogger() | ||
logger.setLevel(logging.INFO) | ||
|
||
#main function | ||
|
||
def lambda_handler(event, context): | ||
message = str(event['detail']['eventDescription'][0]['latestDescription'] + "\n\n<https://phd.aws.amazon.com/phd/home?region=us-east-1#/event-log?eventID=" + event['detail']['eventArn'] + "|Click here> for details.") | ||
json.dumps(message) | ||
slack_message = { | ||
'channel': SLACK_CHANNEL, | ||
'text': message | ||
} | ||
logger.info(str(slack_message)) | ||
|
||
req = Request(HOOK_URL, json.dumps(slack_message)) | ||
try: | ||
response = urlopen(req) | ||
response.read() | ||
logger.info("Message posted to %s", slack_message['channel']) | ||
except HTTPError as e: | ||
logger.error("Request failed: %d %s", e.code, e.reason) | ||
except URLError as e: | ||
logger.error("Server connection failed: %s", e.reason) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
## AWS Health Slack Notifier | ||
|
||
### Description | ||
|
||
This tool can be used to post alerts to a Slack channel when AWS Health events are generated by using AWS Lambda and Amazon CloudWatch Events. | ||
|
||
### Slack Setup | ||
Follow these steps to configure the webhook in Slack: | ||
|
||
1. Navigate to https://<your-team-domain>.slack.com/apps | ||
|
||
2. Search for and select "Incoming WebHooks". | ||
|
||
3. Select "Add Configuration" and choose the default channel where messages will be sent. Then click "Add Incoming WebHooks Integration". | ||
|
||
4. Copy the webhook URL from the setup instructions and use it in the AWS Setup section that follows. | ||
|
||
### AWS Setup | ||
|
||
1. Create an IAM role for the Lambda function to use. Attach the [IAM policy](IAMPolicy) to the role in the IAM console. | ||
Documentation on how to create an IAM policy is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html | ||
Documentation on how to create an IAM role for Lambda is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console | ||
|
||
2. Create a Lambda Python function by using the [sample](LambdaFunction.py) provided and choose the IAM role created in step 1. Update the configuration section of the Lambda function with webhook URL from the Slack setup above and update the Slack channel that you want AWS Health messages posted in. | ||
More information about Lambda is available here: http://docs.aws.amazon.com/lambda/latest/dg/getting-started.html | ||
More information about Slack integration with Lambda is available here: https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/ | ||
|
||
3. Create a CloudWatch Events rule to trigger the Lambda function created in step 2 for AWS Health events. | ||
Documentation on how to create an AWS Health CloudWatch Events rule is available here: http://docs.aws.amazon.com/health/latest/ug/cloudwatch-events-health.html | ||
|
||
More information about AWS Health is available here: http://docs.aws.amazon.com/health/latest/ug/what-is-aws-health.html | ||
|
||
Note that this is a just an example of how to set up automation with AWS Health, Amazon CloudWatch Events, and AWS Lambda. We recommend testing the example and tailoring it to your environment before using it in your production environment. | ||
|
||
### License | ||
AWS Health Tools are licensed under the Apache 2.0 License. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Sid": "Stmt1477516473539", | ||
"Action": [ | ||
"logs:CreateLogGroup", | ||
"logs:CreateLogStream", | ||
"logs:PutLogEvents" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "arn:aws:logs:*:*:*" | ||
}, | ||
{ | ||
"Sid": "Stmt1484080345748", | ||
"Action": [ | ||
"sns:Publish" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "*" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
// Sample Lambda Function to send notifications via text when an AWS Health event happens | ||
var AWS = require('aws-sdk'); | ||
var sns = new AWS.SNS(); | ||
|
||
// define configuration | ||
const phoneNumber ='+16073397201'; //For example, a U.S. phone number in E.164 format would appear as +1XXX5550100 | ||
|
||
//main function which gets AWS Health data from Cloudwatch event | ||
exports.handler = (event, context, callback) => { | ||
//extract details from Cloudwatch event | ||
eventName = event.detail.eventTypeCode | ||
healthMessage = 'The following AWS Health event type has occured: ' + eventName + ' For more details, please see https://phd.aws.amazon.com/phd/home?region=us-east-1#/dashboard/open-issues'; | ||
//prepare message for SNS to publish | ||
var snsPublishParams = { | ||
Message: healthMessage, | ||
PhoneNumber: phoneNumber, | ||
}; | ||
sns.publish(snsPublishParams, function(err, data) { | ||
if (err) { | ||
const snsPublishErrorMessage = `Error publishing AWS Health event to SNS`; | ||
console.log(snsPublishErrorMessage, err); | ||
callback(snsPublishErrorMessage); | ||
} | ||
else { | ||
const snsPublishSuccessMessage = `Successfully got details from AWS Health event, ${eventName} and sent SMS via SNS.`; | ||
console.log(snsPublishSuccessMessage, data); | ||
callback(null, snsPublishSuccessMessage); //return success | ||
} | ||
}); | ||
}; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
## AWS Health SMS Notifier | ||
|
||
### Description | ||
|
||
This tool can be used to send custom text or SMS notifications via Amazon SNS when an AWS Health event happens by using AWS Lambda and Amazon CloudWatch Events. | ||
|
||
### Setup and Usage | ||
|
||
1. Create an IAM role for the Lambda function to use. Attach the [IAM policy](IAMPolicy) to the role in the IAM console. | ||
Documentation on how to create an IAM policy is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html | ||
Documentation on how to create an IAM role for Lambda is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console | ||
|
||
2. Create a Lambda JavaScript function by using the [sample](LambdaFunction.js) provided and choose the IAM role created in step 1. Update the phone number in the configuration section. More information about Lambda is available here: http://docs.aws.amazon.com/lambda/latest/dg/getting-started.html | ||
|
||
3. Create a CloudWatch Events rule to trigger the Lambda function created in step 2 for AWS Health events. | ||
Documentation on how to create an AWS Health CloudWatch Events rule is available here: http://docs.aws.amazon.com/health/latest/ug/cloudwatch-events-health.html | ||
|
||
More information about AWS Health is available here: http://docs.aws.amazon.com/health/latest/ug/what-is-aws-health.html | ||
|
||
Note that this is a just an example of how to set up automation with AWS Health, Amazon CloudWatch Events, and AWS Lambda. We recommend testing this example and tailoring it to your environment before using it in your production environment. | ||
|
||
### License | ||
AWS Health Tools are licensed under the Apache 2.0 License. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Sid": "Stmt1477516473539", | ||
"Action": [ | ||
"logs:CreateLogGroup", | ||
"logs:CreateLogStream", | ||
"logs:PutLogEvents" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "arn:aws:logs:*:*:*" | ||
}, | ||
{ | ||
"Sid": "Stmt1484080345748", | ||
"Action": [ | ||
"sns:Publish" | ||
], | ||
"Effect": "Allow", | ||
"Resource": "*" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
// Sample Lambda Function to send notifications to a SNS topic when an AWS Health event happens | ||
var AWS = require('aws-sdk'); | ||
var sns = new AWS.SNS(); | ||
|
||
// define configuration | ||
const snsTopic ='arn:aws:sns:us-east-1:083010608567:Test_Topic'; //use ARN | ||
|
||
//main function which gets AWS Health data from Cloudwatch event | ||
exports.handler = (event, context, callback) => { | ||
//extract details from Cloudwatch event | ||
healthMessage = event.detail.eventDescription[0].latestDescription + ' For more details, please see https://phd.aws.amazon.com/phd/home?region=us-east-1#/dashboard/open-issues'; | ||
eventName = event.detail.eventTypeCode | ||
//prepare message for SNS to publish | ||
var snsPublishParams = { | ||
Message: healthMessage, | ||
Subject: eventName, | ||
TopicArn: snsTopic | ||
}; | ||
sns.publish(snsPublishParams, function(err, data) { | ||
if (err) { | ||
const snsPublishErrorMessage = `Error publishing AWS Health event to SNS`; | ||
console.log(snsPublishErrorMessage, err); | ||
callback(snsPublishErrorMessage); | ||
} | ||
else { | ||
const snsPublishSuccessMessage = `Successfully got details from AWS Health event, ${eventName} and published to SNS topic.`; | ||
console.log(snsPublishSuccessMessage, data); | ||
callback(null, snsPublishSuccessMessage); //return success | ||
} | ||
}); | ||
}; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
## AWS Health Amazon Simple Notification Service (SNS) Topic Publisher | ||
|
||
### Description | ||
This tool can be used to send custom notifications to a SNS topic when an AWS Health event happens by using AWS Lambda and Amazon CloudWatch Events. SNS topic subscribers (for example, web servers, email addresses, Amazon SQS queues, or AWS Lambda functions) can consume or receive the message or notification over one of the supported protocols (Amazon SQS, HTTP/S, email, SMS, Lambda) when they are subscribed to the topic. More information about SNS is available here: http://docs.aws.amazon.com/sns/latest/dg/welcome.html | ||
|
||
### Setup and Usage | ||
|
||
1. Create an IAM role for the Lambda function to use. Attach the [IAM policy](IAMPolicy) to the role in the IAM console. | ||
Documentation on how to create an IAM policy is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html | ||
Documentation on how to create an IAM role for Lambda is available here: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console | ||
|
||
2. Create a Lambda JavaScript function by using the [sample](LambdaFunction.js) provided and choose the IAM role created in step 1. Update the configuration section of the script with the SNS topic ARN. | ||
More information about Lambda is available here: http://docs.aws.amazon.com/lambda/latest/dg/getting-started.html | ||
|
||
3. Create a CloudWatch Events rule to trigger the Lambda function created in step 2 for AWS Health events. | ||
Documentation on how to create AWS Health CloudWatch Events rules is available here: http://docs.aws.amazon.com/health/latest/ug/cloudwatch-events-health.html | ||
|
||
More information about AWS Health is available here: http://docs.aws.amazon.com/health/latest/ug/what-is-aws-health.html | ||
|
||
Note that this is a just an example of how to set up automation with AWS Health, Amazon CloudWatch Events, and AWS Lambda. We recommend testing the example and tailoring it to your environment before using it in your production environment. | ||
|
||
### License | ||
AWS Health Tools are licensed under the Apache 2.0 License. | ||
|
||
|