Skip to content

Commit

Permalink
rename reaper to ttl, add architecture diagram
Browse files Browse the repository at this point in the history
  • Loading branch information
EvanBoyle committed Oct 26, 2022
1 parent da481cf commit c9d95b7
Show file tree
Hide file tree
Showing 11 changed files with 222 additions and 24 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Deployment driver examples show using the Pulumi Deployment API to create deploy

- [typescript-driver](./deployment-drivers/nodejs/typescript-driver/) - a deployment driver written in Typescript. It can deploy Pulumi programs written in any language.
- [Go CLI driver](./deployment-drivers/go/cli/) - a deployment driver written in Typescript. It can deploy Pulumi programs written in any language.
- [Stack Reaper](./pulumi-programs/stack-reaper/) - infrastructure that enables temporary stacks. Set a stack tag, run an update, and the stack automatically gets destroyed by Pulumi Deployment API after the expiration period.
- [TTL Stacks](./pulumi-programs/ttl-stacks/) - infrastructure that enables temporary stacks. Set a `ttl` stack tag, run an update, and the stack automatically gets destroyed by Pulumi Deployment API after the expiration period.
- [Drift Detection](./pulumi-programs/drift-detection/) - infrastructure that detects when desired state diverges from reality. Automatically detect when manual changes are made to your infrastructure and not synced to the Pulumi program.

## Pulumi Programs
Expand Down
4 changes: 0 additions & 4 deletions pulumi-programs/stack-reaper/Pulumi.dev.yaml

This file was deleted.

File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
name: stack-reaper
name: ttl-stacks
description: A minimal AWS TypeScript Pulumi program
runtime: nodejs
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Stack Reaper
# TTL stacks

The stack reaper is a piece of infrastructure that allows you to set a stack tag that triggers automatic destroy via the Pulumi Deployment API after the expiration time passes.
The TTL stack processor is a piece of infrastructure that allows you to set a stack tag that triggers automatic destroy via the Pulumi Deployment API after the expiration time passes. This enables temporary development infrastructure.

Infrastructure waste is a common problem. It’s too easy to leave development infrastructure running accidentally and end up with a huge bill. Many companies have entire teams devoted to solving this problem.

After deploying this program you can do the following on any program in your pulumi organization:

```console
# on any stack in your organization
$ pulumi stack tag set reap 30 # minutes to wait until destroying the stack
$ pulumi stack tag set ttl 30 # minutes to wait until destroying the stack
$ pulumi up
# 30 minutes later the stack will be cleaned up via the Pulumi Deployment API
```
Expand Down Expand Up @@ -50,4 +52,11 @@ $ pulumi up

1. Create a [Pulumi webhook](https://www.pulumi.com/docs/intro/console/extensions/webhooks/). Use the output from the previous step as the `Payload URL`.

You should now be able to `pulumi stack tag set reap X && pulumi up` (X=minutes) to create stacks that destroy themselves after the specified expiry.
You should now be able to `pulumi stack tag set ttl X && pulumi up` (X=minutes) to create stacks that destroy themselves after the specified expiry.


## Architecture

This stack uses

![](./ttl-stacks-architecture.png)
Original file line number Diff line number Diff line change
Expand Up @@ -43,22 +43,22 @@ function authenticateRequest(req: awsx.apigateway.Request): awsx.apigateway.Resp
return undefined;
}

type ReaperMessage = {
type ttlMessage = {
organization: string;
project: string;
stack: string;
expiration: string;
}

// the queue for scheduling stack deletion
const queue = new aws.sqs.Queue("reaper-queue", {
const queue = new aws.sqs.Queue("ttl-queue", {
visibilityTimeoutSeconds: 181, // TODO: tighten this up as well as lambda timeout
});

// this processor looks for messages in the queue one at a time that have passed their expiry.
// if a message has not passed it's expriy, then it throws an error so the message gets retried.
// expired messages trigger destroy operations via the pulumi deployment api.
queue.onEvent("reaper-queue-processor", async (e) => {
queue.onEvent("ttl-queue-processor", async (e) => {
console.log("queue processor running");
const messagesToRetry = [];
for (let rec of e.Records) {
Expand Down Expand Up @@ -141,10 +141,10 @@ runtime: nodejs
});

/**
* the reaper webhook processes all stack updates, looks up "reap" tags, and schedules corresponding stacks for deletion
* the ttl webhook processes all stack updates, looks up "ttl" tags, and schedules corresponding stacks for deletion
* via messages in an SQS queue
*/
const webhookHandler = new awsx.apigateway.API("reaper-webhook-handler", {
const webhookHandler = new awsx.apigateway.API("ttl-webhook-handler", {
restApiArgs: {
binaryMediaTypes: ["application/json"],
},
Expand Down Expand Up @@ -201,16 +201,16 @@ const webhookHandler = new awsx.apigateway.API("reaper-webhook-handler", {
}

const stackResult = await response.json();
const reapTag = (stackResult as any)?.tags?.reap;
if (!reapTag) {
console.log(`no reap tag found for stack: ${organization}/${project}/${stack}!\n`)
const ttlTag = (stackResult as any)?.tags?.ttl;
if (!ttlTag) {
console.log(`no ttl tag found for stack: ${organization}/${project}/${stack}!\n`)
return { statusCode: 200, body: `noop for stack ${organization}/${project}/${stack}!\n` };
}

console.log(`reap tag found for stack, queueing SQS message: ${organization}/${project}/${stack}!\n`)
console.log(`ttl tag found for stack, queueing SQS message: ${organization}/${project}/${stack}!\n`)

let time = new Date();
const expirationMinutes = parseInt(reapTag) || 30;
const expirationMinutes = parseInt(ttlTag) || 30;
time = new Date(time.getTime() + 60000 * expirationMinutes);

const message = {
Expand Down
Loading

0 comments on commit c9d95b7

Please sign in to comment.