spotinfo

Using Amazon EC2 Spot instances is an excellent way to reduce EC2 on-demand instance cost, up to 90%. Whenever you have a workload that can survive VM interruption or be suspended and resumed later on without impacting business use cases, choosing the Spot pricing model is a no-brainer choice.

You should weigh your application’s tolerance for interruption and your cost saving goals when selecting a Spot instance. The lower your interruption rate, the longer your Spot instances are likely to run.

Amazon provides an excellent web interface AWS Spot Instance Advisor to explore available Spot instances and determine spot instance pools with the least chance of interruption. You can also check the savings you get over on-demand rates. You can also check the savings you get over on-demand rates. And then, you are supposed to use these metrics for selecting appropriate Spot instances.

While the AWS Spot Instance Advisor is a valuable tool, it is not easy to use its data for scripting and automation, and some use cases require too many clicks.

That's why I created the spotinfo tool. It's an easy-to-use command-line tool (open source under Apache 2.0 License) that allows you to explore AWS Spot instances in a terminal and use the spot data it provides for scripting and automation.

Under the hood, the spotinfo is using two public data sources available from AWS:

AWS Spot Instance Advisor data feed
AWS Spot Pricing data feed

Features

The spotinfo allows you to access the same information you can see in the AWS Spot Instance Advisor, but from a command line and can be used for scripting and automation use cases. In addition, the tool provides some useful features that are not available for AWS Spot Instance Advisor web interface.

Advanced Filtering

The first feature is _advanced filtering+. You can filter spot instances by:

vCPU - minimum number of CPU cores
Memory GiB - minimum memory size
Operating system - Linux or Windows
Region - one or more AWS regions (or all AWS regions)
Savings (compared to on-demand)
Frequency of interruption
Hourly rate (in USD/hour)

When filtering by instance type, regular expressions are supported. And this can help you create advanced queries.

Spot Price Visibility

With AWS Spot Instance Advisor, you can see a discount comparing to the on-demand EC2 instance rate. But to find out, what is the actual price, you are going to pay, you must visit a different AWS Spot pricing web page and search it again for the specific instance type.

The spotinfo saves your time and can display the spot price alongside other information. You can also filter and sort by spot price if you like.

Flexible Output Formats

Working with data in a command line and accessing data from scripts and automation requires flexibility of output format. The spotinfo can return results in multiple formats: human-friendly formats, like table and plain text, and automation-friendly: json, csv, or just a saving number. Choose whatever format you need for any concrete use case.

Compare Spots across multiple AWS Regions

One annoying thing about the AWS Spot Instance Advisor, is the inability to compare EC2 spot instances across multiple AWS regions. Only a single region view is available, or you need to open multiple browser tabs and constantly switch between them to compare spot instances across multiple AWS regions.

The spotinfo can help you to compare spot instances across multiple AWS regions. All you need to do is pass a --region command-line flag, and you can use this flag more than once.

Another option is to pass a special all value (with --region=all flag) to see spot instances across all available AWS regions.

Network Resilience

While the spotinfo uses public AWS data feeds, it also embeds the same data within the tool. So, if data feed is not available, for any reason (no connectivity, service not available or other), the spotinfo still will be able to return the same result.

Data snapshoot from the both AWS data feeds is embedded into the spotinfo binary during the build.

Install

OS X with Homebrew

brew tap alexei-led/spotinfo
brew install spotinfo

Download platform binary

Download OS/platform specific binary from the Releases page and add it to the PATH.

Supported OS: OS X, Windows, Linux
Supported Platforms: Intel (amd64) and ARM (arm64)

Usage

With spotinfo command you can get a filtered and sorted list of Spot instance types as a plain text, JSON, pretty table or CSV format.

spotinfo --help
NAME:
   spotinfo - spotinfo CLI

USAGE:
   spotinfo [global options] command [command options] [arguments...]

VERSION:
   1.0.0

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --type value    EC2 instance type (can be RE2 regexp patten)
   --os value      instance operating system (windows/linux) (default: "linux")
   --region value  set one or more AWS regions, use "all" for all AWS regions (default: "us-east-1")
   --output value  format output: number|text|json|table|csv (default: "table")
   --cpu value     filter: minimal vCPU cores (default: 0)
   --memory value  filter: minimal memory GiB (default: 0)
   --price value   filter: maximum price per hour (default: 0)
   --sort value    sort results by interruption|type|savings|price|region (default: "interruption")
   --order value   sort order asc|desc (default: "asc")
   --help, -h      show help (default: false)
   --version, -v   print the version (default: false)

Data Sources

The spotinfo uses the following data sources to get updated information about AWS EC2 Spot instances:

AWS Spot Advisor JSON file, maintained/updated by AWS team
AWS Spot Pricing callback JS file, maintained/updated by AWS team

The spotinfo also includes embedded (during the build) copies of the above files, and thus can continue to work, even if there is no network connectivity, or these files are not available, for any reason.

Examples

Use Case 1

Get all Graviton2 Linux Spot instances in the AWS Oregon (us-west-2) region, with CPU cores > 8 and memory > 64gb, sorted by type, and output the result in a table format.

# run binary
spotinfo --type="^.(6g)(\S)*" --cpu=8 --memory=64 --region=us-west-2 --os=linux --output=table --sort=type

# OR run Docker image
docker run -it --rm ghcr.io/alexei-led/spotinfo --type="^.(6g)(\S)*" --cpu=8 --memory=64 --region=us-west-2 --os=linux --output=table --sort=type

Output

┌───────────────┬──────┬────────────┬────────────────────────┬───────────────────────────┬──────────┐
│ INSTANCE INFO │ VCPU │ MEMORY GIB │ SAVINGS OVER ON-DEMAND │ FREQUENCY OF INTERRUPTION │ USD/HOUR │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ c6g.12xlarge  │   48 │         96 │                    50% │ <5%                       │   0.8113 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ c6g.16xlarge  │   64 │        128 │                    50% │ <5%                       │   1.0818 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ c6g.8xlarge   │   32 │         64 │                    50% │ <5%                       │   0.5409 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ m6g.12xlarge  │   48 │        192 │                    54% │ <5%                       │   0.8519 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ m6g.16xlarge  │   64 │        256 │                    54% │ <5%                       │   1.1358 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ m6g.4xlarge   │   16 │         64 │                    54% │ <5%                       │    0.284 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ m6g.8xlarge   │   32 │        128 │                    54% │ <5%                       │   0.5679 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ m6gd.8xlarge  │   32 │        128 │                    61% │ <5%                       │   0.5679 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.12xlarge  │   48 │        384 │                    63% │ <5%                       │   0.8924 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.16xlarge  │   64 │        512 │                    63% │ <5%                       │   1.1899 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.2xlarge   │    8 │         64 │                    63% │ <5%                       │   0.1487 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.4xlarge   │   16 │        128 │                    63% │ <5%                       │   0.2975 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.8xlarge   │   32 │        256 │                    63% │ <5%                       │    0.595 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6g.metal     │   64 │        512 │                    63% │ <5%                       │   1.1899 │
├───────────────┼──────┼────────────┼────────────────────────┼───────────────────────────┼──────────┤
│ r6gd.4xlarge  │   16 │        128 │                    68% │ 15-20%                    │   0.2975 │
└───────────────┴──────┴────────────┴────────────────────────┴───────────────────────────┴──────────┘

Use Case 2

Compare m5a.xlarge Linux Spot instances across 3 AWS regions, sorted by price. Output the result in a JSON format.

spotinfo --type="m5a.xlarge" --output=json --sort=price --order=asc --region=us-west-1 --region=us-east-1 --region=ap-south-1

Output

[
  {
    "Region": "ap-south-1",
    "Instance": "m5a.xlarge",
    "Range": {
      "label": "<5%",
      "min": 0,
      "max": 5
    },
    "Savings": 50,
    "Info": {
      "cores": 4,
      "emr": true,
      "ram_gb": 16
    },
    "Price": 0.0554
  },
  {
    "Region": "us-west-1",
    "Instance": "m5a.xlarge",
    "Range": {
      "label": "<5%",
      "min": 0,
      "max": 5
    },
    "Savings": 65,
    "Info": {
      "cores": 4,
      "emr": true,
      "ram_gb": 16
    },
    "Price": 0.0715
  },
  {
    "Region": "us-east-1",
    "Instance": "m5a.xlarge",
    "Range": {
      "label": "<5%",
      "min": 0,
      "max": 5
    },
    "Savings": 56,
    "Info": {
      "cores": 4,
      "emr": true,
      "ram_gb": 16
    },
    "Price": 0.0759
  }
]

Docker Image

The spotinfo uses Docker both as a CI tool and for releasing the final spotinfo Multi-Architecture Docker image (scratch with updated ca-credentials package).

Public Docker Image ghcr.io/alexei-led/spotinfo

docker pull ghcr.io/alexei-led/spotinfo:latest

Build Instructions

Makefile

The spotinfo Makefile is used for task automation only: compile, lint, test, etc. The project requires Go version 1.16+.

> make help
all              Build program binary
check_deps       Verify the system has all dependencies installed
test-bench       Run benchmarks
test-short       Run only short tests
test-verbose     Run tests in verbose mode with coverage reporting
test-race        Run tests with race detector
check test tests Run tests
test-xml         Run tests with xUnit output
test-coverage    Run coverage tests
lint             Run golangci-lint
mockgen          Run mockery to re/generate mocks for all interfaces
fmt              Run gofmt on all source files
clean            Cleanup everything

Continuous Integration

The GitHub action docker is used for the spotinfo CI.

Build with Docker

Use Docker buildx plugin to build multi-architecture Docker image.

docker buildx build --platform=linux/arm64,linux/amd64 -t spotinfo -f Dockerfile .

Required GitHub secrets

Please specify the following GitHub secrets:

DOCKER_USERNAME - Docker Registry username
DOCKER_PASSWORD - Docker Registry password or token
CR_PAT - Current GitHub Personal Access Token (with write/read packages permission)
DOCKER_REGISTRY - optional; Docker Registry name, default to docker.io
DOCKER_REPOSITORY - optional; Docker image repository name, default to $GITHUB_REPOSITORY (i.e. user/repo)

Additional secret to create GitHub Release:

RELEASE_TOKEN - GitHub Personal Access Token (with repo scope)

Name		Name	Last commit message	Last commit date
Latest commit History 30 Commits
.github/workflows		.github/workflows
cmd		cmd
public/spot		public/spot
.dockerignore		.dockerignore
.gitignore		.gitignore
.golangci.yaml		.golangci.yaml
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
go.mod		go.mod
go.sum		go.sum

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

spotinfo

Features

Advanced Filtering

Spot Price Visibility

Flexible Output Formats

Compare Spots across multiple AWS Regions

Network Resilience

Install

OS X with Homebrew

Download platform binary

Usage

Data Sources

Examples

Use Case 1

Output

Use Case 2

Output

Docker Image

Build Instructions

Makefile

Continuous Integration

Build with Docker

Required GitHub secrets

About

Releases

Packages

Languages

License

ik-cloud/spotinfo

Folders and files

Latest commit

History

Repository files navigation

spotinfo

Features

Advanced Filtering

Spot Price Visibility

Flexible Output Formats

Compare Spots across multiple AWS Regions

Network Resilience

Install

OS X with Homebrew

Download platform binary

Usage

Data Sources

Examples

Use Case 1

Output

Use Case 2

Output

Docker Image

Build Instructions

Makefile

Continuous Integration

Build with Docker

Required GitHub secrets

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages