An AWS CloudFormation stack to run a serverless password-protected photo gallery
Demo: https://awspics.net
Credentials: "username" / "password"
Host a self-contained, declarative infrastructure, password-protected, data-driven static photo gallery to share personal pictures with friends and family, without needing to run, maintain (or pay for) servers.
There are 7 main components:
- CloudFront with restricted bucket access to prevent unauthenticated access to the site or its pictures.
- Login lambda function to validate authentication and sign cookies to allow access to restricted buckets.
- Source S3 bucket to store original pictures and metadata driving the site.
- Resized S3 bucket to store resized versions of the original pictures.
- Web S3 bucket to store the static website generated from the data in the source bucket.
- Resize lambda function to automatically resize images added to the source S3 bucket and store them in the resized S3 bucket.
- Site builder lambda function to automatically rebuild the static website
when
changes are made to the source S3 bucketa regularly scheduled event fires from Cloudwatch Events.
Requires that aws-cli, docker and htpasswd be installed.
You'll also need a domain whose CNAME DNS value you can update.
A video walkthrough is available on YouTube.
-
Configure
aws-cli(recommended to useus-east-1, see "Miscellaneous" below):$ aws configure AWS Access Key ID [None]: AKIA... AWS Secret Access Key [None]: illx... Default region name [None]: us-east-1 Default output format [None]: $ aws configure set preview.cloudfront true -
Create KMS encryption key:
aws kms create-key. Keep note of itsKeyIdin the response. Note that each KMS key costs $1/month. -
Create CloudFront Key Pair (note, you must do this as the root user of your AWS account, not an administrator), take note of the key pair ID and download the private key: https://console.aws.amazon.com/iam/home?region=us-east-1#/security_credential.
-
Encrypt the CloudFront private key:
aws kms encrypt --key-id $KMS_KEY_ID --plaintext fileb://pk-*.pem \ --query CiphertextBlob --output text -
Create a local
htpasswdfile with your usernames and passwords. You can generate the hashes from the command line:$ htpasswd -nB username > htpasswd New password: ********** Re-type new password: ********** -
Encrypt your
htpasswdfile using KMS again:aws kms encrypt --key-id $KMS_KEY_ID --plaintext fileb://htpasswd) \ --query CiphertextBlob --output text -
Create CloudFront Origin Access Identity, take note of the identity in the response:
aws cloudfront create-cloud-front-origin-access-identity \ --cloud-front-origin-access-identity-config \ "CallerReference=$(cat /dev/urandom | base64 | base64 | head -c 14),Comment=AWSPics OAI" -
Create Image Magick Lambda Layer for Amazon Linux 2 AMIs.
~~While logged into AWS, go to this link:
<https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:145266761615:applications~image-magick-lambda-layer>Click Deploy (currently the orange button on the upper right).
Then click on your Lambda - Layers and you will see a version ARN that looks like:~~
The best practice for image magick for the resize function is now is to create your own lambda layer. Download this zip file from here: https://github.com/CyprusCodes/imagemagick-aws-lambda-2/tree/master/archive Upload it to AWS Lambda as a layer (and select runtime of node.js 20 and architecture arm64) as a runtime, and note the change to arm64 architecture on the image resize function in the app.yml file, because the zip file referenced is actually compiled for arm64 and node.js 20. Use the arn of the lambda layer in your dist/config.json file.
Name Version Version ARN image-magick 1 arn:aws:lambda:us-east-1:000000000000:layer:image-magick:1Hold on to that ARN for later.
Create a configuration file called dist/config.json, based on
config.example.json. Make sure you don't commit this file
to source control (the dist folder is ignored).
It should contain the following info - minus the comments:
[
// -------------------
// PLAIN TEXT SETTINGS
// -------------------
// website domain
"website=website.com",
// title for the website
"websiteTitle=My awesome private photo gallery",
// S3 bucket where the static website generated from the data in the
// source bucket will be stored
"webBucket=html-files-here",
// S3 bucket where original pictures and metadata driving the site will be
// stored
"sourceBucket=original-images-here",
// S3 bucket where resized images will be stored
"resizedBucket=resized-images-here",
// Origin Access Identity from step 7
"originAccessIdentity=EJG...",
// how long the CloudFront access is granted for, in seconds
// note that the cookies are session cookies, and will get deleted when the
// browser is closed anyway
"sessionDuration=86400",
// Optional tracking ID for Google Analytics, if specified then a GA JS
// snippet will be outputted in the site's HTML, or leave blank for no GA
"googleanalytics=",
// Optionally override the path prefix for where original albums and their
// pictures live, or leave blank to have this default to "pics/original/"
"picsOriginalPath=",
// Optionally sort albums by name when building the homepage (if
// groupAlbumsIntoCollections is disabled), or when building collection pages
// (if groupAlbumsIntoCollections is enabled), specify either "asc" or "desc",
// or leave blank to output albums in the order that they're returned from
// the S3 list objects call
"albumSort=",
// Optionally sort pictures in all albums by name when building album pages,
// specify either "asc" or "desc", or leave blank to output pictures in the
// reverse order that they're returned from the S3 list objects call
"pictureSort=",
// Optionally sort collections by name when building the homepage (if
// groupAlbumsIntoCollections is enabled), specify either "asc" or "desc", or
// leave blank to output collections in the order that they're returned from
// the S3 list objects call
"collectionSort=",
// Optionally specify "true" to indicate that the pictures have two grouping
// levels, collections (first-level folders in the bucket) and albums
// (second-level folders in the bucket), or leave blank to indicate that
// the pictures just have one grouping level, albums
"groupAlbumsIntoCollections=",
// Indent homepage and album HTML output with spaces, specify "true" to
// enable, or leave blank to instead indent HTML output with tabs
"spacesInsteadOfTabs=",
// Optionally show the specified custom HTML instead of a design credits link
// to HTML5 UP on the home page, recommended that this be a link in the form:
// <a href="https://site123.com">Site 123</a>
// Note: a design credits link to HTML5 UP will still show on album pages
"homePageCreditsOverride=",
// Optionally hide the design credits link to HTML5 UP on the home page,
// specify "true" to enable, or leave blank to show the link
// Note: a design credits link to HTML5 UP will still show on album pages
"hideHomePageCredits=",
// KMS key ID created in step 2
"kmsKeyId=00000000-0000-0000-0000-000000000000",
// CloudFront key pair ID from step 3
// This is not sensitive, and will be one of the cookie values
"cloudFrontKeypairId=APK...",
// ------------------
// Image Magick Lambda Layer ARN
// - this is needed for ImageMagick to resize images in Node.js 10.x
// - from step 8 above
// - context above in README
// ------------------
"ImageMagickLayer=arn:aws:lambda:us-east-1:........:layer:image-magick:...",
// ------------------
// ENCRYPTED SETTINGS
// ------------------
// encrypted CloudFront private key from step 4
"encryptedCloudFrontPrivateKey=AQICAH...",
// encrypted contents of the <htpasswd> file from step 6
"encryptedHtpasswd=AQICAH...",
// ------------------
// SSL Certificate ARN
// - provide this if you want to use an existing ACM Certificate.
// - see below in the README
// ------------------
"sslCertificateArn=arn:aws:acm:us-east-1..."
]You can then deploy the full stack using:
# name of an S3 bucket for storing the Lambda code
# bucket will be created if it doesn't already exist
./deploy <unique_bucket_name_here>Or optionally deploy and specify the stack name (otherwise it defaults to AWSPics):
./deploy --stack-name=<stack_name_here> <unique_bucket_name_here>Any subsequent changes that you make to this code can be redeployed with the same command. CloudFormation will create a "changeset" to only update the modified resources.
The first deployment should take about 30 minutes since there's a lot to set up. You'll also receive an email to approve the SSL certificate request, which you should complete quickly, so that the rest of the deployment can proceed.
You will want to update the frequency of the Cloudwatch Events Rule from its default setting at 365 days to something more appropriate to your needs. You can adjust this pre-deployment in the app.yml file or after the fact in the AWS Management console.
When Amazon deprecated Node.js 8.10, they removed ImageMagick from the Amazon Linux 2 AMIs that are required to run Node.js 10.x. Again, ImageMagick is no longer bundled with the Node.js 10.x runtime. This fix may also help with running on Node.js 12.x in the future. This provides a Lambda Layer (essentially a library) for your Lambda function that makes the existing code work with Node.js 10.x.
The best practice for image magick for the resize function is now is to create your own lambda layer. Download this zip file from here: https://github.com/CyprusCodes/imagemagick-aws-lambda-2/tree/master/archive
Upload it to AWS Lambda as a layer (and select runtime of node.js 20 and architecture arm64) as a runtime, and note the change to arm64 architecture on the image resize function in the app.yml file, because the zip file referenced is actually compiled for arm64 and node.js 20. Use the arn of the lambda layer in your dist/config.json file.
htpasswd-auth was outdated and using bcrypt which reached end of life and was failing. This was updated to bcrypt 5.0.1+ and node.js 20. Also a minor update to the login function updating the buffer function (Buffer -> Buffer.from in login index.js) for the latest node.js 20.x.
Note that Amazon removed aws-sdk from the Amazon Linux base image that lambda functions run on, so it was added to the package.json, and it increases the size of each lambda function zip file by 13 MB or so. This could be added as a lambda layer to save space if needed, but this was an easier change to make.
When manually uploading files (e.g. photos) to S3 Buckets for this application, because of the encryption, you want to select the option of an encryption by default.
Also, the site-builder function used to be cloudwatch events, but it has been renamed by AWS. The best way to rerun the site-builder is to just hit the test button from AWS Lambda. That generates the site building on demand. Otherwise the default is to run infrequently, as configured in the app.yml.
AWS Certificate Manager now supports SSL cert verification via DNS validation. It is recommended that you manually request the certificate for your hosted zone and chose DNS validation method for much faster validation. Then use the resulting ARN in your config. You can also leave this config key empty to create the certificate as normal.
Once the initial deployment is done, you'll need to point your domain's DNS settings to add a CNAME to the newly created CloudFront Distribution URL, which you can find by visiting the CloudFront section of the AWS Console. Once the DNS update has propagated, you'll be able to access your private photo gallery at that domain. However, there's no content yet, so you won't see much of anything. You can generate sample albums to see it in action.
There's a script included that will pull random images from Unsplash to populate your photo gallery with sample content.
./generate_random_albums <number of albums>The Lambda function responsible for logging in creates signed session cookies when given valid credentials. CloudFront will verify that every request has valid cookies before forwarding them.
Browser CloudFront Lambda S3
| | | |
| ---------- get ---------> | | |
| | | |
| [no cookie] | |
| | | |
| | | |
| | | |
| <------ error page ------ | | |
| | |
| -------------------- login ------------------> | |
| <------------------- cookies ----------------- | |
| |
| ---------- get ---------> | |
| | |
| [has cookie] |
| | |
| | -----------------------------------> |
| | <------------ html page ------------ |
| <------ html page ------- |
This project only works as-is if everything is set up in the us-east-1 AWS
region, because CloudFormation only supports SSL certificates from that region.
It's not too difficult to adapt this to work in another region, but you can't
rely on the SSL certificate being created in CloudFormation. Create it manually
(using either the AWS Console or the CLI) and reference it in the
WebDistribution by its ARN explicitly rather than the !Ref SSLCert
reference.
GeoRestriction is commented out in the CloudFront configuration in the app.yaml. If you are sharing with friends and family in a specific geographic area, this is a slight improvement to security and cost reduction. The US is provided as an example, but additional countries can be added to a (whitelist/blacklist) based on their two letter ISO 3166-1 alpha-2 country code.
S3 Server Side AES256 encryption is enabled for the source and resized photo buckets and encrypts files using the AWS S3 Master key. Each bucket is configured to force encryption of any file it receives (you will need to check the upload box or specify it in the CLI when uploading photo files to the buckets) and you will get access denied messages if you don't. The Resize function re-encrypts the resized photos with AES256 SSE before uploading them into the resized bucket. Cloudfront with an OAI is able to access files using the S3 Master Key without any issue. One cannot at this time use a KMS key for encrypting bucket data to be accessed via Cloudfront without more complexity.
The EventInvoke config is included for SiteBuilder to prevent it from queueing up invocations and causing multiple cloudfront invalidations at the same time. If you need to run sitebuilder more frequently, adjust the rate of events by editing the CloudWatch Events rule in the Management console or the app.yml file.
Also, you can reduce compute costs and lock down the application several ways: 1) by manually throttling the Resize Function and the SiteBuilder Function in Lambda in the Management console or 2) disabling the CloudWatch Events rule that runs SiteBuilder or 3) manually disabling the trigger for a Lambda function in the Management console.
Default directory for photos is S3://BUCKET_NAME/pics/original/YOUR_ALBUMS_GO_HERE
If the project deploys and the login is entered correctly, but you are receiving access denied messages, review your DNS settings. You only need a single DNS A record pointing to the CloudFront Alias for your domain, and time for it to propagate.
If SiteBuilder is hanging or having trouble completing, you may need to adjust the rate limiting delay block in index.js. The current S3 rate limit is 3500 writes a second, and 5500 reads/sec. If you're writing 30 files per album, if you have more than 116 albums, you will hit the rate limit - and SiteBuilder will just hang and you will see the files as a partial listing in the web directory.
This project is mostly a compilation from multiple existing projects out there.
- Multiverse HTML template
- Lens HTML template
- Log In HTML template
- Lazy Load Javascript
- Lambda Cloudfront Cookies
- Lambda as a Cloudfront Origin
- Put S3 behind Cloudfront
- Restrict S3 to only Cloudfront
- Lambda with S3 tutorial
- Generating Cloudfront Key Pair
Credits for update to nodejs 10.x and ImageMagick Layer:
- NPM Mime Package Update v2
- ImageMagick after execution environment updates on AWS Forums
- ImageMagick Lambda Layer for Amazon Linux 2 AMIs
- image-magick-lambda-layer — version 1.0.0
Credits for update to nodejs 20.x and ImageMagick Layer:
- zip file for use as nodejs 20.x ImageMagick Lambda Layer
- Manual update to htpasswd-auth came from this Pull Request
- ChatGPT 4 helped identifiy the Buffer.from update that was needed, and a few other issues. Thanks OpenAI!
AWSPics is MIT licensed.