Dev container cache image generation

.devcontainer/README.md

@@ -32,7 +32,7 @@ Next: **[Try it out!](#try-it)**
 
 ## Quick start - GitHub Codespaces
 
-> **IMPORTANT:** The current user beta for GitHub Codespaces uses a "Basic" sized codespace which is too small to run a full build of VS Code. You'll soon be able to use a "Standard" sized codespace (4-core, 8GB) that will be better suited for this purpose.
+> **IMPORTANT:** The current free user beta for GitHub Codespaces uses a "Basic" sized codespace which does not have enough RAM to run a full build of VS Code and will be considerably slower during codespace start and running VS Code. You'll soon be able to use a "Standard" sized codespace (4-core, 8GB) that will be better suited for this purpose (along with even larger sizes should you need it).
 
 1. From the [microsoft/vscode GitHub repository](https://github.com/microsoft/vscode), click on the **Code** dropdown, select **Open with Codespaces**, and the **New codespace**
 
@@ -81,7 +81,9 @@ To start working with Code - OSS, follow these steps:
     bash scripts/code.sh
     ```
 
-2. After the build is complete, open a web browser and go to [http://localhost:6080](http://localhost:6080) or use a [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/) to connect to `localhost:5901` and enter `vscode` as the password.
+    Note that a previous run of `yarn install`  will already be cached, so this step should simply pick up any recent differences.
+
+2. After the build is complete, open a web browser or a [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/) to the desktop environnement as described in the quick start and enter `vscode` as the password.
 
 3. You should now see Code - OSS!
 

.devcontainer/cache/.gitignore

@@ -0,0 +1 @@
+*.manifest

.devcontainer/cache/before-cache.sh

@@ -0,0 +1,15 @@
+#/bin/bash
+
+# This file establishes a basline for the reposuitory before any steps in the "prepare.sh"
+# are run. Its just a find command that filters out a few things we don't need to watch.
+
+set -e
+
+SCRIPT_PATH="$(cd "$(dirname $0)" && pwd)"
+SOURCE_FOLDER="${1:-"."}"
+
+cd "${SOURCE_FOLDER}"
+echo "[$(date)] Generating ""before"" manifest..."
+find -L . -not -path "*/.git/*" -and -not -path "${SCRIPT_PATH}/*.manifest" -type f >  "${SCRIPT_PATH}/before.manifest"
+echo "[$(date)] Done!"
+

.devcontainer/cache/build-cache-image.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+
+# This file simply wraps the dockeer build command used to build the image with the
+# cached result of the commands from "prepare.sh" and pushes it to the specified
+# container image registry.
+
+set -e
+
+SCRIPT_PATH="$(cd "$(dirname $0)" && pwd)"
+CONTAINER_IMAGE_REPOSITORY="$1"
+BRANCH="${2:-"master"}"
+
+if [ "${CONTAINER_IMAGE_REPOSITORY}" = "" ]; then
+	echo "Container repository not specified!"
+	exit 1
+fi
+
+TAG="${BRANCH//\//-}"
+echo "[$(date)] ${BRANCH} => ${TAG}"
+cd "${SCRIPT_PATH}/../.."
+
+echo "[$(date)] Starting image build..."
+docker build -t ${CONTAINER_IMAGE_REPOSITORY}:"${TAG}" -f "${SCRIPT_PATH}/cache.Dockerfile" .
+echo "[$(date)] Image build complete."
+
+echo "[$(date)] Pushing image..."
+docker push ${CONTAINER_IMAGE_REPOSITORY}:"${TAG}"
+echo "[$(date)] Done!"

.devcontainer/cache/cache-diff.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# This file is used to archive off a copy of any differences in the source tree into another location
+# in the image. Once the codespace is up, this will be restored into its proper location (which is
+# quick and happens parallel to other startup activities)
+
+set -e
+
+SCRIPT_PATH="$(cd "$(dirname $0)" && pwd)"
+SOURCE_FOLDER="${1:-"."}"
+CACHE_FOLDER="${2:-"/usr/local/etc/devcontainer-cache"}"
+
+echo "[$(date)] Starting cache operation..."
+cd "${SOURCE_FOLDER}"
+echo "[$(date)] Determining diffs..."
+find -L . -not -path "*/.git/*" -and -not -path "${SCRIPT_PATH}/*.manifest" -type f > "${SCRIPT_PATH}/after.manifest"
+grep -Fxvf  "${SCRIPT_PATH}/before.manifest" "${SCRIPT_PATH}/after.manifest" > "${SCRIPT_PATH}/cache.manifest"
+echo "[$(date)] Archiving diffs..."
+mkdir -p "${CACHE_FOLDER}"
+tar -cf "${CACHE_FOLDER}/cache.tar" --totals --files-from "${SCRIPT_PATH}/cache.manifest"
+echo "[$(date)] Done! $(du -h "${CACHE_FOLDER}/cache.tar")"

.devcontainer/cache/cache.Dockerfile

@@ -0,0 +1,14 @@
+# This dockerfile is used to build up from a base image to create an image with cached results of running "prepare.sh".
+# Other image contents: https://github.com/microsoft/vscode-dev-containers/blob/master/repository-containers/images/github.com/microsoft/vscode/.devcontainer/base.Dockerfile
+FROM mcr.microsoft.com/vscode/devcontainers/repos/microsoft/vscode:dev
+
+ARG USERNAME=node
+COPY --chown=${USERNAME}:${USERNAME} . /repo-source-tmp/
+RUN mkdir /usr/local/etc/devcontainer-cache \
+	&& chown ${USERNAME} /usr/local/etc/devcontainer-cache /repo-source-tmp \
+	&& su ${USERNAME} -c "\
+		cd /repo-source-tmp \
+		&& .devcontainer/cache/before-cache.sh \
+		&& .devcontainer/prepare.sh \
+		&& .devcontainer/cache/cache-diff.sh" \
+	&& rm -rf /repo-source-tmp

.devcontainer/cache/restore-diff.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# This file restores the results of the "prepare.sh" into their proper locations
+# once the container has been created. It runs as a postCreateCommand which
+# in GitHub Codespaces occurs parallel to other startup activities and does not
+# really add to the overal startup time given how quick the operation ends up being.
+
+set -e
+
+SOURCE_FOLDER="$(cd "${1:-"."}" && pwd)"
+CACHE_FOLDER="${2:-"/usr/local/etc/devcontainer-cache"}"
+
+if [ ! -d "${CACHE_FOLDER}" ]; then
+	echo "No cache folder found."
+	exit 0
+fi
+
+echo "[$(date)] Expanding $(du -h "${CACHE_FOLDER}/cache.tar") file to ${SOURCE_FOLDER}..."
+cd "${SOURCE_FOLDER}"
+tar -xf "${CACHE_FOLDER}/cache.tar"
+rm -f "${CACHE_FOLDER}/cache.tar"
+echo "[$(date)] Done!"
+

.devcontainer/devcontainer.json

@@ -2,7 +2,7 @@
 	"name": "Code - OSS",
 
 	// Image contents: https://github.com/microsoft/vscode-dev-containers/blob/master/repository-containers/images/github.com/microsoft/vscode/.devcontainer/base.Dockerfile
-	"image": "mcr.microsoft.com/vscode/devcontainers/repos/microsoft/vscode:dev",
+	"image": "mcr.microsoft.com/vscode/devcontainers/repos/microsoft/vscode:master",
 
 	"workspaceMount": "source=${localWorkspaceFolder},target=/home/node/workspace/vscode,type=bind,consistency=cached",
 	"workspaceFolder": "/home/node/workspace/vscode",
@@ -25,5 +25,8 @@
 		"GitHub.vscode-pull-request-github"
 	],
 
+	// Optionally loads a cached yarn install for the repo
+	"postCreateCommand": ".devcontainer/cache/restore-diff.sh",
+
 	"remoteUser": "node"
 }

.devcontainer/prepare.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# This file contains the steps that should be run when creating the intermediary image that contains
+# contents for that should be in the image by default. It will be used to build up from the base image
+# to create an image that speeds up first time use of the dev container by "caching" the results
+# of these commands. Developers can still run these commands without an issue once the container is
+# up, but only differences will be processed which also speeds up the first time these operations occur.
+
+yarn install
+yarn electron

.github/workflows/devcontainer-cache.yml

@@ -0,0 +1,42 @@
+name: VS Code Repo Dev Container Cache Image Generation
+
+on:
+  push:
+    # Currently doing this for master, but could be done for PRs as well
+    branch:
+      - 'master'
+
+    # Only updates to these files result in changes to installed packages, so skip otherwise
+    paths:
+      - '**/package.json'
+      - '**/package-lock.json'
+      - '**/yarn.lock'
+
+jobs:
+  devcontainer:
+    name: Generate cache image
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        id: checkout
+        uses: actions/checkout@v2
+
+      - name: Azure CLI login
+        id: az_login
+        uses: azure/login@v1
+        with:
+          creds: ${{ secrets.AZ_ACR_CREDS }}
+
+      - name: Build and push
+        id: build_and_push
+        run: |
+          set -e
+
+          ACR_REGISTRY_NAME=$(echo ${{ secrets.CONTAINER_IMAGE_REGISTRY }} | grep -oP '(.+)(?=\.azurecr\.io)')
+          az acr login --name $ACR_REGISTRY_NAME
+
+          GIT_BRANCH=$(echo "${{ github.ref }}" | grep -oP 'refs/(heads|tags)/\K(.+)')
+          if [ "$GIT_BRANCH" == "" ]; then GIT_BRANCH=master; fi
+
+          .devcontainer/cache/build-cache-image.sh "${{ secrets.CONTAINER_IMAGE_REGISTRY }}/public/vscode/devcontainers/repos/microsoft/vscode" "master"
+