Skip to content

Latest commit

 

History

History
246 lines (220 loc) · 11.6 KB

README.md

File metadata and controls

246 lines (220 loc) · 11.6 KB

Playground deployment on GCP

This guide shows you how to deploy full Playground environment on Google Cloud Platform (GCP) environment. Alternatively, you can automate Playground environment deployment with Cloud Build as described in readme.

Prerequisites:

Following items need to be setup for Playground deployment on GCP:

  1. GCP project

  2. GCP User account (Note: You will find the instruction "How to create User account" for your new project)
    Ensure that the account has at least the following IAM roles:

    • App Engine Admin
    • App Engine Creator
    • Artifact Registry Administrator
    • Cloud Datastore Index Admin
    • Cloud Datastore User
    • Cloud Memorystore Redis Admin
    • Cloud Functions Developer
    • Compute Admin
    • Create Service Accounts
    • DNS Administrator
    • Kubernetes Engine Admin
    • Quota Administrator
    • Role Administrator
    • Security Admin
    • Service Account User
    • Storage Admin

  3. Google Cloud Storage bucket for Terraform state file

  4. DNS name for your Playground deployment instance further referred as <playground.zone>

  5. OS with installed software listed below:

  1. Apache Beam Git repository cloned locally

Prepare deployment configuration:

Playground uses terraform.tfvars located in playground/terraform/environment/<environment_name> to define variables specific to an environment (e.g., prod, test, staging).

  1. Create a folder (further referred as <environment_name>) to define a new environment and place configuration files into it:
  • terraform.tfvars environment variables:
project_id             = "project_id"               # GCP Project ID
network_name           = "playground-network"       # GCP VPC Network Name for Playground deployment
subnetwork_name        = "playground-subnetwork"    # GCP VPC Subnetwork Name for Playground deployment
gke_name               = "playground-backend"       # Playground GKE Cluster name
region                 = "us-east1"                 # Deployment region for all created resources
zone                   = "us-east1-b"               # Deployment zone in the specified region
state_bucket           = "playground-state-bucket"  # GCS bucket name for Terraform state file. The bucket must be created manually before deployment
redis_name             = "playground-redis"         # Name for Redis instance
redis_tier             = "BASIC"                    # Redis tier type. Options: "Basic" or "Standard_HA". Default: "BASIC"
min_count              = 2                          # Min node count for the GKE cluster
max_count              = 6                          # Max node count for the GKE cluster
skip_appengine_deploy  = false                      # AppEngine flag - defined if AppEngine and Datastore need to be installed. Should be "true" if AppEngine and Datastore were installed before
ip_address_name        = "playground-static-ip"     # GCP Static IP Address name
repository_id          = "playground-artifacts"     # GCP Artifact repository name for Playground images
service_account_id     = "playground-gke-sa"   # GCP Service account name
gke_machine_type       = "e2-standard-8"            # Machine type for GKE Nodes
env                    = "prod"                     # Environment. The same value as for <environment_name> parameter
  • state.tfbackend environment variables:
bucket               = "playground-state-bucket"         # GCS bucket name for Terraform state file. The same value as in terraform.tfvars file.
  1. Configure authentication for the Google Cloud Platform
gcloud init

gcloud auth application-default login

Deploy Playground infrastructure:

If you intend to install more than one environment, you may need to manually remove 'playground/terraform/.terraform' folder before a new attempt

  1. Start the following command from the top level repository folder ("beam") to deploy the Payground infrastructure:
./gradlew playground:terraform:InitInfrastructure -Pproject_environment="<environment_name>"
  1. Find a Static IP in your GCP project>VPC Network>IP Addresses>pg-static-ip
    Add following DNS A records for the discovered static IP address:
java.<playground.zone>
python.<playground.zone>
scio.<playground.zone>
go.<playground.zone>
router.<playground.zone>
<playground.zone>

More about DNS zone registration
More about A records in DNS

Deploy Playground to Kubernetes:

  1. Run the following command to authenticate in the Docker registry:
gcloud auth configure-docker <region>-docker.pkg.dev
  1. Run the following command to authenticate in GKE:
gcloud container clusters get-credentials --region <zone> <gke_name> --project <project_id>

Start the following command from the top level repository folder ("beam") to deploy the Payground infrastructure:

./gradlew playground:terraform:gkebackend -Pproject_environment="<environment_name>" -Pdocker-tag="tag" \
  -Pdns-name="<playground.zone>" -Psdk-tag=2.44.0 \
   -Pdocker-repository-root="<region>-docker.pkg.dev/<project_id>/<repository_id>" -Pdatastore-namespace="Playground"

Where tag - image tag for docker images, Psdk-tag - current Apache Beam SDK version, Pdatastore-namespace - namespace for Datastore

During script execution, a Google managed certificate will be created. Provisioning might take up to 60 minutes.

Validate deployed Playground:

  1. Run "helm list" command in the console to ensure that status is "deployed":
NAME            NAMESPACE  REVISION        UPDATED         STATUS          CHART                          APP VERSION
playground      default       1            your time      deployed        playground-2.44.0-SNAPSHOT         1.0.0
  1. Run "kubectl get managedcertificate" command in the console to ensure that status is "Active":
NAME               AGE     STATUS
GCP Project       time     Active
  1. Open Beam Playground frontend webpage in a web browser (e.g. https://playground.zone) to ensure that Playground frontend page is available

Use HELM to update environment

Helm is responsible for deploying Beam Playground on the GKE cluster Use the steps below to update HELM Chart and apply it:

  1. Configure authentication for the Google Cloud Platform:
gcloud init

gcloud auth application-default login
  1. Run the following command to authenticate in the Docker registry:
gcloud auth configure-docker <region>-docker.pkg.dev
  1. Run the following command to authenticate in GKE:
gcloud container clusters get-credentials --region <zone> <gke_name> --project <project_id>
  1. Clone Apache BEAM repository from Git
  2. Make required changes in playground/infrastructure/helm-playground/values.yaml file:
replicaCount: 1
image:
   java_image: beam_playground-backend-java
   go_image: beam_playground-backend-go
   router_image: beam_playground-backend-router
   scio_image: beam_playground-backend-scio
   python_image: beam_playground-backend-python
   frontend_image: beam_playground-frontend
   pullPolicy: Always
service:
   type: NodePort
   targetPort: 8080
   port: 443
healthcheck:
   port: 8080
   livInitialDelaySeconds: 30
   livPeriodSeconds: 30
   readInitialDelaySeconds: 30
   readPeriodSeconds: 30
autoscaling:
   runners:
 	  maxReplicas: 4
 	  minReplicas: 2
   rest:
 	  maxReplicas: 4
 	  minReplicas: 1
   utilization:
 	memoryUtilization: 80
 	cpuUtilization: 95
static_ip: <external IP address>
redis_ip: <REDIS IP address>:6379
project_id: <Project ID>
registry: <region>-docker.pkg.dev/<project_id>/<repository_id>
static_ip_name: <ip_address_name>
tag: <docker-tag>
datastore_name: <datastore-namespace>
dns_name: <dns-name>
func_clean: https://<region>-<project_id>.cloudfunctions.net/playground-function-cleanup-<env>
func_put: https://<region>-<project_id>.cloudfunctions.net/playground-function-put-<env>
func_view: https://<region>-<project_id>.cloudfunctions.net/playground-function-view-<env>
  1. Execute following command to update HELM:
helm upgrade playground /playground/infrastructure/helm-helm-playground

Variables description for "values.yaml"

See Also:

Services

Configure Liveness, Readiness and Startup Probes

Cluster autoscaling

Variable Description
service/type connection type for service in the GKE Cluster
service/targetPort Service target port, connection from Service to POD
service/port Service port, connection from ingress to service
healthcheck/port Internal POD port for Healthcheck
healthcheck/livInitialDelaySeconds pre-polling delay
healthcheck/livPeriodSeconds Poll period
healthcheck/readInitialDelaySeconds Check if POD is ready
healthcheck/readPeriodSeconds Poll period for checking if POD is ready
autoscaling/runners/maxReplicas maximum number of PODs for a runner
autoscaling/runners/minReplicas minimum number of PODs for a runner
autoscaling/rest/maxReplicas max number of PODs per computer
autoscaling/rest/minReplicas minimum number of PODs for a router
autoscaling/memoryUtilization POD scaling activation threshold based on RAM usage
autoscaling/cpuUtilization POD scaling activation threshold based on CPU usage