Skip to content

Create weka cluster on Azure using TF

Notifications You must be signed in to change notification settings

wayveai/terraform-azure-weka

 
 

Repository files navigation

Azure-weka deployment Terraform package

The Weka cluster on Azure provides a fast and scalable platform to run, for example, performance-intensive applications and hybrid cloud workflows. It can also be used for object stores, tiering, and snapshots using the Azure Blob service. The provided Azure-Weka Terraform package contains modules and examples you can customize according to your deployment needs. The installation is based on applying the customized Terraform variables file to a predefined Azure subscription. Applying the Terraform variables file performs the following:

  • Creates resources in a predefined resource group, such as virtual machines, network interfaces, function app, load balancer, and more.
  • Deploys Azure virtual machines.
  • Installs the Weka software.
  • Configures the Weka cluster.

Usage

module "deploy-weka" {
   source                = "weka/weka/azure"
   prefix                = "weka"
   rg_name               = "myResourceGroup"
   vnet_name             = "weka-vpc-0"
   subnets               = ["weka-subnet-0"]
   sg_id                 = "security-group-id"
   get_weka_io_token     = "get_weka_io_token"
   cluster_name          = "myCluster"
   set_obs_integration   = false
   instance_type         = "Standard_L8s_v3"
   cluster_size          = 6
   private_dns_zone_name = "myDns.private.net"
}

Requirements

Name Version
azurerm ~>3.43.0

Providers

Name Version
azurerm ~>3.43.0
local n/a
null n/a
random n/a
template n/a
tls n/a

Modules

No modules.

Resources

Name Type
azurerm_application_insights.application_insights resource
azurerm_key_vault.key_vault resource
azurerm_key_vault_access_policy.function-app-get-secret-permission resource
azurerm_key_vault_access_policy.key_vault_access_policy resource
azurerm_key_vault_access_policy.logic-app-get-secret-permission resource
azurerm_key_vault_access_policy.scale-up-logic-app-get-secret-permission resource
azurerm_key_vault_secret.function_app_default_key resource
azurerm_key_vault_secret.get_weka_io_token resource
azurerm_key_vault_secret.private-ssh-keys resource
azurerm_key_vault_secret.public-ssh-keys resource
azurerm_key_vault_secret.weka_password_secret resource
azurerm_lb.backend-lb resource
azurerm_lb.ui_lb resource
azurerm_lb_backend_address_pool.lb_backend_pool resource
azurerm_lb_backend_address_pool.ui_lb_backend_pool resource
azurerm_lb_probe.backend_lb_probe resource
azurerm_lb_probe.ui_lb_probe resource
azurerm_lb_rule.backend_lb_rule resource
azurerm_lb_rule.ui_lb_rule resource
azurerm_linux_function_app.function_app resource
azurerm_linux_virtual_machine_scale_set.custom_image_vmss resource
azurerm_linux_virtual_machine_scale_set.default_image_vmss resource
azurerm_log_analytics_workspace.la_workspace resource
azurerm_logic_app_action_custom.logic_app_action_fetch resource
azurerm_logic_app_action_custom.logic_app_action_scale_down resource
azurerm_logic_app_action_custom.logic_app_action_scale_up resource
azurerm_logic_app_action_custom.logic_app_action_terminate resource
azurerm_logic_app_action_custom.logic_app_action_transient resource
azurerm_logic_app_action_custom.scale_down_logic_app_action_get_secret resource
azurerm_logic_app_action_custom.scale_up_logic_app_action_get_secret resource
azurerm_monitor_diagnostic_setting.function_diagnostic_setting resource
azurerm_monitor_diagnostic_setting.insights_diagnostic_setting resource
azurerm_monitor_diagnostic_setting.logic_app_diagnostic_setting resource
azurerm_monitor_diagnostic_setting.scale_up_logic_app_diagnostic_setting resource
azurerm_private_dns_a_record.dns_a_record_backend_lb resource
azurerm_proximity_placement_group.ppg resource
azurerm_resource_group_template_deployment.api_connections_template_deployment resource
azurerm_resource_group_template_deployment.workflow_scale_down_template_deployment resource
azurerm_resource_group_template_deployment.workflow_scale_up_template_deployment resource
azurerm_role_assignment.function-app-key-user-access-admin resource
azurerm_role_assignment.function-app-key-vault-secrets-user resource
azurerm_role_assignment.function-app-reader resource
azurerm_role_assignment.function-app-scale-set-machine-owner resource
azurerm_role_assignment.function-assignment resource
azurerm_role_assignment.logic-app-key-vault-secrets-user resource
azurerm_role_assignment.logic-app-storage-account-contributor resource
azurerm_role_assignment.scale-up-logic-app-key-vault-secrets-user resource
azurerm_role_assignment.scale-up-logic-app-storage-account-contributor resource
azurerm_role_assignment.storage-account-contributor resource
azurerm_role_assignment.storage-blob-data-owner resource
azurerm_role_assignment.vm_role_assignment resource
azurerm_service_plan.app_service_plan resource
azurerm_storage_account.deployment_sa resource
azurerm_storage_blob.state resource
azurerm_storage_container.deployment resource
local_file.private_key resource
local_file.public_key resource
null_resource.force-delete-vmss resource
random_password.weka_password resource
tls_private_key.ssh_key resource
azurerm_client_config.current data source
azurerm_function_app_host_keys.function_keys data source
azurerm_resource_group.rg data source
azurerm_subnet.subnets data source
azurerm_subscription.primary data source
azurerm_virtual_network.vnet data source
template_cloudinit_config.cloud_init data source
template_file.init data source

Inputs

Name Description Type Default Required
apt_repo_url The URL of the apt private repository. string "" no
blob_obs_access_key The access key of the existing Blob object store container. string "" no
cluster_name The cluster name. string "poc" no
cluster_size The number of virtual machines to deploy. number 6 no
container_number_map Maps the number of objects and memory size per machine type.
map(object({
compute = number
drive = number
frontend = number
nvme = number
nics = number
memory = string
}))
{
"Standard_L16s_v3": {
"compute": 4,
"drive": 2,
"frontend": 1,
"memory": "72GB",
"nics": 8,
"nvme": 2
},
"Standard_L32s_v3": {
"compute": 4,
"drive": 2,
"frontend": 1,
"memory": "189GB",
"nics": 8,
"nvme": 4
},
"Standard_L48s_v3": {
"compute": 3,
"drive": 3,
"frontend": 1,
"memory": "306GB",
"nics": 8,
"nvme": 6
},
"Standard_L64s_v3": {
"compute": 4,
"drive": 2,
"frontend": 1,
"memory": "418GB",
"nics": 8,
"nvme": 8
},
"Standard_L8s_v3": {
"compute": 1,
"drive": 1,
"frontend": 1,
"memory": "31GB",
"nics": 4,
"nvme": 1
}
}
no
custom_image_id Custom image id string null no
default_disk_size The default disk size. number 48 no
function_app_dist Function app code dist string "dev" no
function_app_log_level Log level for function app (from -1 to 5). See https://github.com/rs/zerolog#leveled-logging number 1 no
function_app_storage_account_container_prefix Weka storage account container name prefix string "weka-tf-functions-deployment-" no
function_app_storage_account_prefix Weka storage account name prefix string "weka" no
function_app_version Function app code version (hash) string "05ace7d4a56ef2f161839b42ce97fe5b" no
get_weka_io_token The token to download the Weka release from get.weka.io. string "" no
hotspare Hot-spare value. number 1 no
install_ofed_url The URL of the Blob with the OFED tgz file. string "" no
install_weka_url The URL of the Weka release download tar file. string "" no
instance_type The virtual machine type (sku) to deploy. string "Standard_L8s_v3" no
linux_vm_image The default azure vm image reference. map(string)
{
"offer": "UbuntuServer",
"publisher": "Canonical",
"sku": "18.04-LTS",
"version": "latest"
}
no
obs_container_name Name of existing obs conatiner name string "" no
obs_name Name of existing obs storage account string "" no
ofed_version The OFED driver version to for ubuntu 18. string "5.7-1.0.2.0" no
prefix The prefix for all the resource names. For example, the prefix for your system name. string "weka" no
private_dns_zone_name The private DNS zone name. string n/a yes
private_network Determines whether to enable a private or public network. The default is public network. bool false no
protection_level Cluster data protection level. number 2 no
rg_name A predefined resource group in the Azure subscription. string n/a yes
set_obs_integration Determines whether to enable object stores integration with the Weka cluster. Set true to enable the integration. bool false no
sg_id The security group id. string n/a yes
ssh_private_key The path to the VM private key. If it is not set, the key is auto-generated. If it is set, also set the ssh_private_key. The private key used for connecting to the deployed virtual machines to initiate the clusterization of Weka. string null no
ssh_public_key The path to the VM public key. If it is not set, the key is auto-generated. If it is set, also set the ssh_private_key. string null no
stripe_width Stripe width = cluster_size - protection_level - 1 (by default). number -1 no
subnet_delegation_id Subnet delegation id string n/a yes
subnets The subnet names list. list(string) n/a yes
subscription_id The subscription id for the deployment. string n/a yes
tags_map A map of tags to assign the same metadata to all resources in the environment. Format: key:value. map(string)
{
"creator": "tf",
"env": "dev"
}
no
tiering_ssd_percent When set_obs_integration is true, this variable sets the capacity percentage of the filesystem that resides on SSD. For example, for an SSD with a total capacity of 20GB, and the tiering_ssd_percent is set to 20, the total available capacity is 100GB. number 20 no
traces_per_ionode The number of traces per ionode. Traces are low-level events generated by Weka processes and are used as troubleshooting information for support purposes. number 10 no
vm_username The user name for logging in to the virtual machines. string "weka" no
vnet_name The virtual network name. string n/a yes
weka_version The Weka version to deploy. string "4.1.0.71" no

Outputs

Name Description
cluster_helpers_commands Useful commands and script to interact with weka cluster

About

Create weka cluster on Azure using TF

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 51.0%
  • HCL 45.0%
  • Shell 4.0%