Add JuiceFSRuntime engine to fluid backend (fluid-cloudnative#1047)

* Add juicefs runtime (fluid-cloudnative#11) * Add juicefs runtime Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * format --no-update when bucket is none & fix typo & fix test (fluid-cloudnative#12) Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * Add doc (fluid-cloudnative#13) * add juicefs_runtime doc & fix bug * add en doc Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * update juicefs_runtime.md (fluid-cloudnative#14) Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * Add node label after scheduling dataset (fluid-cloudnative#1040) * Update docker image, To #37253115 Signed-off-by: cheyang <[email protected]> * Prefer fuse node affinity, To #37253115 Signed-off-by: cheyang <[email protected]> * Prefer fuse node affinity, To #37253115 Signed-off-by: cheyang <[email protected]> * Prefer fuse node affinity, To #37253115 Signed-off-by: cheyang <[email protected]> * Update docker image, To #37253115 Signed-off-by: cheyang <[email protected]> * Update docker image, To #37253115 Signed-off-by: cheyang <[email protected]> * Update docker image, To #37253115 Signed-off-by: cheyang <[email protected]> * Sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * update config Signed-off-by: cheyang <[email protected]> * Sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> * Test for sync up node labels, To #37253115 Signed-off-by: cheyang <[email protected]> Signed-off-by: zwwhdls <[email protected]> * add ut for healthcheck. (fluid-cloudnative#1051) Signed-off-by: LDawns <[email protected]> Signed-off-by: zwwhdls <[email protected]> * Add mount propagation in node-driver-registrar (fluid-cloudnative#1049) node-driver-registrar will mount kubelet-dir on {{ .Values.csi.kubelet.rootDir }} without 'HostToContainer' or 'Bidirectional'. If another pod mount a temporary device on their volume directory like '${kubelet-dir}/lib/kubelet/pods/xxx/volumes/' which create before node-driver-registrar and delete after node-driver-registrar is created will cause device deconstruction to fail. Signed-off-by: tuji.yyf <[email protected]> Signed-off-by: zwwhdls <[email protected]> * add ut for template_engine.go and metadata.go (fluid-cloudnative#1039) Signed-off-by: littletiger123 <[email protected]> Signed-off-by: zwwhdls <[email protected]> * add master and fuse resource option for jindoruntime (fluid-cloudnative#1046) * implement master and fuse resource option Signed-off-by: frankleaf <[email protected]> * fix ut Signed-off-by: frankleaf <[email protected]> Signed-off-by: zwwhdls <[email protected]> * add test case of cert builder (fluid-cloudnative#1042) * add test case of cert Signed-off-by: yangyuliufeng <[email protected]> * prepare the test environment Signed-off-by: yangyuliufeng <[email protected]> Signed-off-by: zwwhdls <[email protected]> * implement SyncScheduleInfoToCacheNodes (fluid-cloudnative#15) Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * Fix ci (fluid-cloudnative#16) * implement SyncScheduleInfoToCacheNodes * fix golangci-lint Signed-off-by: zwwhdls <[email protected]> Signed-off-by: zwwhdls <[email protected]> * fix env/port format & adjust import format (fluid-cloudnative#17) Signed-off-by: zwwhdls <[email protected]> * Add test (fluid-cloudnative#18) * add test * update chart Signed-off-by: zwwhdls <[email protected]> * update juicefsruntime unit test (fluid-cloudnative#19) * add jucefsruntime ut * fix test * update juicefsruntime test Signed-off-by: zwwhdls <[email protected]> * fix confluct Signed-off-by: zwwhdls <[email protected]> * add some test (fluid-cloudnative#20) Signed-off-by: zwwhdls <[email protected]> * Fix cache dir (fluid-cloudnative#21) * fix cache dir Signed-off-by: zwwhdls <[email protected]> * update mountpoint & parse option Signed-off-by: zwwhdls <[email protected]> * fix test Signed-off-by: zwwhdls <[email protected]> * fix nodeSelector in fuse Signed-off-by: zwwhdls <[email protected]> * update doc Signed-off-by: zwwhdls <[email protected]> * fix test (fluid-cloudnative#22) Signed-off-by: zwwhdls <[email protected]> * change JuiceFSRuntimeStatus to RuntimeStatus (fluid-cloudnative#23) * change JuiceFSRuntimeStatus to RuntimeStatus Signed-off-by: ldd91 <[email protected]> * sync metadata after fuse pod up Signed-off-by: ldd91 <[email protected]> * sync metadata after fuse pod up Signed-off-by: ldd91 <[email protected]> * update metaurl in encryptOption (fluid-cloudnative#24) --signoff Signed-off-by: zwwhdls <[email protected]> * fix golangci-lint Signed-off-by: zwwhdls <[email protected]> * fix typo Signed-off-by: zwwhdls <[email protected]> * add test in util Signed-off-by: zwwhdls <[email protected]> * add tests & fix Signed-off-by: zwwhdls <[email protected]> * add tests Signed-off-by: zwwhdls <[email protected]> * add tests Signed-off-by: zwwhdls <[email protected]> * disassemble cmd/main.go into several small files. Signed-off-by: zwwhdls <[email protected]> Co-authored-by: ldd91 <[email protected]> Co-authored-by: cheyang <[email protected]> Co-authored-by: LDawn <[email protected]> Co-authored-by: Yifan Yuan <[email protected]> Co-authored-by: littletiger123 <[email protected]> Co-authored-by: frankleaf <[email protected]> Co-authored-by: yangyuliufeng <[email protected]>
shiwenwenya · Nov 22, 2021 · c3d10e3 · c3d10e3
1 parent eb79559
commit c3d10e3
Show file tree

Hide file tree

Showing 108 changed files with 15,900 additions and 3 deletions.
diff --git a/Makefile b/Makefile
@@ -8,6 +8,7 @@ DATASET_CONTROLLER_IMG ?= ${IMG_REPO}/dataset-controller
 ALLUXIORUNTIME_CONTROLLER_IMG ?= ${IMG_REPO}/alluxioruntime-controller
 JINDORUNTIME_CONTROLLER_IMG ?= ${IMG_REPO}/jindoruntime-controller
 GOOSEFSRUNTIME_CONTROLLER_IMG ?= ${IMG_REPO}/goosefsruntime-controller
+JUICEFSRUNTIME_CONTROLLER_IMG ?= ${IMG_REPO}/juicefsruntime-controller
 CSI_IMG ?= ${IMG_REPO}/fluid-csi
 LOADER_IMG ?= ${IMG_REPO}/fluid-dataloader
 INIT_USERS_IMG ?= ${IMG_REPO}/init-users
@@ -51,7 +52,7 @@ unit-test: generate fmt vet
 
 # Build binary
 
-build: dataset-controller-build alluxioruntime-controller-build jindoruntime-controller-build csi-build webhook-build
+build: dataset-controller-build alluxioruntime-controller-build jindoruntime-controller-build juicefsruntime-controller-build csi-build webhook-build
 
 csi-build: generate fmt vet
 	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=off  go build -o bin/fluid-csi -ldflags '${LDFLAGS}' cmd/csi/main.go
@@ -68,6 +69,9 @@ jindoruntime-controller-build: generate fmt vet
 goosefsruntime-controller-build: generate fmt vet
 	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=off  go build -gcflags="-N -l" -a -o bin/goosefsruntime-controller -ldflags '${LDFLAGS}' cmd/goosefs/main.go
 
+juicefsruntime-controller-build: generate fmt vet
+	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=off  go build -gcflags="-N -l" -a -o bin/juicefsruntime-controller -ldflags '-s -w ${LDFLAGS}' cmd/juicefs/main.go
+
 webhook-build: generate fmt vet
 	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=off  go build -gcflags="-N -l" -a -o bin/fluid-webhook -ldflags '${LDFLAGS}' cmd/webhook/main.go
 
@@ -128,6 +132,9 @@ docker-build-jindoruntime-controller: generate fmt vet
 docker-build-goosefsruntime-controller: generate fmt vet
 	docker build --no-cache . -f docker/Dockerfile.goosefsruntime -t ${GOOSEFSRUNTIME_CONTROLLER_IMG}:${GIT_VERSION}
 
+docker-build-juicefsruntime-controller: generate fmt vet juicefsruntime-controller-build
+	docker build --no-cache . -f docker/Dockerfile.juicefsruntime -t ${JUICEFSRUNTIME_CONTROLLER_IMG}:${GIT_VERSION}
+
 docker-build-csi: generate fmt vet
 	docker build --no-cache . -f docker/Dockerfile.csi -t ${CSI_IMG}:${GIT_VERSION}
 
@@ -153,6 +160,9 @@ docker-push-jindoruntime-controller: docker-build-jindoruntime-controller
 docker-push-goosefsruntime-controller: docker-build-goosefsruntime-controller
 	docker push ${GOOSEFSRUNTIME_CONTROLLER_IMG}:${GIT_VERSION}
 
+docker-push-juicefsruntime-controller: docker-build-juicefsruntime-controller
+	docker push ${JUICEFSRUNTIME_CONTROLLER_IMG}:${GIT_VERSION}
+
 docker-push-csi: docker-build-csi
 	docker push ${CSI_IMG}:${GIT_VERSION}
 

diff --git a/api/v1alpha1/juicefsruntime_types.go b/api/v1alpha1/juicefsruntime_types.go
@@ -0,0 +1,157 @@
+/*
+Copyright 2021 The Fluid Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package v1alpha1
+
+import (
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+// EDIT THIS FILE!  THIS IS SCAFFOLDING FOR YOU TO OWN!
+// NOTE: json tags are required.  Any new fields you add must have json tags for the fields to be serialized.
+
+// JuiceFSRuntimeSpec defines the desired state of JuiceFSRuntime
+type JuiceFSRuntimeSpec struct {
+	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
+	// Important: Run "make" to regenerate code after modifying this file
+
+	// The version information that instructs fluid to orchestrate a particular version of JuiceFS.
+	JuiceFSVersion VersionSpec `json:"juicefsVersion,omitempty"`
+
+	// The spec of init users
+	InitUsers InitUsersSpec `json:"initUsers,omitempty"`
+
+	// The component spec of JuiceFS master
+	Master JuiceFSCompTemplateSpec `json:"master,omitempty"`
+
+	// The component spec of JuiceFS worker
+	Worker JuiceFSCompTemplateSpec `json:"worker,omitempty"`
+
+	// The component spec of JuiceFS job Worker
+	JobWorker JuiceFSCompTemplateSpec `json:"jobWorker,omitempty"`
+
+	// Desired state for JuiceFS Fuse
+	Fuse JuiceFSFuseSpec `json:"fuse,omitempty"`
+
+	// Tiered storage used by JuiceFS
+	TieredStore TieredStore `json:"tieredstore,omitempty"`
+
+	// The replicas of the worker, need to be specified
+	Replicas int32 `json:"replicas,omitempty"`
+
+	// Manage the user to run Juicefs Runtime
+	RunAs *User `json:"runAs,omitempty"`
+
+	// Disable monitoring for JuiceFS Runtime
+	// Prometheus is enabled by default
+	// +optional
+	DisablePrometheus bool `json:"disablePrometheus,omitempty"`
+}
+
+// JuiceFSCompTemplateSpec is a description of the JuiceFS components
+type JuiceFSCompTemplateSpec struct {
+	// Replicas is the desired number of replicas of the given template.
+	// If unspecified, defaults to 1.
+	// +kubebuilder:validation:Minimum=1
+	// replicas is the min replicas of dataset in the cluster
+	// +optional
+	Replicas int32 `json:"replicas,omitempty"`
+
+	// Ports used by JuiceFS
+	// +optional
+	Ports []corev1.ContainerPort `json:"ports,omitempty"`
+
+	// Resources that will be requested by the JuiceFS component.
+	// +optional
+	Resources corev1.ResourceRequirements `json:"resources,omitempty"`
+
+	// Environment variables that will be used by JuiceFS component.
+	Env []corev1.EnvVar `json:"env,omitempty"`
+
+	// Enabled or Disabled for the components.
+	// +optional
+	Enabled bool `json:"enabled,omitempty"`
+
+	// NodeSelector is a selector
+	// +optional
+	NodeSelector map[string]string `json:"nodeSelector,omitempty"`
+}
+
+type JuiceFSFuseSpec struct {
+	// Image for JuiceFS fuse
+	Image string `json:"image,omitempty"`
+
+	// Image for JuiceFS fuse
+	ImageTag string `json:"image_tag,omitempty"`
+
+	// One of the three policies: `Always`, `IfNotPresent`, `Never`
+	ImagePullPolicy string `json:"image_pull_policy,omitempty"`
+
+	// Environment variables that will be used by JuiceFS Fuse
+	Env []corev1.EnvVar `json:"env,omitempty"`
+
+	// Resources that will be requested by JuiceFS Fuse.
+	Resources corev1.ResourceRequirements `json:"resources,omitempty"`
+
+	// If the fuse client should be deployed in global mode,
+	// otherwise the affinity should be considered
+	// +optional
+	Global bool `json:"global,omitempty"`
+
+	// NodeSelector is a selector which must be true for the fuse client to fit on a node,
+	// this option only effect when global is enabled
+	// +optional
+	NodeSelector map[string]string `json:"node_selector,omitempty"`
+}
+
+//+kubebuilder:object:root=true
+//+kubebuilder:subresource:status
+// +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.currentWorkerNumberScheduled,selectorpath=.status.selector
+// +kubebuilder:printcolumn:name="Ready Workers",type="integer",JSONPath=`.status.workerNumberReady`,priority=10
+// +kubebuilder:printcolumn:name="Desired Workers",type="integer",JSONPath=`.status.desiredWorkerNumberScheduled`,priority=10
+// +kubebuilder:printcolumn:name="Worker Phase",type="string",JSONPath=`.status.workerPhase`,priority=0
+// +kubebuilder:printcolumn:name="Ready Fuses",type="integer",JSONPath=`.status.fuseNumberReady`,priority=10
+// +kubebuilder:printcolumn:name="Desired Fuses",type="integer",JSONPath=`.status.desiredFuseNumberScheduled`,priority=10
+// +kubebuilder:printcolumn:name="Fuse Phase",type="string",JSONPath=`.status.fusePhase`,priority=0
+// +genclient
+
+// JuiceFSRuntime is the Schema for the juicefsruntimes API
+type JuiceFSRuntime struct {
+	metav1.TypeMeta   `json:",inline"`
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+
+	Spec   JuiceFSRuntimeSpec `json:"spec,omitempty"`
+	Status RuntimeStatus      `json:"status,omitempty"`
+}
+
+//+kubebuilder:object:root=true
+
+// JuiceFSRuntimeList contains a list of JuiceFSRuntime
+type JuiceFSRuntimeList struct {
+	metav1.TypeMeta `json:",inline"`
+	metav1.ListMeta `json:"metadata,omitempty"`
+	Items           []JuiceFSRuntime `json:"items"`
+}
+
+func init() {
+	SchemeBuilder.Register(&JuiceFSRuntime{}, &JuiceFSRuntimeList{})
+}
+
+// Replicas gets the replicas of runtime worker
+func (r *JuiceFSRuntime) Replicas() int32 {
+	return r.Spec.Replicas
+}
diff --git a/api/v1alpha1/openapi_generated.go b/api/v1alpha1/openapi_generated.go
diff --git a/api/v1alpha1/zz_generated.deepcopy.go b/api/v1alpha1/zz_generated.deepcopy.go