`/Workspace` not prepended automatically when root_path starts with a variable #2181

angus-dawson-idexx · 2025-01-17T21:57:29Z

Describe the issue

If workspace.root_path starts with a variable, it is not automatically prepended with /Workspace.

Configuration

# databricks.yml

[...]

workspace:
  profile: profile_name
  root_path: ${var.workspace_root}/path/to/root

variables:
  workspace_root:
    description: "root directory in the Databricks workspace to store the asset bundle and associated artifacts"
    default: /Users/${workspace.current_user.userName}

targets:
  dev:
    default: true
  prod:
    variables:
      workspace_root: /Shared

resources:
  jobs:
    my_job:
      tasks:
        - python_wheel_task:
            named_parameters:
              conf-file: "${workspace.file_path}/path/to/config.yaml"
      [...]

Steps to reproduce the behavior

Run databricks bundle validate -t dev -o json > validated.json
Check the value of conf-file in validated.json

Expected Behavior

The value of conf-file should be prepended with /Workspace, because the resolved value of root_path after variable substitution does not start with /Volumes.

Actual Behavior

The value conf-file starts with /Users. If /Workspace is manually prepended in the value of conf-file in the bundle configuration instead, databricks bundle validate emits a warning and removes it, leaving just /Users/... again.

OS and CLI version

macOS, CLI v0.239.0

Is this a regression?

Unknown, did not try previous versions

Debug Logs

Debug Log (redacted)


13:53:49  INFO start pid=63830 version=0.239.0 args="/Users/<...>/Library/Caches/pypoetry/virtualenvs/<...>/bin/databricks, bundle, validate, -t, dev, --log-level=debug"
13:53:49 DEBUG Found bundle root at /Users/<...>/<...> (file /Users/<...>/<...>/databricks.yml) pid=63830
13:53:49 DEBUG Apply pid=63830 mutator=load
13:53:49  INFO Phase: load pid=63830 mutator=load
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=EntryPoint
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=scripts.preinit
13:53:49 DEBUG No script defined for preinit, skipping pid=63830 mutator=load mutator=seq mutator=scripts.preinit
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=ProcessRootIncludes
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=ProcessRootIncludes mutator=seq
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=VerifyCliVersion
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=EnvironmentsToTargets
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=ComputeIdToClusterId
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=InitializeVariables
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=DefineDefaultTarget(default)
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=PythonMutator(load)
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=validate:unique_resource_keys
13:53:49 DEBUG Apply pid=63830 mutator=load mutator=seq mutator=SelectTarget(dev)
13:53:49 DEBUG Apply pid=63830 mutator=
13:53:49 DEBUG Apply pid=63830 mutator=initialize
13:53:49  INFO Phase: initialize pid=63830 mutator=initialize
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=validate:AllResourcesHaveValues
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=RewriteSyncPaths
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=SyncDefaultPath
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=SyncInferRoot
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=InitializeWorkspaceClient
13:53:49 DEBUG Loading <...> profile from /Users/<...>/.databrickscfg pid=63830 sdk=true
13:53:49 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PopulateCurrentUser
13:53:49 DEBUG Loading <...> profile from /Users/<...>/.databrickscfg pid=63830 sdk=true
13:53:50 DEBUG GET /api/2.0/preview/scim/v2/Me
< HTTP/2.0 200 OK
< {
<   "active": true,
<   "displayName": "<...>",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "<...>"
<     }
<   ],
<   "groups": [
<     <...>
<   ],
<   "id": "2110990998431807",
<   "name": {
<     "familyName": "<...>",
<     "givenName": "<...>,"
<   },
<   "roles": [
<     {
<       "value": "arn:aws:iam::<...>:instance-profile/DataBricks-EC2-Role"
<     }
<   ],
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "urn:ietf:params:scim:schemas:extension:workspace:2.0:User"
<   ],
<   "userName": "<...>"
< } pid=63830 mutator=initialize mutator=seq mutator=PopulateCurrentUser sdk=true
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=LoadGitDetails
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ApplySourceLinkedDeploymentPreset
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=DefineDefaultWorkspaceRoot
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ExpandWorkspaceRoot
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=DefaultWorkspacePaths
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PrependWorkspacePrefix
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=RewriteWorkspacePrefix
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=SetVariables
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PythonMutator(init)
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PythonMutator(load_resources)
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PythonMutator(apply_mutators)
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ResolveVariableReferences
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ResolveResourceReferences
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ResolveVariableReferences
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=MergeJobClusters
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=MergeJobParameters
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=MergeJobTasks
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=MergePipelineClusters
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=MergeApps
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=CaptureSchemaDependency
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=CheckPermissions
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=SetRunAs
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=OverrideCompute
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ConfigureDashboardDefaults
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ConfigureVolumeDefaults
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ProcessTargetMode
13:53:50  INFO Development mode: disabling deployment lock since bundle.deployment.lock.enabled is not set to true pid=63830 mutator=initialize mutator=seq mutator=ProcessTargetMode
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ApplyPresets
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=DefaultQueueing
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ExpandPipelineGlobPaths
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ConfigureWSFS
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=TranslatePaths
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=PythonWrapperWarning
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=apps.Validate
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ValidateSharedRootPermissions
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=ApplyBundlePermissions
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=FilterCurrentUserFromPermissions
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=metadata.AnnotateJobs
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=metadata.AnnotatePipelines
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=terraform.Initialize
13:53:50 DEBUG DATABRICKS_TF_EXEC_PATH is not defined pid=63830 mutator=initialize mutator=seq mutator=terraform.Initialize
13:53:50 DEBUG Using Terraform at /Users/<...>/<...>/bin/terraform pid=63830 mutator=initialize mutator=seq mutator=terraform.Initialize
13:53:50 DEBUG DATABRICKS_TF_CLI_CONFIG_FILE is not defined pid=63830 mutator=initialize mutator=seq mutator=terraform.Initialize
13:53:50 DEBUG Environment variables for Terraform: TMPDIR, DATABRICKS_USER_AGENT_EXTRA, DATABRICKS_HOST, DATABRICKS_TOKEN, DATABRICKS_CLI_PATH, DATABRICKS_AUTH_TYPE, HOME, PATH pid=63830 mutator=initialize mutator=seq mutator=terraform.Initialize
13:53:50 DEBUG Apply pid=63830 mutator=initialize mutator=seq mutator=scripts.postinit
13:53:50 DEBUG No script defined for postinit, skipping pid=63830 mutator=initialize mutator=seq mutator=scripts.postinit
13:53:50 DEBUG Apply pid=63830 mutator=validate
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:folder_permissions
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly)
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:validate_sync_patterns
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly) mutator (read-only)=parallel
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly) mutator (read-only)=parallel mutator (read-only)=validate:artifact_paths
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly) mutator (read-only)=parallel mutator (read-only)=validate:SingleNodeCluster
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly) mutator (read-only)=parallel mutator (read-only)=validate:job_task_cluster_spec
13:53:50 DEBUG ApplyReadOnly pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=fast_validate(readonly) mutator (read-only)=parallel mutator (read-only)=validate:job_cluster_key_defined
13:53:50 DEBUG GET /api/2.0/workspace/get-status?path=/Users/<...>/<...>/files
< HTTP/2.0 200 OK
< {
<   "object_id": 3161334096948787,
<   "object_type": "DIRECTORY",
<   "path": "/Users/<...>/<...>/files",
<   "resource_id": "3161334096948787"
< } pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync sdk=true
13:53:50 DEBUG Path /Users/<...>/<...>/files has type directory (ID: 3161334096948787) pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:files_to_sync
13:53:50 DEBUG GET /api/2.0/workspace/get-status?path=/Users/<...>/<...>
< HTTP/2.0 200 OK
< {
<   "object_id": 3161334096948786,
<   "object_type": "DIRECTORY",
<   "path": "/Users/<...>/<...>",
<   "resource_id": "3161334096948786"
< } pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:folder_permissions sdk=true
13:53:50 DEBUG GET /api/2.0/permissions/directories/3161334096948786
< HTTP/2.0 200 OK
< {
<   "access_control_list": [
<     {
<       "all_permissions": [
<         {
<           "inherited": true,
<           "inherited_from_object": [
<             "/directories/1491521541917617"
<           ],
<           "permission_level": "CAN_MANAGE"
<         }
<       ],
<       "display_name": "<...>",
<       "user_name": "<...>"
<     },
<     {
<       "all_permissions": [
<         {
<           "inherited": false,
<           "permission_level": "CAN_RUN"
<         }
<       ],
<       "display_name": "<...>",
<       "service_principal_name": "<...>"
<     },
<     {
<       "all_permissions": [
<         {
<           "inherited": false,
<           "permission_level": "CAN_MANAGE"
<         }
<       ],
<       "group_name": "<...>"
<     },
<     {
<       "all_permissions": [
<         {
<           "inherited": true,
<           "inherited_from_object": [
<             "/directories/"
<           ],
<           "permission_level": "CAN_MANAGE"
<         }
<       ],
<       "group_name": "admins"
<     }
<   ],
<   "object_id": "/directories/3161334096948786",
<   "object_type": "directory"
< } pid=63830 mutator=validate mutator (read-only)=parallel mutator (read-only)=validate:folder_permissions sdk=true
Warning: substring "/Workspace/${workspace.file_path}" found in "/Workspace/${workspace.file_path}/<...>/config.yaml". Please update this to "${workspace.file_path}/<...>/config.yaml".
  at resources.jobs.<...>.tasks[0].python_wheel_task.package_name.this-task-run-id
  in databricks.yml:133:26
For more information, please refer to: https://docs.databricks.com/en/release-notes/dev-tools/bundles.html#workspace-paths
Warning: untracked permissions apply to target workspace path
The following permissions apply to the workspace folder at "/Users/<...>/<...>" but are not configured in the bundle:

level: CAN_MANAGE, user_name: <...>

Name: deployment

Target: dev

Workspace:

User: <...>

Path: /Users/<...>/<...>

Found 2 warnings 13:53:50 INFO completed execution pid=63830 exit_code=0

The text was updated successfully, but these errors were encountered:

andrewnester · 2025-01-20T12:49:51Z

Indeed, this is expected as DABs does not interpolate the values of variables before adding /Workspace syntax.
Can you just change the value for root_path variable instead?

variables:
  workspace_root:
    description: "root directory in the Databricks workspace to store the asset bundle and associated artifacts"
    default: /Workspace/Users/${workspace.current_user.userName}

angus-dawson-idexx · 2025-01-20T17:09:24Z

Indeed, this is expected as DABs does not interpolate the values of variables before adding /Workspace syntax. Can you just change the value for root_path variable instead?
variables:
  workspace_root:
    description: "root directory in the Databricks workspace to store the asset bundle and associated artifacts"
    default: /Workspace/Users/${workspace.current_user.userName}

Yes, that is what I ended up doing to work around this.

If this is intended behavior, I think it should be documented somewhere.

andrewnester · 2025-01-21T09:22:40Z

@angus-dawson-idexx we have the change documented here in the release notes https://docs.databricks.com/en/release-notes/dev-tools/bundles.html#workspace-paths-are-now-automatically-prefixed

But I agree that we might expand it even more, will pass the feedback to the corresponding team, thank you!

angus-dawson-idexx · 2025-01-23T20:08:08Z

@andrewnester I don't think that release notes page really describes what I encountered above--it discusses the automatic /Workspace prefixing generally, and warns about manually prefixing them when they are used as variables elsewhere, but doesn't say that it won't work if the value itself starts with a variable, which is the issue here.

andrewnester · 2025-01-24T10:58:01Z

@angus-dawson-idexx yes, exactly, hence me mentioning that we should have documented this behaviour in the docs as well so it's less confusing. At the moment we're looking at all things variables and might revisit this behaviour, so we keep this issue open until more updates

angus-dawson-idexx added the DABs DABs related issues label Jan 17, 2025

denik self-assigned this Jan 20, 2025

denik added a commit that referenced this issue Jan 20, 2025

Add regression test for #2181

aa1c50a

pietern mentioned this issue Jan 20, 2025

Add test for #2181 /Workspace not prepended #2188

Merged

andrewnester closed this as completed Jan 21, 2025

denik added a commit that referenced this issue Jan 21, 2025

add link to #2181 in script

9e147a1

github-merge-queue bot pushed a commit that referenced this issue Jan 21, 2025

Add test for #2181 /Workspace not prepended (#2188)

33613b5

andrewnester reopened this Jan 21, 2025

andrewnester mentioned this issue Feb 6, 2025

Warning when using workspace.file_path variable in base_parameters #2300

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`/Workspace` not prepended automatically when root_path starts with a variable #2181

`/Workspace` not prepended automatically when root_path starts with a variable #2181

angus-dawson-idexx commented Jan 17, 2025

andrewnester commented Jan 20, 2025

angus-dawson-idexx commented Jan 20, 2025

andrewnester commented Jan 21, 2025

angus-dawson-idexx commented Jan 23, 2025

andrewnester commented Jan 24, 2025

/Workspace not prepended automatically when root_path starts with a variable #2181

/Workspace not prepended automatically when root_path starts with a variable #2181

Comments

angus-dawson-idexx commented Jan 17, 2025

Describe the issue

Configuration

Steps to reproduce the behavior

Expected Behavior

Actual Behavior

OS and CLI version

Is this a regression?

Debug Logs

andrewnester commented Jan 20, 2025

angus-dawson-idexx commented Jan 20, 2025

andrewnester commented Jan 21, 2025

angus-dawson-idexx commented Jan 23, 2025

andrewnester commented Jan 24, 2025

`/Workspace` not prepended automatically when root_path starts with a variable #2181

`/Workspace` not prepended automatically when root_path starts with a variable #2181