Skip to content

Commit

Permalink
Document pipeline CSVW variables.
Browse files Browse the repository at this point in the history 
Document the CSVW pipelines used by pipelines which can be referenced
within URI template (not all columns are necessarily documented).

Document the Propertize transformation in the docs.
  • Loading branch information
@lkitching
lkitching committed Jan 21, 2019
1 parent 1d345b5 commit b75f2f5
Show file tree
Hide file tree
Showing 3 changed files with 38 additions and 12 deletions.
12 changes: 12 additions & 0 deletions doc/usage.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,18 @@ For example the text "date of birth" is converted into `DateOfBirth`.

Note this transformation is only used internally for generating some URIs and is not a valid value for the `value_transformation` in the data cube columns configuration.

#### Propertize

The `propertize` transformation is defined as:

1. Lower-case the first letter of the first word
2. Upper-case the first letter of all other words
3. Remove the whitespace around words

For example the test "date of birth" is converted into `dateOfBirth`.

Note this transformation is only used internally for generating some URIs and is not a valid value for the `value_transformation` in the data cube columns configuration.

## Customising URIs

Table2qb defines default conventions for the structure of URIs for generated resources, for example the default structure of a component URI was shown above:
Expand Down
23 changes: 18 additions & 5 deletions resources/table2qb-config.edn
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,9 @@
:example "uris.edn"
:optional? true}]
:uris-resource "uris/cube-pipeline-uris.edn"
:uri-vars {base-uri "Base URI"
dataset-slug "Provided slug for the dataset"}}
:template-vars {base-uri "Base URI"
dataset-slug "Provided slug for the dataset"}
:csvw-vars {}}
:table2qb.pipelines.components/components-pipeline {:parameters [{:name input-csv
:description "File containing component definitions"
:type :file
Expand All @@ -41,7 +42,14 @@
:example "uris.edn"
:optional? true}]
:uris-resource "uris/components-pipeline-uris.edn"
:uri-vars {base-uri "Base URI"}}
:template-vars {base-uri "Base URI"}
:csvw-vars {label "Component label"
description "Component description"
component_type "Component property e.g. qb:Dimension, qb:Measure"
codelist "Optional codelist URI"
component_type_slug "Slugized version of Component Type input value"
property_slug "Propertized version of the component Label"
class_slug "Classized version of the component Label"}}
:table2qb.pipelines.codelist/codelist-pipeline {:parameters [{:name codelist-csv
:description "File containing codelist data"
:type :file
Expand All @@ -64,8 +72,13 @@
:example "uris.edn"
:optional? true}]
:uris-resource "uris/codelist-pipeline-uris.edn"
:uri-vars {base-uri "Base URI"
codelist-slug "Provided slug for the codelist"}}
:template-vars {base-uri "Base URI"
codelist-slug "Provided slug for the codelist"}
:csvw-vars {label "Code Label"
notation "Code Notation"
parent_notation "Optional parent code"
sort_priority "Optional code Sort Priority"
description "Optional code Description"}}

:table2qb.pipelines/pipelines [#ig/ref :table2qb.pipelines.cube/cube-pipeline
#ig/ref :table2qb.pipelines.components/components-pipeline
Expand Down
15 changes: 8 additions & 7 deletions src/table2qb/cli/tasks.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -242,27 +242,28 @@
(throw (ex-info "Pipeline name required"
{:error-lines ["Usage: table2qb describe pipeline-name"]}))))

(defn- load-user-uris-file [file-str]
;;TODO: handle file errors, validate loaded EDN
(util/read-edn (io/file file-str)))

(defmethod exec-task :uris [{:keys [pipelines] :as uri-task} _all-tasks [pipeline-name uris-file & _ignored]]
(if (some? pipeline-name)
(if-let [{:keys [uris-resource uri-vars] :as pipeline} (find-pipeline pipelines pipeline-name)]
(if-let [{:keys [uris-resource template-vars csvw-vars] :as pipeline} (find-pipeline pipelines pipeline-name)]
(if (some? uris-file)
(let [resolved-uris (uri-config/resolve-uri-defs (io/resource uris-resource) (io/file uris-file))
rows (cons ["Name" "Template"] (map (fn [[key uri]] [(str " " key) uri]) resolved-uris))]
(doseq [row (pad-rows rows)]
(println (row->string row))))
(let [uris (util/read-edn (io/resource uris-resource))
var-rows (cons ["Name" "Description"] (util/map-keys name uri-vars))
var-rows (cons ["Name" "Description"] (util/map-keys name template-vars))
uri-rows (cons ["Name" "Default"] (map (fn [[key uri]] [(str " " key) uri]) uris))]
(println "URIs:")
(doseq [row (pad-rows uri-rows)]
(println (row->string row)))
(println)
(println "Template variables:")
(doseq [row (pad-rows var-rows)]
(println (row->string row)))))
(println (row->string row)))
(println)
(println "CSVW variables:")
(let [csvw-uri-rows (cons ["Name" "Description"] (util/map-keys name csvw-vars))]
(doseq [row (pad-rows csvw-uri-rows)]
(println (row->string row))))))
(unknown-pipeline pipelines pipeline-name))
(describe-task uri-task)))

0 comments on commit b75f2f5

Please sign in to comment.