Skip to content

Commit

Permalink
Updating standard naming to JobPack. Adding initial samples and schema.
Browse files Browse the repository at this point in the history
  • Loading branch information
Jonathan Meyer committed Jun 4, 2016
1 parent 385657d commit b10a0a4
Show file tree
Hide file tree
Showing 6 changed files with 264 additions and 26 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
## JetBrains IDE directorys:
.idea/
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# cjds
Containerized Job Definition Standard
# JobPack
JobPack - containerized job package standard
24 changes: 10 additions & 14 deletions detail.adoc
Original file line number Diff line number Diff line change
@@ -1,35 +1,31 @@
= Containerized Job Definition Standard Details
= JobPack Details
Jonathan Meyer <jon@gisjedi.com>
v0.0.1, 2016-05-26
:toc:

== Introduction

The Containerized Job Definition Standard (CJDS) is a general standard to aid in the discovery and consumption of
JobPack is a general standard to aid in the discovery and consumption of
discrete units of work contained within a Docker image. While initially developed to support the Scale data processing
system with job discovery, it is designed to be readily applied to other systems as well.

CJDS compliant images must also be named in a specific fashion due to the lack of label search capability on Docker Hub
and Registry services. The suffix `-cjds` must be used when naming images to enable discovery, prior to Hub or Registry
JobPack compliant images must be named in a specific fashion due to the lack of label search capability on Docker Hub
and Registry services. The suffix `-jobpack` must be used when naming images to enable discovery, prior to Hub or Registry
push. This requirement may be lifted in future versions of the standard if support is provided for label search by
Docker Hub and Registry APIs.

=== Examples

Dockerfile snippet containing required labels for CJDS compliance:
Dockerfile snippet containing required labels for JobPack compliance:

----
FROM centos
LABEL com.cjds.version="0.0.1" \
com.cjds.name="watermark-image" \
com.cjds.description="Job to add a watermark to an existing image" \
com.cjds.system="Scale" \
com.cjds.manifest="{...}"
LABEL com.jobpack.version="0.0.1"
LABEL com.jobpack.manifest="{...}"
----

The com.cjds.manifest label should contain the system specific job manifest. The following is a JSON example of the Scale
job-type manifest and must be serialized as a string-escaped value of the label:
The com.jobpack.manifest label contents must be serialized as a string-escaped value:

----
{
Expand Down Expand Up @@ -116,7 +112,7 @@ offerings and the following sections describe the steps that may be taken to acc

=== Docker Hub

https://hub.docker.com/v2/search/repositories/?query=-cjds
https://hub.docker.com/v2/search/repositories/?query=-jobpack

Use https://github.com/CenturyLinkLabs/imagelayers[ImageLayers] /registry/analyze endpoint to extrapolate the Dockerfile
contents and retrieve label metadata.
Expand All @@ -134,7 +130,7 @@ http://myreg.org/v2/{image-id}/manifests/{tag}

== Labels

TODO: Give details on contents and formatting of all CJDS labels
TODO: Give details on contents and formatting of all JobPack labels

== Manifest

Expand Down
75 changes: 75 additions & 0 deletions examples/jobpack-full.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
[
{
"name": "my-algorithm",
"version": "1.0.0",
"title": "My first algorithm",
"description": "Reads an HDF5 file and outputs two TIFF images and a CSV",
"author_name": "John_Doe",
"author_url": "http://www.example.com",
"timeout": 3600,
"cpus_required": 10.0,
"mem_required": 10240.0,
"disk_out_const_required": 0.0,
"disk_out_mult_required": 0.0,
"interface": {
"output_data": [
{
"media_type": "image/tiff",
"required": true,
"type": "file",
"name": "output_file_tif",
"glob": "outfile1*.tif"
},
{
"media_type": "image/tiff",
"required": true,
"type": "file",
"name": "output_file_tif2",
"glob": "outfile2*.tif"
},
{
"media_type": "text/csv",
"required": true,
"type": "file",
"name": "output_file_csv",
"glob": "outfile*.csv"
}
],
"command_arguments": "${input_file} ${job_output_dir}",
"input_data": [
{
"media_types": [
"image/x-hdf5-image"
],
"required": true,
"type": "file",
"name": "input_file"
}
],
"command": "/app/my_wrapper.sh"
},
"error_mapping": [
{
"code": 1,
"name": "ErrorName",
"title": "Error Title",
"description": "Error Description",
"category": "SYSTEM"
},
{
"code": 2,
"name": "ErrorName",
"title": "Error Title",
"description": "Error Description",
"category": "DATA"
},
{
"code": 3,
"name": "ErrorName",
"title": "Error Title",
"description": "Error Description",
"category": "ALGORITHM"
}
]
}
]
17 changes: 7 additions & 10 deletions index.adoc
Original file line number Diff line number Diff line change
@@ -1,23 +1,20 @@
= Containerized Job Definition Standard
= JobPack
Jonathan Meyer <jon@gisjedi.com>
v0.0.1, 2016-05-26
:toc:

The Containerized Job Definition Standard (CJDS) is a general standard to aid in the discovery and consumption of
JobPack is a general standard to aid in the discovery and consumption of
discrete units of work contained within a Docker image.

To comply with the CJDS a Docker image must have at a minimum 5 labels:
To comply with the JobPack standard a Docker image must have at a minimum 2 labels:

----
LABEL com.cjds.version="0.0.1"
LABEL com.cjds.name="watermark-image"
LABEL com.cjds.description="Job to add a watermark to an existing image"
LABEL com.cjds.system="Scale"
LABEL com.cjds.manifest="{...}"
LABEL com.jobpack.version="0.0.1"
LABEL com.jobpack.manifest="{...}"
----

CJDS compliant images must also be named in a specific fashion due to the lack of label search capability on Docker Hub
and Registry services. The suffix `-cjds` must be used when naming images to enable discovery, prior to Hub or Registry
JobPack compliant images must be named in a specific fashion due to the lack of label search capability on Docker Hub
and Registry services. The suffix `-jobpack` must be used when naming images to enable discovery, prior to Hub or Registry
push.

See the http://detail[full specification] for more detail.
Expand Down
168 changes: 168 additions & 0 deletions schema/jobpack.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"version": {
"type": "string"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
},
"author_name": {
"type": "string"
},
"author_url": {
"type": "string"
},
"timeout": {
"type": "integer"
},
"cpus_required": {
"type": "number"
},
"mem_required": {
"type": "number"
},
"disk_out_const_required": {
"type": "number"
},
"disk_out_mult_required": {
"type": "number"
},
"interface": {
"type": "object",
"properties": {
"output_data": {
"type": "array",
"items": {
"type": "object",
"properties": {
"media_type": {
"type": "string"
},
"required": {
"type": "boolean",
"default": true
},
"type": {
"type": "string"
},
"name": {
"type": "string"
},
"glob": {
"type": "string"
}
},
"required": [
"media_type",
"type",
"name",
"glob"
]
}
},
"command_arguments": {
"type": "string"
},
"input_data": {
"type": "array",
"items": {
"type": "object",
"properties": {
"media_types": {
"type": "array",
"items": {
"type": "string"
}
},
"required": {
"type": "boolean"
},
"type": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"media_types",
"required",
"type",
"name"
]
}
},
"command": {
"type": "string"
}
},
"required": [
"output_data",
"command_arguments",
"input_data",
"command"
]
},
"error_mapping": {
"type": "array",
"items": {
"type": "object",
"properties": {
"code": {
"type": "integer"
},
"name": {
"type": "string"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
},
"category": {
"type": "string",
"enum": [
"ALGORITHM",
"DATA",
"SYSTEM"
]
}
},
"required": [
"code",
"name",
"title",
"description",
"category"
]
}
}
},
"required": [
"name",
"version",
"title",
"description",
"author_name",
"author_url",
"timeout",
"cpus_required",
"mem_required",
"disk_out_const_required",
"disk_out_mult_required",
"interface",
"error_mapping"
]
}
}

0 comments on commit b10a0a4

Please sign in to comment.