-
Notifications
You must be signed in to change notification settings - Fork 76
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
rework validate using official jsonschema
* Remove concept of `sections` throughout. * Removed custom json validation framework, switch to official `jsonschema` * Rewrite meta schema. * `validate()` is run on file write automatically * **API CHANGE** SigMFFile.validate() no longer returns `True` or `ValidationError`. Will return `None` or raise `ValidationError` -> increment minor version * Improve validation testing. * **Issue** Unable to get defaults working in schema definition, but when that occurs `get_default_metadata` for SigMFFile should be created dynamically. * **Issue** There are no tests for the collection schema. * **Issue** The schema can be improved dramatically.
- Loading branch information
Showing
14 changed files
with
618 additions
and
550 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,68 +1,171 @@ | ||
| { | ||
| "collection": { | ||
| "required": false, | ||
| "type": "dict", | ||
| "sort": "core:sample_start", | ||
| "keys": { | ||
| "core:version": { | ||
| "type": "string", | ||
| "required": true, | ||
| "default": null, | ||
| "help": "The version of the SigMF specification used to create the collection file." | ||
| }, | ||
| "core:description": { | ||
| "type": "string", | ||
| "required": false, | ||
| "help": "A text description of the SigMF collection." | ||
| }, | ||
| "core:author": { | ||
| "type": "string", | ||
| "required": false, | ||
| "help": "The author's name (and optionally e-mail address) of the form \"Bruce Wayne [email protected]\"." | ||
| }, | ||
| "core:collection_doi": { | ||
| "type": "string", | ||
| "required": false, | ||
| "help": "The registered DOI (ISO 26324) for a collection." | ||
| }, | ||
| "core:license": { | ||
| "type": "string", | ||
| "required": false, | ||
| "help": "A URL for the license document under which the collection is offered; when possible, use the canonical document provided by the license author, or, failing that, a well-known one." | ||
| }, | ||
| "core:hagl": { | ||
| "type": "list", | ||
| "required": false, | ||
| "help": "Recording Tuple pointing to a F32 type dataset representing aperture HAGL." | ||
| }, | ||
| "core:extensions": { | ||
| "required": false, | ||
| "help": "A list of extensions used by this collection.", | ||
| "type": "dict_list", | ||
| "sort": "name", | ||
| "keys": { | ||
| "name": { | ||
| "type": "string", | ||
| "required": true, | ||
| "help": "The name of the extension." | ||
| }, | ||
| "version": { | ||
| "type": "string", | ||
| "required": true, | ||
| "help": "The version of the extension." | ||
| }, | ||
| "optional": { | ||
| "type": "boolean", | ||
| "required": true, | ||
| "help": "Whether or not this extension is required to parse this collection." | ||
| "$id": "https://github.com/gnuradio/SigMF", | ||
| "$schema": "http://json-schema.org/draft-07/schema", | ||
| "default": {}, | ||
| "required": [ | ||
| "collection" | ||
| ], | ||
| "type": "object", | ||
| "properties": { | ||
| "collection": { | ||
| "$id": "#/properties/collection", | ||
| "default": {}, | ||
| "description": "This field is used to indicate that this Recording is part of a SigMF Collection (described later in this document). It is strongly RECOMMENDED that if you are building a Collection, that each Recording referenced by that Collection use this field to associate up to the relevant sigmf-collection file.", | ||
| "required": [ | ||
| "core:version" | ||
| ], | ||
| "type": "object", | ||
| "properties": { | ||
| "core:version": { | ||
| "$id": "#/properties/collection/properties/core%3Aversion", | ||
| "description": "The version of the SigMF specification used to create the Collection file.", | ||
| "examples": [ | ||
| "1.0.0" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "core:description": { | ||
| "$id": "#/properties/collection/properties/core%3Adescription", | ||
| "default": "", | ||
| "description": "A text description of the SigMF Collection.", | ||
| "type": "string" | ||
| }, | ||
| "core:author": { | ||
| "$id": "#/properties/collection/properties/core%3Aauthor", | ||
| "default": "", | ||
| "description": "A text identifier for the author potentially including name, handle, email, and/or other ID like Amateur Call Sign.", | ||
| "examples": [ | ||
| "Bruce Wayne [email protected]", | ||
| "Bruce (K3X)" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "core:collection_doi": { | ||
| "$id": "#/properties/collection/properties/core%3Acollection_doi", | ||
| "default": "", | ||
| "description": "The registered DOI (ISO 26324) for a Collection.", | ||
| "type": "string" | ||
| }, | ||
| "core:license": { | ||
| "$id": "#/properties/collection/properties/core%3Alicense", | ||
| "default": "", | ||
| "description": "A URL for the license document under which this Collection metadata is offered.", | ||
| "examples": [ | ||
| "https://creativecommons.org/licenses/by-sa/4.0/" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "core:extensions": { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions", | ||
| "default": [], | ||
| "description": "The `core:extensions` field in the Global Object is an array of extension objects that describe SigMF extensions. Extension Objects MUST contain the three key/value pairs defined below, and MUST NOT contain any other fields.", | ||
| "type": "array", | ||
| "additionalItems": true, | ||
| "items": { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions/items", | ||
| "anyOf": [ | ||
| { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions/items/anyOf/0", | ||
| "type": "object", | ||
| "title": "The first anyOf schema", | ||
| "description": "An explanation about the purpose of this instance.", | ||
| "default": {}, | ||
| "examples": [ | ||
| { | ||
| "name": "capture_details", | ||
| "version": "1.0.0", | ||
| "optional": false | ||
| } | ||
| ], | ||
| "required": [ | ||
| "name", | ||
| "version", | ||
| "optional" | ||
| ], | ||
| "properties": { | ||
| "name": { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions/items/anyOf/0/properties/name", | ||
| "default": "", | ||
| "description": "The name of the SigMF extension namespace.", | ||
| "type": "string" | ||
| }, | ||
| "version": { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions/items/anyOf/0/properties/version", | ||
| "default": "", | ||
| "description": "The version of the extension namespace specification used.", | ||
| "type": "string" | ||
| }, | ||
| "optional": { | ||
| "$id": "#/properties/collection/properties/core%3Aextensions/items/anyOf/0/properties/optional", | ||
| "default": false, | ||
| "description": "If this field is `true`, the extension is REQUIRED to parse this Recording.", | ||
| "type": "boolean" | ||
| } | ||
| }, | ||
| "additionalProperties": true | ||
| } | ||
| ] | ||
| } | ||
| }, | ||
| "core:streams": { | ||
| "$id": "#/properties/collection/properties/core%3Astreams", | ||
| "default": [], | ||
| "description": "An ordered array of SigMF Recording Tuples, indicating multiple recorded streams of data (e.g., channels from a phased array).", | ||
| "type": "array", | ||
| "additionalItems": true, | ||
| "items": { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items", | ||
| "anyOf": [ | ||
| { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/0", | ||
| "default": [], | ||
| "examples": [ | ||
| [ | ||
| "example-channel-0-basename", | ||
| "hash" | ||
| ] | ||
| ], | ||
| "type": "array", | ||
| "additionalItems": true, | ||
| "items": { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/0/items", | ||
| "anyOf": [ | ||
| { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/0/items/anyOf/0", | ||
| "default": "", | ||
| "type": "string" | ||
| } | ||
| ] | ||
| } | ||
| }, | ||
| { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/1", | ||
| "default": [], | ||
| "examples": [ | ||
| [ | ||
| "example-channel-1-basename", | ||
| "hash" | ||
| ] | ||
| ], | ||
| "type": "array", | ||
| "additionalItems": true, | ||
| "items": { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/1/items", | ||
| "anyOf": [ | ||
| { | ||
| "$id": "#/properties/collection/properties/core%3Astreams/items/anyOf/1/items/anyOf/0", | ||
| "default": "", | ||
| "type": "string" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| }, | ||
| "core:streams": { | ||
| "type": "list", | ||
| "required": false, | ||
| "help": "The base filename of a `collection` with which this Recording is associated." | ||
| } | ||
| "additionalProperties": true | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "additionalProperties": true | ||
| } |
Oops, something went wrong.
c014ce1There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes to schema-meta.json have removed the 'core:geolocation' parameter from the global section.
c014ce1There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be nice to see some tags or releases. Ex. release the previous commit, 92b0ffb, as v1.0.1. Would be much easier to see when breaking changes happen to the API instead of having to look through commits.
This commit broke my CI/CD because
validate()no longer returnsTrue. Which is fine as the API changed, but took me a little bit to track down where things went wrong.c014ce1There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion, it is not falling on deaf ears. The python module has not historically received the level of API stability and release rigor that we would like to see and the project has several plans to address that. I did take your suggestion and tag that commit as v1.0.1, though this wont be pushed to pip. There will be a v1.1.0 release very soon though, and we will be splitting the python and specification repositories immediately after that release which will help with some of the other management issues.