forked from ourfavoritefruits/breakpad
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add optional new symbol upload API to sym_upload.
Change-Id: I6a49e9f4a699fa6f5f8e9f0fc86afb4cb342a442 Reviewed-on: https://chromium-review.googlesource.com/c/breakpad/breakpad/+/1422400 Reviewed-by: Mark Mentovai <[email protected]> Reviewed-by: Ivan Penkov <[email protected]> Reviewed-by: Mike Frysinger <[email protected]>
- Loading branch information
Nelson Billing
committed
Feb 20, 2020
1 parent
216cea7
commit bbad9f2
Showing
11 changed files
with
875 additions
and
94 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,214 @@ | ||
| # Introduction | ||
|
|
||
| The `sym_upload` tool is able to operate in `sym-upload-v2` protocol mode, in | ||
| addition to the legacy protocol (which will be referred to as `sym-upload-v1` | ||
| for the rest of this document). For now `sym-upload-v2` is HTTP/REST-based but | ||
| it could be extended to operate over gRPC instead, in the future. | ||
|
|
||
| # Table of Contents | ||
| * [Why](#why) | ||
| * [How](#how) | ||
| * [Uploading](#uploading) | ||
| * [Uploading with `sym_upload`](#uploading-with-sym_upload) | ||
| * [Uploading with curl](#uploading-with-curl) | ||
| * [Serving the `sym-upload-v2` protocol](#serving-the-sym-upload-v2-protocol) | ||
| * [Authenticate using `key`](#authenticate-using-key) | ||
| * [Symbol `checkStatus`](#symbol-checkstatus) | ||
| * [Upload `create`](#upload-create) | ||
| * [Uploading the symbol file](#uploading-the-symbol-file) | ||
| * [Upload complete](#upload-complete) | ||
|
|
||
|
|
||
| # Why | ||
|
|
||
| Using `sym_upload` in `sym-upload-v2` protocol mode has the following features | ||
| beyond `sym-upload-v1`: | ||
| * Authentication via `key` (arbitrary secret). | ||
| * Symbol identifier (product of `debug_file` and `debug_id`, as recorded in | ||
| output from `dump_syms`) can be checked against existing symbol information on | ||
| server. If it's present, then the upload is skipped entirely. | ||
|
|
||
| # How | ||
|
|
||
| ## Uploading | ||
|
|
||
| ### Uploading with `sym_upload` | ||
|
|
||
| Uploading in `sym-upload-v2` protocol mode is easy. Invoke `sym_upload` like | ||
| ``` | ||
| $ ./sym_upload -p sym-upload-v2 [-k <API-key>] <symbol-file> <API-URL> | ||
| ``` | ||
|
|
||
| Where `symbol-file` is a symbol file created by `dump_syms`, `API-URL` is the | ||
| URL of your `sym-upload-v2` API service (see next section for details), and | ||
| `API-key` is a secret known to your uploader and server. | ||
|
|
||
| For more options see `sym_upload --help`. | ||
|
|
||
| ### Uploading with curl | ||
|
|
||
| As an example, if: | ||
| * Your API's URL was "https://sym-upload-api". | ||
| * Your service has assigned you `key` "myfancysecret123". | ||
| * You wanted to upload the symbol file at "path/to/file_name", with | ||
| `debug_file` being "file_name" and `debug_id` being | ||
| "123123123123123123123123123". Normally you would read these values from | ||
| "path/to/file_name", which in turn was generated by `dump_syms`. | ||
|
|
||
| Then you might run: | ||
| ``` | ||
| $ curl https://sym-upload-api/symbols/file_name/123123123123123123123123123:checkStatus?key=myfancysecret123 | ||
| ``` | ||
|
|
||
| And, upon seeing that this `debug_file`/`debug_id` combo is missing from symbol | ||
| storage then you could run: | ||
| ``` | ||
| $ curl --request POST https://sym-upload-api/uploads:create?key=myfancysecret123 | ||
| ``` | ||
|
|
||
| Which returns `upload_url` "https://upload-server/42?creds=shhhhh" and | ||
| `upload_key` "42". Next you upload the file directly like: | ||
| ``` | ||
| $ curl -T path/to/file_name "https://upload-server/42?creds=shhhhh" | ||
| ``` | ||
|
|
||
| Once the HTTP PUT is complete, run: | ||
| ``` | ||
| $ curl --header "Content-Type: application/json" \ | ||
| --request POST \ | ||
| --data '{symbol_id:{"debugFile":"file_name",'\ | ||
| '"debugId":"123123123123123123123123123"}}' \ | ||
| https://sym-upload-api/uploads/42:complete?key=myfancysecret123 | ||
| ``` | ||
|
|
||
| ### Serving the `sym-upload-v2` Protocol | ||
|
|
||
| The protocol is currently defined only in HTTP/REST. There are three necessary | ||
| REST operations to implement in your service: | ||
| * `/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>` | ||
| * `/uploads:create?key=<key>` | ||
| * `/uploads/<upload_key>:complete?key=<key>` | ||
|
|
||
| #### Authenticate Using `key` | ||
|
|
||
| The query string arg `key` contains some secret that both the uploader and | ||
| server understand. It is up to the service implementer to decide on what | ||
| constitutes a valid `key`, how the uploader acquires one, and how to handle | ||
| requests made with invalid ones. | ||
|
|
||
| #### Symbol `checkStatus` | ||
|
|
||
| ``` | ||
| /symbols/<debug_file>/<debug_id>:checkStatus?key=<key> | ||
| ``` | ||
|
|
||
| This operation expects an empty (or no) JSON payload in the request. | ||
|
|
||
| This operation should return the status of the symbol file uniquely identified | ||
| by the given `debug_file` and `debug_id`. JSON schema: | ||
| ``` | ||
| { | ||
| "type": object", | ||
| "properties": { | ||
| "status": { | ||
| "type": "string", | ||
| "enum": ["STATUS_UNSPECIFIED", "MISING", "FOUND"], | ||
| "required": true | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Where `MISSING` denotes that the symbol file does not exist on the server and | ||
| `FOUND` denotes that the symbol file exists on the server. | ||
|
|
||
| #### Upload `create` | ||
|
|
||
| ``` | ||
| /uploads:create?key=<key> | ||
| ``` | ||
|
|
||
| This operation expects an empty (or no) JSON payload in the request. | ||
|
|
||
| This operation should return a URL that uploader can HTTP PUT their symbol file | ||
| to, along with an "upload key" that can be used to notify the service once the | ||
| file upload is completed. JSON schema: | ||
| ``` | ||
| { | ||
| "type": "object", | ||
| "properties": { | ||
| "upload_url": { | ||
| "type: "string", | ||
| "required": true | ||
| }, | ||
| "upload_key": { | ||
| "type": "string", | ||
| "required": true | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Since this REST API operation can be authenticated via the `key` query string | ||
| arg, the service can return a URL that encodes permission delegation to the | ||
| upload endpoint resource and thereby constrain the ability to upload to those | ||
| with valid `key`s. | ||
|
|
||
| #### Uploading the Symbol File | ||
|
|
||
| Note that the actual symbol upload step is _not_ part of the REST API. The | ||
| upload URL obtained in the above operation is meant to be used as the endpoint | ||
| for a normal HTTP PUT request for the contents of the symbol file. Once that | ||
| HTTP PUT request is completed use the upload `complete` operation. | ||
|
|
||
| #### Upload `complete` | ||
|
|
||
| ``` | ||
| /uploads/<upload_key>:complete?key=<key> | ||
| ``` | ||
|
|
||
| This operation expects a JSON payload in the HTTP request body with the | ||
| following schema: | ||
| ``` | ||
| { | ||
| "type": "object", | ||
| "properties": { | ||
| "symbol_id": { | ||
| "type": "object", | ||
| "properties": { | ||
| "debug_file": { | ||
| "type": "string", | ||
| "required": true | ||
| }, | ||
| "debug_id": { | ||
| "type": "string", | ||
| "required": true | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| This operation should cause the symbol storage back-end (however implemented) | ||
| to consume the symbol file identified by `upload_key`. It is up to the service | ||
| implementation to decide how uploads are assigned `upload_key`s and how to | ||
| retrieve a completed upload by its `upload_key`. If the symbol file cannot be | ||
| found, is malformed, or the operation cannot be completed for any other reason | ||
| then an HTTP error will be returned. JSON schema of non-error responses: | ||
| ``` | ||
| { | ||
| "type": "object", | ||
| "properties": { | ||
| "result": { | ||
| "type": string, | ||
| "enum": ["RESULT_UNSPECIFIED", "OK", "DUPLICATE_DATA"], | ||
| "required": true | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Where `OK` denotes that the symbol storage was updated with the new symbol file | ||
| and `DUPLICATE_DATA` denotes that the symbol file data was identical to data | ||
| already in symbol storage and therefore nothing changed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.