Skip to content

Commit

Permalink
All Makefile proto commands use Docker (cosmos#7931)
Browse files Browse the repository at this point in the history 
* Update tooling around docker

* Run commands

* Remove swagger gen for now

* Update proto-gen-any

* Fix indentn

* Use CURDIR

Co-authored-by: Alessio Treglia <[email protected]>
  • Loading branch information
@amaury1093
amaury1093 and Alessio Treglia authored Nov 13, 2020
1 parent 6e3fab8 commit a8ef4a3
Show file tree
Hide file tree
Showing 35 changed files with 489 additions and 374 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/proto.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ jobs:
steps:
- uses: actions/checkout@master
- name: lint
run: make proto-lint-docker
run: make proto-lint
breakage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- name: check-breakage
run: make proto-check-breaking-docker
run: make proto-check-breaking
1 change: 0 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ docs/node_modules
docs/modules
dist
tools-stamp
proto-tools-stamp
buf-stamp
artifacts

Expand Down
14 changes: 9 additions & 5 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -142,11 +142,15 @@ build, in which case we can fall back on `go mod tidy -v`.

We use [Protocol Buffers](https://developers.google.com/protocol-buffers) along with [gogoproto](https://github.com/gogo/protobuf) to generate code for use in Cosmos-SDK.

For determinstic behavior around Protobuf tooling, everything is containerized using Docker. Make sure to have Docker installed on your machine, or head to [Docker's website](https://docs.docker.com/get-docker/) to install it.

For formatting code in `.proto` files, you can run `make proto-format` command.

For linting and checking breaking changes, we use [buf](https://buf.build/). There are two options for linting and to check if your changes will cause a break. The first is that you can install [buf](https://buf.build/docs/installation) locally, the commands for running buf after installing are `make proto-lint` and the breaking changes check will be `make proto-check-breaking`. If you do not want to install buf and have docker installed already then you can use these commands `make proto-lint-docker` and `make proto-check-breaking-docker`.
For linting and checking breaking changes, we use [buf](https://buf.build/). You can use the commands `make proto-lint` and `make proto-check-breaking` to respectively lint your proto files and check for breaking changes.

To generate the protobuf stubs, you can run `make proto-gen`.

To generate the protobuf stubs you must have `protoc` and `protoc-gen-gocosmos` installed. To install these tools run `make proto-tools`. After this step you will be able to run `make proto-gen` to generate the protobuf stubs.
We also added the `make proto-all` command to run all the above commands sequentially.

In order for imports to properly compile in your IDE, you may need to manually set your protobuf path in your IDE's workspace settings/config.

Expand Down Expand Up @@ -268,8 +272,8 @@ and PRs are merged into `master`, if a contributor wishes the PR to be released
- **[Impact]** Explanation of how the bug affects users or developers.
- **[Test Case]** section with detailed instructions on how to reproduce the bug.
- **[Regression Potential]** section with a discussion how regressions are most likely to manifest, or might
manifest even if it's unlikely, as a result of the change. **It is assumed that any SRU candidate PR is
well-tested before it is merged in and has an overall low risk of regression**.
manifest even if it's unlikely, as a result of the change. **It is assumed that any SRU candidate PR is
well-tested before it is merged in and has an overall low risk of regression**.
It is the PR's author's responsibility to fix merge conflicts, update changelog entries, and
ensure CI passes. If a PR originates from an external contributor, it may be a core team member's
Expand All @@ -280,7 +284,7 @@ Finally, when a point release is ready to be made:
1. Create `release/v0.38.N` branch
2. Ensure changelog entries are verified
2. Be sure changelog entries are added to `RELEASE_CHANGELOG.md`
1. Be sure changelog entries are added to `RELEASE_CHANGELOG.md`
3. Add release version date to the changelog
4. Push release branch along with the annotated tag: **git tag -a**
5. Create a PR into `master` containing ONLY `CHANGELOG.md` updates
Expand Down
27 changes: 7 additions & 20 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ BUILDDIR ?= $(CURDIR)/build
SIMAPP = ./simapp
MOCKS_DIR = $(CURDIR)/tests/mocks
HTTPS_GIT := https://github.com/cosmos/cosmos-sdk.git
DOCKER_BUF := docker run -v $(shell pwd):/workspace --workdir /workspace bufbuild/buf
DOCKER_BUF := docker run -v $(CURDIR):/workspace --workdir /workspace bufbuild/buf

export GO111MODULE = on

Expand Down Expand Up @@ -356,43 +356,30 @@ devdoc-update:
### Protobuf ###
###############################################################################

proto-all: proto-tools proto-gen proto-lint proto-check-breaking proto-swagger-gen proto-format
proto-all: proto-format proto-lint proto-check-breaking proto-gen

proto-gen:
@./scripts/protocgen.sh

proto-gen-docker:
@echo "Generating Protobuf files"
docker run -v $(shell pwd):/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen.sh
.PHONY: proto-gen-docker
docker run -v $(CURDIR):/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen.sh

proto-format:
@echo "Formatting Protobuf files"
docker run -v $(shell pwd):/workspace \
docker run -v $(CURDIR):/workspace \
--workdir /workspace tendermintdev/docker-build-proto \
find ./ -not -path "./third_party/*" -name *.proto -exec clang-format -i {} \;
.PHONY: proto-format

# This generates the SDK's custom wrapper for google.protobuf.Any. It should only be run manually when needed
proto-gen-any:
@./scripts/protocgen-any.sh
docker run -v $(CURDIR):/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen-any.sh

proto-swagger-gen:
@./scripts/protoc-swagger-gen.sh

proto-lint:
@buf check lint --error-format=json

proto-check-breaking:
@buf check breaking --against-input '.git#branch=master'

proto-lint-docker:
@$(DOCKER_BUF) check lint --error-format=json
.PHONY: proto-lint

proto-check-breaking-docker:
proto-check-breaking:
@$(DOCKER_BUF) check breaking --against-input $(HTTPS_GIT)#branch=master
.PHONY: proto-check-breaking-ci

TM_URL = https://raw.githubusercontent.com/tendermint/tendermint/v0.34.0-rc6/proto/tendermint
GOGO_PROTO_URL = https://raw.githubusercontent.com/regen-network/protobuf/cosmos
Expand Down Expand Up @@ -445,7 +432,7 @@ proto-update-deps:
## Issue link: https://github.com/confio/ics23/issues/32
@sed -i '4ioption go_package = "github.com/confio/ics23/go";' $(CONFIO_TYPES)/proofs.proto

.PHONY: proto-all proto-gen proto-lint proto-check-breaking proto-update-deps
.PHONY: proto-all proto-gen proto-gen-any proto-swagger-gen proto-format proto-lint proto-check-breaking proto-update-deps

###############################################################################
### Localnet ###
Expand Down
239 changes: 239 additions & 0 deletions client/docs/swagger-ui/swagger.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19992,6 +19992,221 @@ paths:
type: boolean
tags:
- Query
/ibc/client/v1beta1/params:
get:
summary: ClientParams queries all parameters of the ibc client.
operationId: ClientParams
responses:
'200':
description: A successful response.
schema:
type: object
properties:
params:
description: params defines the parameters of the module.
type: object
properties:
allowed_clients:
type: array
items:
type: string
description: >-
allowed_clients defines the list of allowed client state
types.
description: >-
QueryClientParamsResponse is the response type for the
Query/ClientParams RPC method.
default:
description: An unexpected error response.
schema:
type: object
properties:
error:
type: string
code:
type: integer
format: int32
message:
type: string
details:
type: array
items:
type: object
properties:
type_url:
type: string
description: >-
A URL/resource name that uniquely identifies the type of
the serialized

protocol buffer message. This string must contain at
least

one "/" character. The last segment of the URL's path
must represent

the fully qualified name of the type (as in

`path/google.protobuf.Duration`). The name should be in
a canonical form

(e.g., leading "." is not accepted).


In practice, teams usually precompile into the binary
all types that they

expect it to use in the context of Any. However, for
URLs which use the

scheme `http`, `https`, or no scheme, one can optionally
set up a type

server that maps type URLs to message definitions as
follows:


* If no scheme is provided, `https` is assumed.

* An HTTP GET on the URL must yield a
[google.protobuf.Type][]
value in binary format, or produce an error.
* Applications are allowed to cache lookup results based
on the
URL, or have them precompiled into a binary to avoid any
lookup. Therefore, binary compatibility needs to be preserved
on changes to types. (Use versioned type names to manage
breaking changes.)

Note: this functionality is not currently available in
the official

protobuf release, and it is not used for type URLs
beginning with

type.googleapis.com.


Schemes other than `http`, `https` (or the empty scheme)
might be

used with implementation specific semantics.
value:
type: string
format: byte
description: >-
Must be a valid serialized protocol buffer of the above
specified type.
description: >-
`Any` contains an arbitrary serialized protocol buffer
message along with a

URL that describes the type of the serialized message.


Protobuf library provides support to pack/unpack Any values
in the form

of utility functions or additional generated methods of the
Any type.


Example 1: Pack and unpack a message in C++.

Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
...
}

Example 2: Pack and unpack a message in Java.

Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
foo = any.unpack(Foo.class);
}

Example 3: Pack and unpack a message in Python.

foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
any.Unpack(foo)
...

Example 4: Pack and unpack a message in Go

foo := &pb.Foo{...}
any, err := ptypes.MarshalAny(foo)
...
foo := &pb.Foo{}
if err := ptypes.UnmarshalAny(any, foo); err != nil {
...
}

The pack methods provided by protobuf library will by
default use

'type.googleapis.com/full.type.name' as the type URL and the
unpack

methods only use the fully qualified type name after the
last '/'

in the type URL, for example "foo.bar.com/x/y.z" will yield
type

name "y.z".



JSON

====

The JSON representation of an `Any` value uses the regular

representation of the deserialized, embedded message, with
an

additional field `@type` which contains the type URL.
Example:

package google.profile;
message Person {
string first_name = 1;
string last_name = 2;
}

{
"@type": "type.googleapis.com/google.profile.Person",
"firstName": <string>,
"lastName": <string>
}

If the embedded message type is well-known and has a custom
JSON

representation, that representation will be embedded adding
a field

`value` which holds the custom JSON in addition to the
`@type`

field. Example (for message [google.protobuf.Duration][]):

{
"@type": "type.googleapis.com/google.protobuf.Duration",
"value": "1.212s"
}
tags:
- Query
/ibc/core/client/v1beta1/client_states:
get:
summary: ClientStates queries all the IBC light clients of a chain.
Expand Down Expand Up @@ -33015,6 +33230,30 @@ definitions:
description: >-
ConsensusStateWithHeight defines a consensus state with an additional
height field.
ibc.core.client.v1.Params:
type: object
properties:
allowed_clients:
type: array
items:
type: string
description: allowed_clients defines the list of allowed client state types.
description: Params defines the set of IBC light client parameters.
ibc.core.client.v1.QueryClientParamsResponse:
type: object
properties:
params:
description: params defines the parameters of the module.
type: object
properties:
allowed_clients:
type: array
items:
type: string
description: allowed_clients defines the list of allowed client state types.
description: >-
QueryClientParamsResponse is the response type for the Query/ClientParams
RPC method.
ibc.core.client.v1.QueryClientStateResponse:
type: object
properties:
Expand Down
Loading

0 comments on commit a8ef4a3

Please sign in to comment.