Skip to content

Commit

Permalink
[BEAM-3005] Add resource limits to provision proto
Browse files Browse the repository at this point in the history
  • Loading branch information
herohde committed Oct 26, 2017
1 parent d5aff5d commit 28fbb01
Show file tree
Hide file tree
Showing 2 changed files with 177 additions and 21 deletions.
38 changes: 38 additions & 0 deletions model/fn-execution/src/main/proto/beam_provision_api.proto
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,42 @@ message ProvisionInfo {
// (required) Pipeline options. For non-template jobs, the options are
// identical to what is passed to job submission.
google.protobuf.Struct pipeline_options = 3;

// (optional) Resource limits that the SDK harness worker should respect.
// Runners may -- but are not required to -- enforce any limits provided.
Resources resource_limits = 4;
}

// Resources specify limits for local resources, such memory and cpu. It
// is used to inform SDK harnesses of their allocated footprint.
message Resources {
// Memory limits.
message Memory {
// (optional) Hard limit in bytes. A zero value means unspecified.
uint64 size = 1;

// TOOD(herohde) 10/20/2017: consider soft limits, shm usage?
}
// (optional) Memory usage limits. SDKs can use this value to configure
// internal buffer sizes and language specific sizes.
Memory memory = 1;

// CPU limits.
message Cpu {
// (optional) Shares of a cpu to use. Fractional values, such as "0.2"
// or "2.5", are fine. Any value <= 0 means unspecified.
float shares = 1;

// TODO(herohde) 10/20/2017: consider cpuset?
}
// (optional) CPU usage limits.
Cpu cpu = 2;

// Disk limits.
message Disk {
// (optional) Hard limit in bytes. A zero value means unspecified.
uint64 size = 1;
}
// (optional) Disk size limits for the semi-persistent location.
Disk semi_persistent_disk = 3;
}
160 changes: 139 additions & 21 deletions sdks/go/pkg/beam/model/fnexecution_v1/beam_provision_api.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 28fbb01

Please sign in to comment.