From c9fb06f1ac5e0f51cbd9de79f19905bfa7491819 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Tue, 22 Aug 2017 19:41:28 -0700 Subject: [PATCH 1/7] Propose adopting the Rust RFC process I believe it is time to adopt a new development process given the current scale of Kubernetes. The inadequacy of our current process is particularly apparent within SIGs with cross project concerns such as SIG Release, SIG Testing, SIG Architecture, and SIG PM so additional process around proposing changes to Kubernetes is suggested. --- .../standardization-of-development-process.md | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) create mode 100644 contributors/design-proposals/standardization-of-development-process.md diff --git a/contributors/design-proposals/standardization-of-development-process.md b/contributors/design-proposals/standardization-of-development-process.md new file mode 100644 index 00000000000..3a6ec404735 --- /dev/null +++ b/contributors/design-proposals/standardization-of-development-process.md @@ -0,0 +1,282 @@ +# Proposal to Standardize Development Process + +## Links +``` +--- +Links: +- Issues: + - [someIssueURL]() +- PRs: + - [somePullRequestURL]() +- MailingListDiscussions: + - [someGoogleGroupsURL]() +- Documentation: + - [someDocsLinkURL]() +``` + +## Responsible SIG(s) + +- SIG Release +- SIG PM +- SIG Architecture +- SIG Governance +- Technical Steering Committee + +## Summary + +A standardized development process for Kubernetes is proposed in order to: +- provide a common structure for proposing changes to Kubernetes +- ensure that the motivation for a change is clear +- allow for the enumeration stability milestones and stability graduation + criteria +- persist project information in a Version Control System (VCS) for future + Kubernauts +- support the creation of _high value user facing_ information such as: + - release notes + - release announcement blog + - a cross project development road map +- support development across multiple repositories beyond `kubernetes/kubernetes` +- begin to move away from using GitHub issues for feature requests and reserve them + for tracking work in flight + +## Motivation + +Today it is becoming fairly difficult to track changes being made across all +Special Interest Groups (SIGs) within Kubernetes. For cross project SIGs such +as SIG PM and SIG Release an abstraction beyond a single GitHub Issue or Pull +request seems to be required in order to understand and communicate upcoming +changes to Kubernetes. Particularly the generation of release notes and the +release announcement blog post are rather difficult and have sometimes delayed +a release due to incompleteness. In a blog post describing the [road to Go 2][], +Russ Cox explains + +> that it is difficult but essential to describe the significance of a problem +> in a way that someone working in a different environment can understand + +unfortunately I believe that our current mechanism for proposing changes by +opening a GitHub issue against `kubernetes/features` falls short of providing +an effective vehicle for agreeing on the motivation and the design of a proposed +change. + +The use of GitHub issues when proposing changes does not provide SIGs good +facilities for signaling approval or rejection of a proposed change to Kubernetes +since anyone can open a GitHub issue at any time. Additionally managing a proposed +change across multiple releases is somewhat cumbersome as labels and milestones +need to be updated for every release that a change spans which leads to an ever +increasing number of issues open against `kubernetes/features` which itself has +become a management problem. + +We have chosen to use Git as our VCS with GitHub currently used to host our +repositories. While GitHub provides a rich feature set as an open source project, +managing issues over time and searching for text within an issue can be +challenging. The flat hierarchy of issues can also make navigation and +categorization tricky. While not all community members might not be comfortable +using Git directly, I believe it is imperative that as a community we work to +educate people on a standard set of tools so they can take their experience to +other projects they may decide to work on in the future. + +It is my hope that by increasing the requirements for proposing a change to +Kubernetes we can prevent important work like generating release notes from +falling out of the release cycle and in general make working with and speaking +about Kubernetes an easier and more pleasant experience. + +[road to Go 2]: https://blog.golang.org/toward-go2 + +## Detailed Design + +### RFC Template + +In order to implement an RFC like process a template of the following form would +be created + +``` +# Title +## Links +## Responsible SIG(s) +## Summary +## Motivation +## Examples [optional] +## Detailed Design +## Graduation Criteria +## Drawbacks [optional] +## Alternatives [optional] +## Unresolved Questions [optional] +## Mentors [optional] +``` + +where sections marked `[optional]` may be omitted by RFC authors.It is expected +that most RFCs will begin their lives in discussion with the responsible SIGs, +preferably in a mailing list so that the discussion is saved for future Kubernauts. + +It may be very helpful to provide links to experience reports of other Kubernetes +users and developers within the `Motivation` section in order to provide additional +context for the necessity of a proposed change. The `Examples` section could be +used to provide motivating examples for documentation writers looking to teach +others how to consume the proposed change once implemented. As an open source +community we must constantly be looking for ways to better serve newcomers and +a `Mentors` section could be used to help connect interested parties with elders +in the community who are able to mentor people towards an implementation. + +It is intended for RFCs to be a conversation, the comment part of the acronym, +so it is not expected for every section of the template to be completed by the +time an RFC is filed, however, the `Summary`, `Motivation`, and `Examples` section +should probably be filled out by the time an RFC is filed. Merging an RFC only +signals agreement of the sections which have been provided thus far and an RFC +may be completed through a few rounds of review in order to achieve alignment. + +### Git and GitHub Implementation + +Practically an RFC would be implemented as a pull request to a central repository +with the following example structure + +``` +├── 0000-rfc-template.md +├── CODEOWNERS +├── index.md +├── sig-architecture +├── sig-network +│   └── kube-dns +├── sig-node +│   └── kubelet +├── sig-release +├── sig-storage +├── unsorted-to-be-used-by-newcomers-only +└── wg-resource-management +``` + +where each SIG or working group is given a top level directory with components +maintained by the SIG listed in sub directories. For newcomers to the community +an `unsorted-to-be-used-by-newcomers-only` directory may be used before an RFC +can be properly routed to a SIG although hopefully if discussion for a potential +RFC begins on the mailing lists proper routing information will be provided to +the RFC author. Additionally a top level index of RFCs may be helpful for people +looking for a complete list of RFCs. There should be basic CI to ensure that a +`index.md` remains up to date. + +Ideally no work would begin on an RFC before it has been approved by the +responsible SIG or working group. In order to help combat the explosion of GitHub +issues against `kubernetes/kubernetes` it is further proposed for each chartered +SIG to create a top level repository within the `kubernetes` GitHub organization +where implementation issues would be filed and linked to in the RFC. + +### RFC Editor Role + +Taking a cue from the [Python PEP process][], I believe that a group of RFC editors +will be required to make this process successful; the job of an RFC editor is +likely very similar to the [PEP editor responsibilities][] and will hopefully +provide another opportunity for people who do not write code daily to contribute +to Kubernetes. + +[Python PEP process]: https://www.python.org/dev/peps/pep-0001/ +[PEP editor responsibilities]: https://www.python.org/dev/peps/pep-0001/#pep-editor-responsibilities-workflow + +### Important Metrics + +I believe the primary metrics which would signal the success or failure of the RFC +process would be + +- distribution of RFC merge times +- RFC rejection rate +- distribution of time between RFC acceptance and implementation issue creation +- PRs referencing an RFC merged per week + +### Customization of the RFC Process + +Given the general independence of SIGs I believe it will be important for SIGs to +decide if they would like to opt out of the RFC process entirely or if they would +like to modify the RFC process to better serve their needs. For example I can +imagine RFCs targeting SIG Architecture or SIG API Machinery would likely require +a section on backwards compatibility and migration strategies. I believe the best +place for a SIG to describe its development process would be in its charter. + +### Prior Art + +The RFC process as proposed was essentially stolen from the [Rust RFC process] which +itself seems to be very similar to the [Python PEP process][] + +[Rust RFC process]: https://github.com/rust-lang/rfcs + +## Graduation Criteria + +Before we can consider this process successful for our current size I believe we +should hit at least the following milestones + +- a release note draft can be generated by referring to RFC content +- a road map can be generated by referring to RFC content + +The process for creating draft release notes and development status reports +should eventually rely largely on automation, but I do not believe that the +implementation of such automation should prevent the RFC process from being +considered stable. I can certainly imagine a future where a release notes draft +is produced nightly along with the collection of binaries that users deploy. + +## Drawbacks + +Any additional process has the potential to engender resentment within the +community. There is also a risk that the RFC process as designed will not +sufficiently address the scaling challenges we face today. PR review bandwidth is +already at a premium and we may find that the RFC process introduces an unreasonable +bottleneck on our development velocity. + +It certainly can be argued that the lack of a dedicated issue/defect tracker +beyond GitHub issues contributes to our challenges in managing a project as large +as Kubernetes, however, given that other large organizations, including GitHub +itself, make effective use of GitHub issues perhaps the argument is overblown. + +The centrality of Git and GitHub within the RFC process also may place too high +a barrier to potential contributors, however, given that both Git and GitHub are +required to contribute code changes to Kubernetes today I would argue that we +should focus on providing support to those unfamiliar with this tooling. + +Expanding the proposal template beyond the single sentence description currently +required in the [features issue template][] may be a heavy burden for non native +English speakers and here I believe that the role of the RFC editor combined with +kindness and empathy will be crucial to making the process successful. + +[features issue template]: https://github.com/kubernetes/features/blob/master/ISSUE_TEMPLATE.md + +## Alternatives + +This RFC process is related to +- the generation of our [architectural road map][] +- the fact that the [what constitutes a feature][] is still undefined +- how we [manage issues][] +- the difference between an [accepted design and a proposal][] +- [the organization of design proposals][] + +and I believe that this RFC process attempts to place these concerns within a +more general framework + +[architectural road map]: https://github.com/kubernetes/community/issues/952 +[what constitutes a feature]: https://github.com/kubernetes/community/issues/531 +[manage issues]: https://github.com/kubernetes/community/issues/580 +[accepted design and a proposal]: https://github.com/kubernetes/community/issues/914 +[the organization of design proposals]: https://github.com/kubernetes/community/issues/918 + +## Unresolved Questions + +Whether we believe it makes sense for a SIG repository to serve a mini mono +repository where code lives in a structure like + +``` +kubernetes:sig-node/pkg/kublet +``` + +or whether SIGs maintain a list of repositories maintained by the SIG in an index +like + +``` +kubernetes:sig-node/repositories.md +``` + +is not clear to me. I believe that users would generally think about Kubernetes +from a component standpoint if they are aware of the component at all but I have +no real data to support that assertion. + +## Mentors + +- caleb miles + - github: [calebamiles](https://github.com/calebamiles/) + - slack: [calebamiles](https://coreos.slack.com/team/caleb.miles) + - email: [caleb.miles@coreos.com](mailto:caleb.miles@coreos.com) + - pronoun: "he" From 78873a56a8677d92d76bd0b76b121784431b63b4 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Tue, 22 Aug 2017 19:57:33 -0700 Subject: [PATCH 2/7] Update links in proposal to adopt RFC process Adds links to Google Groups discussion and self link to PR --- .../standardization-of-development-process.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contributors/design-proposals/standardization-of-development-process.md b/contributors/design-proposals/standardization-of-development-process.md index 3a6ec404735..b8f5e6202c0 100644 --- a/contributors/design-proposals/standardization-of-development-process.md +++ b/contributors/design-proposals/standardization-of-development-process.md @@ -7,9 +7,9 @@ Links: - Issues: - [someIssueURL]() - PRs: - - [somePullRequestURL]() + - https://github.com/kubernetes/community/pull/967 - MailingListDiscussions: - - [someGoogleGroupsURL]() + - https://groups.google.com/forum/#!topic/kubernetes-dev/65A-3ULYPB0 - Documentation: - [someDocsLinkURL]() ``` From c9764bd65be12f9c2abd4a9e2796a90c172c4edd Mon Sep 17 00:00:00 2001 From: caleb miles Date: Thu, 31 Aug 2017 16:09:19 -0700 Subject: [PATCH 3/7] extract KEP template from KEP proposal and address comments - RFC -> KEP - attempt to remove first person tone - add possible states of a KEP --- .../kep-development-process.md | 356 ++++++++++++++++++ .../standardization-of-development-process.md | 282 -------------- 2 files changed, 356 insertions(+), 282 deletions(-) create mode 100644 contributors/design-proposals/kep-development-process.md delete mode 100644 contributors/design-proposals/standardization-of-development-process.md diff --git a/contributors/design-proposals/kep-development-process.md b/contributors/design-proposals/kep-development-process.md new file mode 100644 index 00000000000..dc3e4f974d7 --- /dev/null +++ b/contributors/design-proposals/kep-development-process.md @@ -0,0 +1,356 @@ +# Proposal to Standardize Development Process + +## Metadata +``` +--- +metadata: + number: 0001 + state: opened + authors: + - author: + name: caleb miles + github: @calebamiles + slack: @calebamiles + owners: + - sig-release + - sig-pm + - sig-architecture + - sig-testing + - steering-committee + links: + issues: + - [someIssueURL]() + prs: + - https://github.com/kubernetes/community/pull/967 + discussions: + - https://groups.google.com/forum/#!topic/kubernetes-dev/65A-3ULYPB0 + - https://groups.google.com/forum/#!topic/kubernetes-sig-architecture/t-1EqeEoLPA + documentation: + - [someDocsLinkURL]() + related: + - [someRelatedKEPURL]() +``` + +## Responsible SIG(s) + +- SIG Release +- SIG PM +- SIG Architecture +- SIG Testing +- Steering Committee + +## Summary + +A standardized development process for Kubernetes is proposed in order to: +- provide a common structure for proposing changes to Kubernetes +- ensure that the motivation for a change is clear +- allow for the enumeration stability milestones and stability graduation + criteria +- persist project information in a Version Control System (VCS) for future + Kubernauts +- support the creation of _high value user facing_ information such as: + - release notes + - release announcement blog + - an overall project development roadmap +- support development across multiple repositories beyond `kubernetes/kubernetes` +- reserve GitHub issues for tracking work in flight rather than creating "umbrella" + issues +- ensure community participants are successfully able to drive changes to + completion across one or more releases while stakeholders are adequately + represented throughout the process + +## Motivation + +For cross project SIGs such as SIG PM and SIG Release an abstraction beyond a +single GitHub Issue or Pull request seems to be required in order to understand +and communicate upcoming changes to Kubernetes. Particularly the generation of +release notes and the release announcement blog post are rather difficult and +have sometimes delayed a release due to incompleteness. In a blog post +describing the [road to Go 2][], Russ Cox explains + +> that it is difficult but essential to describe the significance of a problem +> in a way that someone working in a different environment can understand + +as a project it is vital to be able to track the chain of custody for a proposed +enhancement from conception through implementation. This proposal does not +attempt to mandate how SIGs track their work internally, however, it is +suggested that SIGs which do not adhere to a process which allows for their hard +work to be explained to others in the wider Kubernetes community will see their +work wallow in the shadows of obscurity. At the very least [survey data][] +suggest that high quality documentation is crucial to project adoption. +Documentation can take many forms and it is imperative to ensure that it is easy +to produce high quality user or developer focused documentation for a complex +project like Kubernetes. + +The use of GitHub issues when proposing changes does not provide SIGs good +facilities for signaling approval or rejection of a proposed change to Kubernetes +since anyone can open a GitHub issue at any time. Additionally managing a proposed +change across multiple releases is somewhat cumbersome as labels and milestones +need to be updated for every release that a change spans. These long lived GitHub +issues lead to an ever increasing number of issues open against +`kubernetes/features` which itself has become a management problem. + +In addition to the challenge of managing issues over time, searching for text +within an issue can be challenging. The flat hierarchy of issues can also make +navigation and categorization tricky. While not all community members might +not be comfortable using Git directly, it is imperative that as a community we +work to educate people on a standard set of tools so they can take their +experience to other projects they may decide to work on in the future. While +git is a fantastic version control system (VCS), it is not a project management +tool nor a cogent way of managing an architectural catalog or backlog; this +proposal is limited to motivating the creation of a standardized definition of +work in order to facilitate project management. This primitive for describing +a unit of work may also allow contributors to create their own personalized +view of the state of the project while relying on Git and GitHub for consistency +and durable storage. + +Ideally release notes should [tell a story][] which +is compelling enough to encourage users and operators to upgrade their clusters. +Without a standardized mechanism for describing important enhancements our +talented technical writers and product managers struggle to weave a coherent +narrative explaining why a particular release is important. Additionally for +critical infrastructure such as Kubernetes adopters need a forward looking road map +in order to plan their adoption strategy. + +[tell a story]: https://blog.rust-lang.org/2017/08/31/Rust-1.20.html +[road to Go 2]: https://blog.golang.org/toward-go2 +[survey data]: http://opensourcesurvey.org/2017/ + +## Detailed design + +### What type of work should be tracked by a KEP + +The definition of what constitutes an "enhancement" is a foundational concern +for the Kubernetes project. Roughly any Kubernetes user or operator facing +enhancement should follow the KEP process: if an enhancement would be described +in either written or verbal communication to anyone besides the KEP author or +developer then consider creating a KEP. One concrete example is an enhancement +which should be communicated to SIG Release or SIG PM. + +Without detailed information explaining the motivation for an enhancement SIGs +must first approve a proposal, agreeing to a motivation over a mailing list, +video call, or hallway conversation. During the release process this motivation +must be rediscovered by the SIG, hopefully by finding a design proposal. The +process of announcing an enhancement through release notes suggests another +heuristic for describing what work should be tracked through an KEP: anything +that would require a design proposal. In fact it is possible to consider a KEP +an enhancement to the design proposal process in which design proposals are +used throughout the process of proposing an enhancement, scoping its design, +tracking its implementation, and agreeing on criteria for graduation to general +availability. + +As the local bodies of governance SIGs should have broad latitude in describing +what constitutes an enhancement which should be tracked through the KEP process; +hopefully the desire to be included in the rich narrative of a release announcement +will encourage broad participation in the KEP process across SIGs. SIGs should +also have the freedom to customize the KEP template according to their SIG +specific concerns. For example the KEP template used to track API changes will +likely have different subsections than the template for proposing governance +changes. + +### KEP Template + +The template for a KEP will be submitted in a separate proposal + +### KEP Workflow + +A KEP is proposed to have the following states + +- **opened**: a new KEP has been filed but not triaged by the responsible SIG or + working group +- **accepted**: the motivation has been accepted by the SIG or working group as in + road map +- **scoped**: the design has been approved by the SIG or working group +- **started**: the implementation of the KEP has begun +- **implemented**: the implementation of the KEP is complete +- **deferred**: the KEP has been postponed by the SIG or working group despite + agreement on the motivation +- **superseded**: the KEP has been superseded by another KEP +- **retired**: the implementation of the KEP has been removed +- **rejected**: the KEP has been rejected by the SIG or working group +- **orphaned**: the author or developer of the KEP is no longer willing or able + to complete implementation + +with possible paths through the state space + +- opened -> deferred (a) +- opened -> rejected (b) +- opened -> orphaned (c) +- opened -> accepted -> orphaned (d) +- opened -> accepted -> scoped -> superseded (e) +- opened -> accepted -> scoped -> orphaned (f) +- opened -> accepted -> scoped -> started -> retired (g) +- opened -> accepted -> scoped -> started -> orphaned (h) +- opened -> accepted -> scoped -> started -> superseded (i) +- opened -> accepted -> scoped -> started -> implemented (j) +- opened -> accepted -> scoped -> started -> implemented -> retired (k) + +the happy path is denoted by (j) where an KEP is opened; accepted by a SIG as in +their roadmap; fleshed out with a design; started; and finally implemented. As +Kubernetes continues to mature, hopefully metrics on the utilization of features +will drive decisions on what features to maintain and which to deprecate and so +it is possible that a KEP would be retired if its functionality no longer provides +sufficient value to the community. + +### Git and GitHub Implementation + +Practically an KEP would be implemented as a pull request to a central repository +with the following example structure + +``` +├── 0000-kep-template.md +├── CODEOWNERS +├── index.md +├── sig-architecture +│   ├── deferred +│   ├── orphaned +│   └── retired +├── sig-network +│   ├── deferred +│   ├── kube-dns +│   ├── orphaned +│   └── retired +├── sig-node +│   ├── deferred +│   ├── kublet +│   ├── orphaned +│   └── retired +├── sig-release +│   ├── deferred +│   ├── orphaned +│   └── retired +├── sig-storage +│   ├── deferred +│   ├── orphaned +│   └── retired +├── unsorted-to-be-used-by-newcomers-only +└── wg-resource-management + ├── deferred + ├── orphaned + └── retired +``` + +where each SIG or working group is given a top level directory with subprojects +maintained by the SIG listed in sub directories. For newcomers to the community +an `unsorted-to-be-used-by-newcomers-only` directory may be used before an KEP +can be properly routed to a SIG although hopefully if discussion for a potential +KEP begins on the mailing lists proper routing information will be provided to +the KEP author. Additionally a top level index of KEPs may be helpful for people +looking for a complete list of KEPs. There should be basic CI to ensure that an +`index.md` remains up to date. + +Ideally no work would begin within the repositories of the Kubernetes organization +before a KEP has been approved by the responsible SIG or working group. While the +details of how SIGs organize their work is beyond the scope of this proposal one +possibility would be for each charter SIG to create a top level repository within +the Kubernetes org where implementation issues managed by that SIG would be filed. + +### KEP Editor Role + +Taking a cue from the [Python PEP process][], I believe that a group of KEP editors +will be required to make this process successful; the job of an KEP editor is +likely very similar to the [PEP editor responsibilities][] and will hopefully +provide another opportunity for people who do not write code daily to contribute +to Kubernetes. + +In keeping with the PEP editors which + +> Read the PEP to check if it is ready: sound and complete. The ideas must make +> technical sense, even if they don't seem likely to be accepted. +> The title should accurately describe the content. +> Edit the PEP for language (spelling, grammar, sentence structure, etc.), markup +> (for reST PEPs), code style (examples should match PEP 8 & 7). + +KEP editors should generally not pass judgement on a KEP beyond editorial +corrections. + +[Python PEP process]: https://www.python.org/dev/peps/pep-0001/ +[PEP editor responsibilities]: https://www.python.org/dev/peps/pep-0001/#pep-editor-responsibilities-workflow + +### Important Metrics + +It is proposed that the primary metrics which would signal the success or +failure of the KEP process are + +- distribution of time a KEP spends in each state +- KEP rejection rate +- PRs referencing a KEP merged per week +- number of issued open which reference a KEP +- number of contributors who authored a KEP +- number of contributors who authored a KEP for the first time +- number of orphaned KEPs +- number of retired KEPs +- number of superseded KEPs + +### Prior Art + +The KEP process as proposed was essentially stolen from the [Rust RFC process] which +itself seems to be very similar to the [Python PEP process][] + +[Rust RFC process]: https://github.com/rust-lang/rfcs + +## Graduation Criteria + +should hit at least the following milestones + +- a release note draft can be generated by referring primarily to KEP content +- a yearly road map is expressed as a KEP + +## Drawbacks + +Any additional process has the potential to engender resentment within the +community. There is also a risk that the KEP process as designed will not +sufficiently address the scaling challenges we face today. PR review bandwidth is +already at a premium and we may find that the KEP process introduces an unreasonable +bottleneck on our development velocity. + +It certainly can be argued that the lack of a dedicated issue/defect tracker +beyond GitHub issues contributes to our challenges in managing a project as large +as Kubernetes, however, given that other large organizations, including GitHub +itself, make effective use of GitHub issues perhaps the argument is overblown. + +The centrality of Git and GitHub within the KEP process also may place too high +a barrier to potential contributors, however, given that both Git and GitHub are +required to contribute code changes to Kubernetes today perhaps it would be reasonable +to invest in providing support to those unfamiliar with this tooling. + +Expanding the proposal template beyond the single sentence description currently +required in the [features issue template][] may be a heavy burden for non native +English speakers and here the role of the KEP editor combined with kindness and +empathy will be crucial to making the process successful. + +[features issue template]: https://github.com/kubernetes/features/blob/master/ISSUE_TEMPLATE.md + +## Alternatives + +This KEP process is related to +- the generation of a [architectural roadmap][] +- the fact that the [what constitutes a feature][] is still undefined +- [issue management][] +- the difference between an [accepted design and a proposal][] +- [the organization of design proposals][] + +this proposal attempts to place these concerns within a general framework. + +[architectural roadmap]: https://github.com/kubernetes/community/issues/952 +[what constitutes a feature]: https://github.com/kubernetes/community/issues/531 +[issue management]: https://github.com/kubernetes/community/issues/580 +[accepted design and a proposal]: https://github.com/kubernetes/community/issues/914 +[the organization of design proposals]: https://github.com/kubernetes/community/issues/918 + +## Unresolved Questions + +- How reviewers and approvers are assigned to a KEP +- Approval decision process for a KEP +- Example schedule, deadline, and time frame for each stage of a KEP +- Communication/notification mechanisms +- Review meetings and escalation procedure +- Decision on where development should occur + +## Mentors + +- caleb miles + - github: [calebamiles](https://github.com/calebamiles/) + - slack: [calebamiles](https://coreos.slack.com/team/caleb.miles) + - email: [caleb.miles@coreos.com](mailto:caleb.miles@coreos.com) + - pronoun: "he" diff --git a/contributors/design-proposals/standardization-of-development-process.md b/contributors/design-proposals/standardization-of-development-process.md deleted file mode 100644 index b8f5e6202c0..00000000000 --- a/contributors/design-proposals/standardization-of-development-process.md +++ /dev/null @@ -1,282 +0,0 @@ -# Proposal to Standardize Development Process - -## Links -``` ---- -Links: -- Issues: - - [someIssueURL]() -- PRs: - - https://github.com/kubernetes/community/pull/967 -- MailingListDiscussions: - - https://groups.google.com/forum/#!topic/kubernetes-dev/65A-3ULYPB0 -- Documentation: - - [someDocsLinkURL]() -``` - -## Responsible SIG(s) - -- SIG Release -- SIG PM -- SIG Architecture -- SIG Governance -- Technical Steering Committee - -## Summary - -A standardized development process for Kubernetes is proposed in order to: -- provide a common structure for proposing changes to Kubernetes -- ensure that the motivation for a change is clear -- allow for the enumeration stability milestones and stability graduation - criteria -- persist project information in a Version Control System (VCS) for future - Kubernauts -- support the creation of _high value user facing_ information such as: - - release notes - - release announcement blog - - a cross project development road map -- support development across multiple repositories beyond `kubernetes/kubernetes` -- begin to move away from using GitHub issues for feature requests and reserve them - for tracking work in flight - -## Motivation - -Today it is becoming fairly difficult to track changes being made across all -Special Interest Groups (SIGs) within Kubernetes. For cross project SIGs such -as SIG PM and SIG Release an abstraction beyond a single GitHub Issue or Pull -request seems to be required in order to understand and communicate upcoming -changes to Kubernetes. Particularly the generation of release notes and the -release announcement blog post are rather difficult and have sometimes delayed -a release due to incompleteness. In a blog post describing the [road to Go 2][], -Russ Cox explains - -> that it is difficult but essential to describe the significance of a problem -> in a way that someone working in a different environment can understand - -unfortunately I believe that our current mechanism for proposing changes by -opening a GitHub issue against `kubernetes/features` falls short of providing -an effective vehicle for agreeing on the motivation and the design of a proposed -change. - -The use of GitHub issues when proposing changes does not provide SIGs good -facilities for signaling approval or rejection of a proposed change to Kubernetes -since anyone can open a GitHub issue at any time. Additionally managing a proposed -change across multiple releases is somewhat cumbersome as labels and milestones -need to be updated for every release that a change spans which leads to an ever -increasing number of issues open against `kubernetes/features` which itself has -become a management problem. - -We have chosen to use Git as our VCS with GitHub currently used to host our -repositories. While GitHub provides a rich feature set as an open source project, -managing issues over time and searching for text within an issue can be -challenging. The flat hierarchy of issues can also make navigation and -categorization tricky. While not all community members might not be comfortable -using Git directly, I believe it is imperative that as a community we work to -educate people on a standard set of tools so they can take their experience to -other projects they may decide to work on in the future. - -It is my hope that by increasing the requirements for proposing a change to -Kubernetes we can prevent important work like generating release notes from -falling out of the release cycle and in general make working with and speaking -about Kubernetes an easier and more pleasant experience. - -[road to Go 2]: https://blog.golang.org/toward-go2 - -## Detailed Design - -### RFC Template - -In order to implement an RFC like process a template of the following form would -be created - -``` -# Title -## Links -## Responsible SIG(s) -## Summary -## Motivation -## Examples [optional] -## Detailed Design -## Graduation Criteria -## Drawbacks [optional] -## Alternatives [optional] -## Unresolved Questions [optional] -## Mentors [optional] -``` - -where sections marked `[optional]` may be omitted by RFC authors.It is expected -that most RFCs will begin their lives in discussion with the responsible SIGs, -preferably in a mailing list so that the discussion is saved for future Kubernauts. - -It may be very helpful to provide links to experience reports of other Kubernetes -users and developers within the `Motivation` section in order to provide additional -context for the necessity of a proposed change. The `Examples` section could be -used to provide motivating examples for documentation writers looking to teach -others how to consume the proposed change once implemented. As an open source -community we must constantly be looking for ways to better serve newcomers and -a `Mentors` section could be used to help connect interested parties with elders -in the community who are able to mentor people towards an implementation. - -It is intended for RFCs to be a conversation, the comment part of the acronym, -so it is not expected for every section of the template to be completed by the -time an RFC is filed, however, the `Summary`, `Motivation`, and `Examples` section -should probably be filled out by the time an RFC is filed. Merging an RFC only -signals agreement of the sections which have been provided thus far and an RFC -may be completed through a few rounds of review in order to achieve alignment. - -### Git and GitHub Implementation - -Practically an RFC would be implemented as a pull request to a central repository -with the following example structure - -``` -├── 0000-rfc-template.md -├── CODEOWNERS -├── index.md -├── sig-architecture -├── sig-network -│   └── kube-dns -├── sig-node -│   └── kubelet -├── sig-release -├── sig-storage -├── unsorted-to-be-used-by-newcomers-only -└── wg-resource-management -``` - -where each SIG or working group is given a top level directory with components -maintained by the SIG listed in sub directories. For newcomers to the community -an `unsorted-to-be-used-by-newcomers-only` directory may be used before an RFC -can be properly routed to a SIG although hopefully if discussion for a potential -RFC begins on the mailing lists proper routing information will be provided to -the RFC author. Additionally a top level index of RFCs may be helpful for people -looking for a complete list of RFCs. There should be basic CI to ensure that a -`index.md` remains up to date. - -Ideally no work would begin on an RFC before it has been approved by the -responsible SIG or working group. In order to help combat the explosion of GitHub -issues against `kubernetes/kubernetes` it is further proposed for each chartered -SIG to create a top level repository within the `kubernetes` GitHub organization -where implementation issues would be filed and linked to in the RFC. - -### RFC Editor Role - -Taking a cue from the [Python PEP process][], I believe that a group of RFC editors -will be required to make this process successful; the job of an RFC editor is -likely very similar to the [PEP editor responsibilities][] and will hopefully -provide another opportunity for people who do not write code daily to contribute -to Kubernetes. - -[Python PEP process]: https://www.python.org/dev/peps/pep-0001/ -[PEP editor responsibilities]: https://www.python.org/dev/peps/pep-0001/#pep-editor-responsibilities-workflow - -### Important Metrics - -I believe the primary metrics which would signal the success or failure of the RFC -process would be - -- distribution of RFC merge times -- RFC rejection rate -- distribution of time between RFC acceptance and implementation issue creation -- PRs referencing an RFC merged per week - -### Customization of the RFC Process - -Given the general independence of SIGs I believe it will be important for SIGs to -decide if they would like to opt out of the RFC process entirely or if they would -like to modify the RFC process to better serve their needs. For example I can -imagine RFCs targeting SIG Architecture or SIG API Machinery would likely require -a section on backwards compatibility and migration strategies. I believe the best -place for a SIG to describe its development process would be in its charter. - -### Prior Art - -The RFC process as proposed was essentially stolen from the [Rust RFC process] which -itself seems to be very similar to the [Python PEP process][] - -[Rust RFC process]: https://github.com/rust-lang/rfcs - -## Graduation Criteria - -Before we can consider this process successful for our current size I believe we -should hit at least the following milestones - -- a release note draft can be generated by referring to RFC content -- a road map can be generated by referring to RFC content - -The process for creating draft release notes and development status reports -should eventually rely largely on automation, but I do not believe that the -implementation of such automation should prevent the RFC process from being -considered stable. I can certainly imagine a future where a release notes draft -is produced nightly along with the collection of binaries that users deploy. - -## Drawbacks - -Any additional process has the potential to engender resentment within the -community. There is also a risk that the RFC process as designed will not -sufficiently address the scaling challenges we face today. PR review bandwidth is -already at a premium and we may find that the RFC process introduces an unreasonable -bottleneck on our development velocity. - -It certainly can be argued that the lack of a dedicated issue/defect tracker -beyond GitHub issues contributes to our challenges in managing a project as large -as Kubernetes, however, given that other large organizations, including GitHub -itself, make effective use of GitHub issues perhaps the argument is overblown. - -The centrality of Git and GitHub within the RFC process also may place too high -a barrier to potential contributors, however, given that both Git and GitHub are -required to contribute code changes to Kubernetes today I would argue that we -should focus on providing support to those unfamiliar with this tooling. - -Expanding the proposal template beyond the single sentence description currently -required in the [features issue template][] may be a heavy burden for non native -English speakers and here I believe that the role of the RFC editor combined with -kindness and empathy will be crucial to making the process successful. - -[features issue template]: https://github.com/kubernetes/features/blob/master/ISSUE_TEMPLATE.md - -## Alternatives - -This RFC process is related to -- the generation of our [architectural road map][] -- the fact that the [what constitutes a feature][] is still undefined -- how we [manage issues][] -- the difference between an [accepted design and a proposal][] -- [the organization of design proposals][] - -and I believe that this RFC process attempts to place these concerns within a -more general framework - -[architectural road map]: https://github.com/kubernetes/community/issues/952 -[what constitutes a feature]: https://github.com/kubernetes/community/issues/531 -[manage issues]: https://github.com/kubernetes/community/issues/580 -[accepted design and a proposal]: https://github.com/kubernetes/community/issues/914 -[the organization of design proposals]: https://github.com/kubernetes/community/issues/918 - -## Unresolved Questions - -Whether we believe it makes sense for a SIG repository to serve a mini mono -repository where code lives in a structure like - -``` -kubernetes:sig-node/pkg/kublet -``` - -or whether SIGs maintain a list of repositories maintained by the SIG in an index -like - -``` -kubernetes:sig-node/repositories.md -``` - -is not clear to me. I believe that users would generally think about Kubernetes -from a component standpoint if they are aware of the component at all but I have -no real data to support that assertion. - -## Mentors - -- caleb miles - - github: [calebamiles](https://github.com/calebamiles/) - - slack: [calebamiles](https://coreos.slack.com/team/caleb.miles) - - email: [caleb.miles@coreos.com](mailto:caleb.miles@coreos.com) - - pronoun: "he" From 1685ce65c31fa3c0946ae906dd7352e36dfa2cb6 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Sun, 24 Sep 2017 21:50:38 -0700 Subject: [PATCH 4/7] Update KEP development process with link to KEP template --- .../design-proposals/kep-development-process.md | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/contributors/design-proposals/kep-development-process.md b/contributors/design-proposals/kep-development-process.md index dc3e4f974d7..596aa8bf91a 100644 --- a/contributors/design-proposals/kep-development-process.md +++ b/contributors/design-proposals/kep-development-process.md @@ -1,4 +1,4 @@ -# Proposal to Standardize Development Process +# Kubernetes Enhancement Proposal (KEP) Process ## Metadata ``` @@ -28,17 +28,9 @@ metadata: documentation: - [someDocsLinkURL]() related: - - [someRelatedKEPURL]() + - [KEP template](https://github.com/kubernetes/community/pull/1124) ``` -## Responsible SIG(s) - -- SIG Release -- SIG PM -- SIG Architecture -- SIG Testing -- Steering Committee - ## Summary A standardized development process for Kubernetes is proposed in order to: @@ -150,7 +142,9 @@ changes. ### KEP Template -The template for a KEP will be submitted in a separate proposal +The template for a KEP is precisely defined in the [template proposal][] + +[template proposal]: https://github.com/kubernetes/community/pull/1124 ### KEP Workflow From 71344d2917dff540437e93b77689453cdba13b0c Mon Sep 17 00:00:00 2001 From: caleb miles Date: Mon, 25 Sep 2017 12:58:03 -0700 Subject: [PATCH 5/7] address more comments --- .../kep-development-process.md | 39 ++++++++++++------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/contributors/design-proposals/kep-development-process.md b/contributors/design-proposals/kep-development-process.md index 596aa8bf91a..03f5a3360e9 100644 --- a/contributors/design-proposals/kep-development-process.md +++ b/contributors/design-proposals/kep-development-process.md @@ -96,17 +96,26 @@ a unit of work may also allow contributors to create their own personalized view of the state of the project while relying on Git and GitHub for consistency and durable storage. -Ideally release notes should [tell a story][] which -is compelling enough to encourage users and operators to upgrade their clusters. -Without a standardized mechanism for describing important enhancements our -talented technical writers and product managers struggle to weave a coherent -narrative explaining why a particular release is important. Additionally for -critical infrastructure such as Kubernetes adopters need a forward looking road map -in order to plan their adoption strategy. +Ideally release notes should [tell a story][] which is compelling enough to +encourage users and operators to upgrade their clusters. Without a standardized +mechanism for describing important enhancements our talented technical writers +and product managers struggle to weave a coherent narrative explaining why a +particular release is important. Additionally for critical infrastructure such +as Kubernetes adopters need a forward looking road map in order to plan their +adoption strategy. + +A KEP is broken into sections which can be merged into source control +incrementally in order to support an iterative development process. An important +goal of the KEP process is ensuring that the process for submitting the content +contained in [design proposals][] is both clear and efficient. The KEP process +is intended to create high quality uniform design and implementation documents +for SIGs to deliberate. [tell a story]: https://blog.rust-lang.org/2017/08/31/Rust-1.20.html [road to Go 2]: https://blog.golang.org/toward-go2 [survey data]: http://opensourcesurvey.org/2017/ +[design proposals]: https://github.com/kubernetes/community/tree/master/contributors/design-proposals + ## Detailed design @@ -131,14 +140,13 @@ used throughout the process of proposing an enhancement, scoping its design, tracking its implementation, and agreeing on criteria for graduation to general availability. -As the local bodies of governance SIGs should have broad latitude in describing -what constitutes an enhancement which should be tracked through the KEP process; -hopefully the desire to be included in the rich narrative of a release announcement -will encourage broad participation in the KEP process across SIGs. SIGs should -also have the freedom to customize the KEP template according to their SIG -specific concerns. For example the KEP template used to track API changes will -likely have different subsections than the template for proposing governance -changes. +As the local bodies of governance, SIGs should have broad latitude in describing +what constitutes an enhancement which should be tracked through the KEP process. +SIGs may find that helpful to enumerate what _does not_ require a KEP rather +than what does. SIGs also have the freedom to customize the KEP template +according to their SIG specific concerns. For example the KEP template used +to track API changes will likely have different subsections than the template +for proposing governance changes. ### KEP Template @@ -266,6 +274,7 @@ corrections. It is proposed that the primary metrics which would signal the success or failure of the KEP process are +- how many "features" are tracked with a KEP - distribution of time a KEP spends in each state - KEP rejection rate - PRs referencing a KEP merged per week From ddeb2b4b547109d3e757cc0ee27ccb4acfd25959 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Tue, 26 Sep 2017 01:29:57 -0700 Subject: [PATCH 6/7] move KEP process proposal to architecture directory. ...also change filename to match title --- .../kubernetes-enhancement-proposal-process.md} | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) rename contributors/design-proposals/{kep-development-process.md => architecture/kubernetes-enhancement-proposal-process.md} (99%) diff --git a/contributors/design-proposals/kep-development-process.md b/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md similarity index 99% rename from contributors/design-proposals/kep-development-process.md rename to contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md index 03f5a3360e9..c0b97d0722d 100644 --- a/contributors/design-proposals/kep-development-process.md +++ b/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md @@ -1,4 +1,4 @@ -# Kubernetes Enhancement Proposal (KEP) Process +# Kubernetes Enhancement Proposal Process ## Metadata ``` @@ -34,6 +34,7 @@ metadata: ## Summary A standardized development process for Kubernetes is proposed in order to: + - provide a common structure for proposing changes to Kubernetes - ensure that the motivation for a change is clear - allow for the enumeration stability milestones and stability graduation @@ -51,6 +52,9 @@ A standardized development process for Kubernetes is proposed in order to: completion across one or more releases while stakeholders are adequately represented throughout the process +This process is supported by a unit of work called a Kubernetes Enhancement +Proposal or KEP. + ## Motivation For cross project SIGs such as SIG PM and SIG Release an abstraction beyond a From 4b0856be7af6d32a6657027f59a24d6098e988bb Mon Sep 17 00:00:00 2001 From: caleb miles Date: Tue, 26 Sep 2017 01:41:04 -0700 Subject: [PATCH 7/7] address comments from: @bgrant0607 - https://github.com/kubernetes/community/pull/967#issuecomment-332007969 --- ...kubernetes-enhancement-proposal-process.md | 43 +++++++++++++++---- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md b/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md index c0b97d0722d..be4b21ed103 100644 --- a/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md +++ b/contributors/design-proposals/architecture/kubernetes-enhancement-proposal-process.md @@ -31,9 +31,29 @@ metadata: - [KEP template](https://github.com/kubernetes/community/pull/1124) ``` +## Table of Contents + +- [Kubernetes Enhancement Proposal Process](#kubernetes-enhancement-proposal-process) + - [Metadata](#metadata) + - [Summary](#summary) + - [Motivation](#motivation) + - [Reference-level explanation](#reference-level-explanation) + - [What type of work should be tracked by a KEP](#what-type-of-work-should-be-tracked-by-a-kep) + - [KEP Template](#kep-template) + - [KEP Workflow](#kep-workflow) + - [Git and GitHub Implementation](#git-and-github-implementation) + - [KEP Editor Role](#kep-editor-role) + - [Important Metrics](#important-metrics) + - [Prior Art](#prior-art) + - [Graduation Criteria](#graduation-criteria) + - [Drawbacks](#drawbacks) + - [Alternatives](#alternatives) + - [Unresolved Questions](#unresolved-questions) + - [Mentors](#mentors) + ## Summary -A standardized development process for Kubernetes is proposed in order to: +A standardized development process for Kubernetes is proposed in order to - provide a common structure for proposing changes to Kubernetes - ensure that the motivation for a change is clear @@ -53,7 +73,14 @@ A standardized development process for Kubernetes is proposed in order to: represented throughout the process This process is supported by a unit of work called a Kubernetes Enhancement -Proposal or KEP. +Proposal or KEP. A KEP attempts to combine aspects of a + +- feature, effort, and launch tracking document +- a product requirements document +- design document + +into one file which is created incrementally in collaboration with one or more +Special Interest Groups (SIGs). ## Motivation @@ -115,13 +142,13 @@ contained in [design proposals][] is both clear and efficient. The KEP process is intended to create high quality uniform design and implementation documents for SIGs to deliberate. -[tell a story]: https://blog.rust-lang.org/2017/08/31/Rust-1.20.html +[tell a story]: https://blog.rust-lang.org/2017/08/31/Rust-1.20.html [road to Go 2]: https://blog.golang.org/toward-go2 [survey data]: http://opensourcesurvey.org/2017/ [design proposals]: https://github.com/kubernetes/community/tree/master/contributors/design-proposals -## Detailed design +## Reference-level explanation ### What type of work should be tracked by a KEP @@ -148,7 +175,7 @@ As the local bodies of governance, SIGs should have broad latitude in describing what constitutes an enhancement which should be tracked through the KEP process. SIGs may find that helpful to enumerate what _does not_ require a KEP rather than what does. SIGs also have the freedom to customize the KEP template -according to their SIG specific concerns. For example the KEP template used +according to their SIG specific concerns. For example the KEP template used to track API changes will likely have different subsections than the template for proposing governance changes. @@ -181,10 +208,10 @@ with possible paths through the state space - opened -> deferred (a) - opened -> rejected (b) -- opened -> orphaned (c) +- opened -> orphaned (c) - opened -> accepted -> orphaned (d) - opened -> accepted -> scoped -> superseded (e) -- opened -> accepted -> scoped -> orphaned (f) +- opened -> accepted -> scoped -> orphaned (f) - opened -> accepted -> scoped -> started -> retired (g) - opened -> accepted -> scoped -> started -> orphaned (h) - opened -> accepted -> scoped -> started -> superseded (i) @@ -259,7 +286,7 @@ likely very similar to the [PEP editor responsibilities][] and will hopefully provide another opportunity for people who do not write code daily to contribute to Kubernetes. -In keeping with the PEP editors which +In keeping with the PEP editors which > Read the PEP to check if it is ready: sound and complete. The ideas must make > technical sense, even if they don't seem likely to be accepted.