Updating protobuf

You will create and merge the following Pull Requests.

1st PR: add the new protobuf version to the Bazel tree

1. Fetch the desired protobuf version and copy it in a folder new_proto under third_party/protobuf.

   Example: to upgrade to 3.6.1, download and unpack protobuf-all-3.6.1.zip.

2. Build the Java proto library from source and, in case you cloned an upstream version of protobuf, remove the .git folders:

   cd third_party/protobuf/new_proto
   bazel build :protobuf_java :protobuf_java_util
   cp bazel-bin/libprotobuf_java.jar .
   cp bazel-bin/libprotobuf_java_util.jar .
   bazel clean --expunge
   rm -rf .git .gitignore .gitmodules
   

   Reason: Bazel uses upstream protobuf from source, except for Java, as Bazel's bootstrapping scripts currently don't build protobuf Java when bootstrapping Bazel but use pre-built jars instead.

3. Modify protobuf's BUILD file to not build java from source, but to use the jars instead:

   1. In the BUILD file delete the rules listed under Java support.

   2. From the third_party/protobuf/<old_proto>/BUILD file copy the rules under the "Modifications made by bazel" section to the new BUILD file. The java rules in there should have the same names as the ones you just deleted under "Java support". You might need to update the names of the jars in the rules sources to the ones you just build.

4. Copy third_party/protobuf/<old_proto>/com_google_protobuf_java.BUILD to the new directory.

5. From third_party/protobuf/<old_proto>/util/python/BUILD, third_party/protobuf/<old_proto>/examples/BUILD, and third_party/protobuf/<old_proto>/third_party/googletest/BUILD.bazel: copy the licenses declaration and the srcs filegroup to the corresponding file under third_party/protobuf/<new_proto>.

6. In third_party/protobuf/<new_proto>/BUILD, in the srcs filegroup rule, update the version number referring to the newly added srcs rules.

7. Rename third_party/protobuf/<new_proto> directory according to the protobuf version number.

8. In third_party/protobuf/BUILD:

   1. Add a new variable _NEW_PROTOBUF_VERSION, set to value of the version.

   2. In the srcs filegroup rule, add:

       srcs = [
           "//third_party/protobuf/" + PROTOBUF_VERSION + ":srcs",
      +    "//third_party/protobuf/" + _NEW_PROTOBUF_VERSION + ":srcs",
       ],

9. Create a PR of these changes and merge it directly to https://bazel.googlesource.com/bazel/+/master (without the usual process of importing it to the Google-internal version control).

2nd and 3rd PRs: update references in the Bazel tree

1. In third_party/protobuf/BUILD:

   1. rename PROTOBUF_VERSION to _OLD_PROTOBUF_VERSION

   2. rename _NEW_PROTOBUF_VERSION to PROTOBUF_VERSION

2. In the root WORKSPACE file update relative paths of protobuf to point to the new version.

3. Update version number in src/main/protobuf/BUILD and src/test/shell/testenv.sh.

4. Update the current version in this file.

5. Create a PR of these changes and get it imported. Some files won't be imported (those that are only hosted on GitHub); this is expected.

6. Wait for the imported PR to be pushed back to GitHub. Rebase the PR from the previous step, and merge it to https://bazel.googlesource.com/bazel/+/master .

4th PR: remove the old directory

1. Delete the third_party/protobuf/<old_proto> directory.

2. Remove _OLD_PROTOBUF_VERSION from third_party/protobuf/BUILD.

3. Create a PR of these changes and merge it directly to https://bazel.googlesource.com/bazel/+/master .

Update this file if you found these instructions to be wrong or incomplete.

Current protobuf version

The current version of protobuf is 3.6.1.