KAFKA-8729: Add upgrade docs for KIP-467 on augmented produce response (

apache#7522) Add a paragraph explaining the producer caller's expected behavior change on record validation failure scenarios that are improved by KIP-467. Reviewers: Tu V. Tran <[email protected]>, Jason Gustafson <[email protected]>
agam · Oct 25, 2019 · 4f682b3 · 4f682b3
1 parent 1df01d2
commit 4f682b3
Showing 1 changed file with 7 additions and 0 deletions.
diff --git a/docs/upgrade.html b/docs/upgrade.html
@@ -87,6 +87,13 @@ <h5><a id="upgrade_240_notable" href="#upgrade_240_notable">Notable changes in 2
         The old overloaded functions are deprecated and we would recommend users to make their code changes to leverage the new methods (details
         can be found in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-520%3A+Add+overloaded+Consumer%23committed+for+batching+partitions">KIP-520</a>).
     </li>
+    <li>We've introduced a new <code>INVALID_RECORD</code> error in the produce response to distinguish from the <code>CORRUPT_MESSAGE</code> error.
+        To be more concrete, previously when a batch of records were sent as part of a single request to the broker and one or more of the records failed
+        the validation due to various causes (mismatch magic bytes, crc checksum errors, null key for log compacted topics, etc), the whole batch would be rejected
+        with the same and misleading <code>CORRUPT_MESSAGE</code>, and the caller of the producer client would see the corresponding exception from either
+        the future object of <code>RecordMetadata</code> returned from the <code>send</code> call as well as in the <code>Callback#onCompletion(RecordMetadata metadata, Exception exception)</code>
+        Now with the new error code and improved error messages of the exception, producer callers would be better informed about the root cause why their sent records were failed.
+    </li>
     <li>We are introducing incremental cooperative rebalancing to the clients' group protocol, which allows consumers to keep all of their assigned partitions during a rebalance
         and at the end revoke only those which must be migrated to another consumer for overall cluster balance. The <code>ConsumerCoordinator</code> will choose the latest <code>RebalanceProtocol</code>
         that is commonly supported by all of the consumer's supported assignors. You can use the new built-in <code>CooperativeStickyAssignor</code> or plug in your own custom cooperative assignor. To do