Skip to content

Latest commit

 

History

History
1025 lines (788 loc) · 77.1 KB

release-notes.md

File metadata and controls

1025 lines (788 loc) · 77.1 KB

Apache

2.4.2 — 2019-12-04

Fixes

  • Fixed don't set interrupt flag again after catching interrupt exception in Pulsar Client #5643
  • Fixed data is not deleted after expiration due to connected readers #5621
  • Fixed the go client docs missing in website #5595
  • Fixed schema def build error with protobuf schema #5569
  • Fixed docs about reset cursor #5551
  • Fixed repeated initialization of connectorsManager #5545
  • Fixed Functions unnecessarily restart during FunctionRuntimeManager init phase #5527
  • Fixed list non-persistent topics shows the persistent topics #5502
  • Return after triggering callback with empty result #5500
  • Fixed dispatcher skipping delivery of a batch during concurrent replays #5499
  • Fixed reader_listener option for Python API #5487
  • Fixed wrongly report "3600 messages have timed-out" #5477
  • Fixed broken custom auth-provider that uses authenticationData #5462
  • Fixed negative ack tracker constructor sequence #5453
  • Fixed StringSchema static initialization #5445
  • Fixed message corruption on OOM for batch messages #5443
  • Fixed couple functions related integration tests #5434
  • Fixed bug that namespace policies does not take effect due to NPE #5408
  • Fixed race condition : Failed to read-more entries on dispatcher #5391
  • Fixed potential deadlock that can occur in addConsumer #5371
  • Fixed proxy to be able to re-send request body #5361
  • Fixed pulsar can't load the customized SerDe #5357
  • Fixed instability in Pulsar Function window integration test #5337
  • Fixed bk write failure, use signals to resume writing #5322
  • Fixed bad_weak_ptr error when closing producer #5315
  • Fixed typo(massage->message) causing prometheus metrics display error #5299
  • Fixed invalidate cache on zk-cache timeout #5298
  • Fixed memory leak caused by not being executed ClientConnection destructor #5286
  • Fixed producer blocked after send an over size message while batch enabled #5282
  • Fixed race condition while triggering message redelivery after an ack-timeout event #5276
  • Fixed behavior when getting a key from functions state that doesn't exist #5272
  • Fixed Cmake to build _pulsar for osx #5263
  • Fixed client backoff setting does not take effect #5261
  • Fixed memory leak caused by deadline_timer holding object reference #5246
  • Fixed in Message Deduplication that may cause incorrect client/broker interaction #5243
  • Fixed bug that fails to search namespace bundle due to NPE #5191
  • Fixed bug that message delivery stops after resetting cursor for failover subscription #5185
  • Fixed exception type check order bug #5174
  • Fixed spark receiver to account for all the consumer config options #5152
  • Fixed broker fails to start with function worker enabled and broker client using TLS #5151
  • Fixed deadlock when resetting cursor #5136
  • Fixed windowed functions were broken when we changed java function instance to use classloaders #5117
  • Fixed storage size always 0 without subscription #5108

Enhancements

  • Add subscribe position param for consumer of sink #5532
  • Efficiency improvements for delay delivery tracker #5498
  • Add is_read_compacted to create_reader() in python API #5483
  • Make some common use method of ManagedLedger public #5472
  • Avoid leak on publish failure on batch message #5442
  • Record message failure and avoid exiting from process on publish failure #5441
  • Add support for partitioned topic consumer seek by time #5435
  • Add default loader for latest pyyaml #5432
  • Trim messages which less than mark delete position for message redelivery #5378
  • Make skip all messages async #5375
  • Set default ensemble size to 2 in service conf, to match broker.conf #5339
  • Only seek when reading unexpected entry #5356
  • Don't require both region and endpoint to be specified #5355
  • If cursor is not durable, close dispatcher when all consumers are removed from subscription #5340
  • Disable stickyRead by default #5321
  • Allow to specify delivery delay in C++ client #5317
  • Add debug log + fix thread-factory name #5302
  • Don't attempt to append on read-only cursor ledger #5297
  • Close previous dispatcher when subscription type changes #5288
  • Improve error handling logic for effectively once #5271
  • Upgrade dependencies for security fixes #5232
  • Allow for topic deletions with regex consumers #5230
  • Ensure consumer background tasks are cancelled after subscribe failures #5204
  • Added missing enum value KeyShared on the python wrapper #5196
  • Make some member variables of Dispatcher volatile #5193
  • Ensure getting list of topics for namespace is handled asynchronously #5188
  • Close RateLimiter instance #5155
  • Throw an error if the key was not specified for querying state #5145
  • Allow configuring region aware placement related settings #5100
  • DeleteBookieRack should remove the rack info from zookeeper #5084
  • Use "info" as the default root logger level #5079
  • Modify BatcherBuilder interface and it's subs to implement java.io.Serializable, otherwise java.io.NotSerializableException occurs when we use plusar-flink-connector #5068
  • Don't return DEFAULT_RACK if ZkBookieRackAffinityMapping can't resolve network location #5067
  • Reload zk cache asynchronously #5049
  • Add different cache flags to ConcurrentOpenLongPairRangeSet for size() and toString() #5040
  • Introduce number of threads in perf producer program #5036
  • Completing connector configuration #4999
  • Add checkstyle validation and fix style violations in the common module #4989
  • Trim deleted entries after recover cursor #4987
  • Expose getLastMessageId method in ConsumerImpl #4911
  • Add a documentation page for metrics reference #4910
  • Provide a convenient method for C++ client producer batch container #4885
  • Add schema admin api get schema info with schema version #4877
  • Return Message ID for send for cpp and cgo client #4811
  • Add the schema admin api #4800
  • Clarify how retention interacts with readers #4780
  • Get schema info with topic partition #4751
  • Remove failed stale producer from the connection #4741
  • Update logic for picking active consumer for failover subscription on non-partitioned topic #4604

2.4.1 — 2019-08-30

Fixes

  • Fixed wrong serialize of batchBuilder in ProducerConfigurationData #4620
  • Fixed SchemaInfo properties losing when registering schema using admin api #4617
  • Fixed wrong topic domain returned by get partitioned topic #4613
  • Fixed pulsar-dashboard wrong count number of bundles #4637
  • Fixed partitionIndex error in consumer for a single partitioned topic #4591
  • Fixed deadlock on get-status rest-api call in broker #4616
  • Fixed deadlock in subscribeAsync demo #4649
  • Fixed C++ client lookup error over HTTP in standalone #4625
  • Fixed NPE at managed-ledger when fetch reader internal-stats #4615
  • Fixed C++ client producer sendAsync() hang when no enough batched message #4657
  • Fixed issue when submitting NAR via file url #4577
  • Renamed C++ logger enum names to avoid conflicts with compiler macros #4664
  • Fixed leaking of pulsar-io-influxdb in distribution #4678
  • Fixed the default port for https and http in admin client #4623
  • Fixed wrong schema delete when checking compatibility #4669
  • Fixed docker/publish.sh doesn't work for pulsar-all and pulsar-standalone images #4705
  • Fixed integration-test failure when execute pip3 install pulsar_client #4754
  • Added log folder in pulsa-function-go back #4736
  • Fixed Pulsar SQL NPE when predicate pushdown for publish_time #4744
  • Fixed redelivered message logic of partition topic #4653
  • Fixed C++ log level names in Log4cxxLogger #4735
  • Fixed go function package not executable #4743
  • Added anonymous role to proxy configuration #4733
  • Added kubernetes namespace to function instance url #4701
  • Fixed go function not parse conf content first #4746
  • Made PulsarKafkaProducer thread safe #4745
  • Fixed messages not get acked if there is not sink topic #4815
  • Fixed bug when function package jar/py/go and runtime is not set #4814
  • Fixed topic loading in rest-api not time out in zooKeeperOperationTimeoutSeconds #4805
  • Fixed default retention policy miss upload to zk #4810
  • Added checking function implements correct interface #4844
  • Fixed retention size policy bug #4825
  • Catch throwable in interceptors of consumer and producer #4860
  • Fixed first position in managedLedger is ahead of the last #4853
  • Fixed concurrent access of uninitializedCursors in ManagedLedgerImpl.asyncOpenCursor #4837
  • Fixed schema not found handling in pulsar-sql #4890
  • Fixed requests not respect no_proxy env variable in dashboard #4867
  • Fixed broken replication msg to specific cluster #4930
  • Fixed dashboard peek parse message error #4918
  • Fixed StructSchema reader cache loading logic #4962
  • Fixed fd leakage in FunctionActioner.downloadFile #4970
  • Fixed python function str bytes convert in example custom_object_function.py #4946
  • Fixed NPE while cleaning up namespace node #4965
  • Fixed the getSchema logic in pulsar proxy #4975
  • Fixed warning by add default loader for latest pyyaml #4974
  • Fixed snappy compressor compile error in pulsar-client-cpp #4972
  • Reinitialize certain components for externally managed runtimes when moving functions #5007
  • Upgraded jackson-databind #5011
  • Fixed the problem of missing dependencies less in docker #5034
  • Fixed duplicated Prometheus TYPE in broker metrics #4183
  • Fixed pulsar sink and source state by init state earlier #5046

Enhancements

  • Re-factored Component implementation #4541
  • Provided a clock for generating publish timestamp for producers #4562
  • Reduced unnecessary track message calls #4595
  • Supported dynamic configure with escape char #4611
  • Added null check for function/source/sink configs #4627
  • Added delete dynamic config api #4614
  • Made broker replication mtls configuration dynamic #4609
  • Added authorization to function worker REST endpoints #4628
  • Improved and add authorization to function download and upload #4644
  • Allowed consumer retrieve the sequence id that the producer set #4645
  • Added perPartition parameter to partitioned-stats API #4639
  • Supported Pulsar schema for pulsar kafka client wrapper #4534
  • Supported delete and update event for JDBC Sink #4358
  • Cleaned up tests in the presto module #4683
  • Added allowAutoTopicCreation to broker.conf and related configuration #4694
  • Changed to use classloaders to load Java functions #4685
  • Removed fixed server type check in kerberos #4758
  • Changed type of publish_time to timestamp #4757
  • Added read-timeout to admin-request #4762
  • Added checking of deleted schema when adding schema #4731
  • Added strand to C++ client for exclusive control #4750
  • Added support to create partitioned topic with 1 partition #4764
  • Added getters and setters to PulsarService & BrokerService #4709
  • Added configure ack-timeout tick time #4760
  • Added options to rewrite namespace delimiter for pulsar sql #4749
  • Made Pulsar SQL supports pulsar's primitive schema #4728
  • Added basic authentication capabilities to Pulsar SQL #4779
  • Improved SchemaInfoProvider to fetch schema info asynchronously #4836
  • Supported KeyValue schema use AUTO_CONSUME as key/value schema #4839
  • Provided action type for insert in pulsar-io-jdbc #4862
  • Made partition as internal column in pulsar-sql #4888
  • Added option to disable authentication for proxy /metrics #4921

2.4.0 — 2019-06-30

PIPs

Fixes

  • Fix possible message loss using peer-cluster feature #3426
  • Fix pulsar standalone does not read zk port from conf/standalone.conf #3790
  • Fix some issues of ZkIsolatedBookieEnsemblePlacementPolicy #3917 #3918
  • Fix NPE when unload non-existent topic #3946
  • Fix race condition while deleting global topic #4173
  • Fix deadlock on skip messages #4411
  • Fix NPE when closing batch during a reconnection #4427
  • Fix race condition of read-timeout task in managed ledger #4437
  • Disable sticky read by default #4526
  • Fix race condition between timeout-task and add-call complete #4455

Enhancements

  • Optimize message replay for large backlog consumer #3732
  • Added support for websocket produce/consume command #3835
  • Added support for TTL config in broker.conf #3898
  • Reduce memory used in ClientCnx for pending lookups #4104
  • Reduce number of hashmap sections for ledger handles cache #4102
  • Added backlog and offloaded size in Prometheus stats #4150
  • Added support for configure the managed ledger cache eviction frequency #4066
  • Added support to avoid payload copy when inserting into managed ledger cache #4197
  • Added support to cache unack-messageId into OpenRangeSet #3819
  • Added support configure static PulsarByteBufAllocator to handle OOM errors #4196
  • Auto refresh new tls certs for jetty webserver #3645
  • Create non-persistent topic by pulsar-admin/rest api #3625
  • Consumer priority-level in Failover subscription #2954

Security

  • Added support for other algorithms in token auth #4528

Namespace Policies

  • Added support for tenant based bookie isolation #3933
  • Added support for secondary bookie isolation group at namespace #4458
  • Added support for secondary bookie-isolation-group #4261
  • Added support for replicator rate limit between clusters #4273
  • Disable backlog quota check by default #4320

Tiered Storage

  • Added support for Pulsar SQL to read data from tiered storage #4045

Pulsar Schema

  • Added schema versioning to support multi version messages produce and consume #3876 #3670 #4211 #4325 #4548
  • Added TRANSITIVE schema check strategies to support compatibility check over all existing schemas #4214
  • Added schema data validator #4360
  • Added support for delete schema when deleting a topic #3941
  • Added generic record builder #3690

Pulsar IO

  • Added IO connector for flume source and sink #3597
  • Added IO connector for redis sink #3700
  • Added IO connector for solr sink #3885
  • Hide kafka-connecter details for easy use debezium connector #3825
  • Added IO connector for debezium PostgreSQL source #3924
  • Enhancements for RabbitMQ source configuration #3937
  • Added IO connector for RabbitMQ sink #3967
  • Added IO connector for InfluxDB sink #4017

Pulsar functions

  • Added support for authentication #3735 #3874 #4198
  • Fix NPE when stats manager not initialized #3891
  • Added async state manipulation methods #3798
  • Fix fail to update functions in effectively-once mode #3993
  • Added labels to function statefulsets and services #4038
  • Added support for set key for message when using function publish #4005
  • Use negative acknowledge to instead ackTimeout #4103
  • Fix backward compatibility with 2.2 auth not working #4241

Java client

  • Added negative acks #3703
  • Added support for backoff strategy configuration #3848
  • Added support for configure TypedMessageBuilder through a Map conf object #4015
  • Added interceptor for negative ack send #3962
  • Added support for seek operate on reader #4031
  • Store key part of KeyValue schema into pulsar message keys #4117
  • Added interceptor for ack timeout #4300
  • Added support for snappy compression #4259
  • Added support for key based batcher #4435

Python client

  • Added negative acks #3816
  • Added support for snappy compression #4319

Go client

  • Added negative acks #3817
  • Added support for go schema #3904
  • Added support for snappy compression #4319
  • Added support for Key_Shared subscription #4465

C++ client

  • Added negative acks #3750
  • Fix ack timeout when subscribing to regex topic #3897
  • Added support for Key_Shared subscription #4366

Adaptors

  • In Kafka client wrapper, added some configurations #3753 #3797 #3843 #3887 #3991
  • In Apache Flink connector, allow to specify a custom Pulsar producer #3894 and client authentication #3949
  • In Apache Flink connector, added support for accept ClientConfigurationData, ProducerConfigurationData, ConsumerConfigurationData #4232
  • In Apache Storm connector, fix NPE while emitting next tuple #3991 and some add some enhancements #4280 #4239 #4238 #4236 #4495 #4494

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/20?closed=1

https://github.com/apache/pulsar/releases/tag/v2.4.0

2.3.2 — 2019-05-30

Fixes

  • Validate admin operation on topic with authoritative parameter #4270
  • fix bug with source local run #4278
  • fix cannot use size (type _Ctype_int) as type _Ctype_ulong #4212
  • Fix segfault in c++ producer #4219
  • AlwaysCompatible doesn't use AlwaysSchemaValidator in 2.3.1 component/schemaregistry #4181
  • Avoid potentially blocking method during topic ownership check #4190
  • [pulsar-broker]Fix: client-producer can't reconnect due to failed producer-future on cnx cache #4138
  • Removing # TYPE comment from topic metrics in Prometheus #4136
  • For functions metrics in prometheus also remove TYPE #4081
  • Fix: set receive queue size for sinks #4091
  • Fix: Exception when switch cluster from auth enabled to auth disabled #4069
  • Fix update cli source sink #4061
  • Fix connectors nested configs #4067
  • For functions metrics, avoid having HELP #4029
  • Fix Python functions state which is completely broken #4027
  • [issue #3975] Bugfix NPE on non durable consumer #3988
  • Fix: Function auth should ignore exception because it might be anonymous user #4185
  • [pulsar-function] fix worker-stats broken admin-api #4068
  • fix errors in sql doc #4030
  • Fix the swagger files generated by removing troublesome class #4024
  • [pulsar-function] fix broken backward compatibility with v1-namespace while registering function #4224
  • Revert dup consumer and related code #4142
  • [issue 4274][pulsar-io]Add double quotation marks for metrics with remote_cluster #4295

Enhancements

  • By default, auto configure the size of Bookie read/write cache #4297
  • Upgrade to BookKeeper 4.9.2 #4288
  • [pulsar-function] support bookie authentication from function-worker #4088
  • Optimizing performance for Pulsar function archive download #4082
  • allow users to update output topics for functions and sources #4092
  • improve data-generator source performance #4058
  • [client] Set actual topic name to partitioned consumer #4064
  • ack records in datagenerator print sink #4052
  • [security] Upgrade athenz libraries #4056
  • [python client] Handle subrecords in JsonSchema encoding #4023
  • [Issue 3987][pulsar-broker]Handle config is null when create tenant #4019
  • Add bookkeeper client version constraint #4013
  • Improve error handling for triggering function when there is a schema mismatch #3995
  • [pulsar-broker] add producer/consumer id in error-logging #3961

2.3.1 — 2019-04-12

Fixes

  • Fixed C++ batch acks tracker to evict message from sendList array. This was causing a slowdown in C++ consumers #3618
  • Allow publishing messages >5MB with batching (when they compress to <5MB) #3673 and #3718
  • Use at least 8 threads in Jetty thread pool. This fixed deadlocks in Jetty requests handling #3776
  • Fixed Reader.HasNext() in Go client #3764
  • Fixed increasing consumer permits after ack dedup operation. #3787
  • Set the dedup cursor as "inactive" after recovery #3612
  • Fix read batching message by pulsar reader #3830
  • Fix submit function with code specified via URL #3934
  • Fixed reader reading from a partition #3960
  • Fixed issue with Authorization header missing after client gets redirected #3869

Enhancements

  • Added producer.flush() on Python #3685
  • Introduced schema builder to define schema #3682
  • Support passing schema definition for JSON and AVRO schemas #3766
  • Exposing InitialPosition setting in Python consumer #3714

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/21?closed=1

https://github.com/apache/pulsar/releases/tag/v2.3.1

2.3.0 — 2019-02-20

General

  • Support for schema definitions in the Pulsar Python client library
  • PIP-25: Token based authentication #2888, #3067 and #3089
  • Updated to Apache BookKeeper 4.9.0
  • ZStandard compression codec #3159. Note that when a producer choose Zstd compression, a consumer will need to be at least at version 2.3.0 to be able to correctly receive the messages.
  • Support for Java 11 #3006
  • Added Schema.AUTO_PRODUCE type to allow to publish serialized data and validate it against the topic schema #2685
  • Added Schema.KeyValue to allow for schema to be validated on message keys as well as payloads. #2885
  • Support TLS authentication and authorization in standalone mode #3360
  • When creating namespace, use local cluster by default #3571
  • Tag BookKeeper ledgers created by Pulsar with topic/subscription names for info/debug purposes #3525
  • Enabled sticky reads in BooKeeper reads to increase IO efficiency with read-ahead #3569
  • Several optimization in Pulsar SQL Presto connector (#3128, #3135, #3139, #3144, #3143)
  • Configure Pulsar broker data cache automatically from JVM settings #3573
  • Reuse the SSL context objects #3550
  • Automatic schema update can be disabled through admin interface #2691
  • Support Dead-Letter-Queue from WebSocket proxy #2968
  • Pull-mode for WebSocket proxy #3058
  • Export Jetty stats to Prometheus #2804
  • Added stats for Pulsar proxy #2740
  • Allow subscribers to access subscription admin-api #2981
  • Make brokers read on closest Bookie in a multi-region deployment #3171

Fixes

  • Fixed deadlock in reusing ZookKeeper event thread #3591
  • In functions log topic appender, don't set producer name #3544
  • When cursor recovery encounters empty cursor ledger, fallback to latest snapshot #3487
  • Fixed C++ regex-consumer when using HTTP service URL #3407
  • Fix race condition: broker not scheduling read for active consumer #3411

Pulsar IO

  • Added Debezium connector for Change-Data-Capture into Pulsar #2791
  • Added MongoDB connector #3561
  • Added Elastic Search connector #2546
  • Added HBase sink #3368
  • Added Local files connector #2869
  • Report source/sink stats in Prometheus #3261
  • Allow filtering in Twitter Firehose connector #3298
  • Sources/Sinks can be launched using fat jars as well #3166

Pulsar Functions

  • Added Kubernetes runtime #1950
  • Secrets interface #2826
  • Cleanup subscriptions when deleting functions #3299
  • Add Windowfunction interface to functions api #3324
  • Support for accessing state in Python #2714
  • Support submitting Python functions as wheel file
  • Support submitting Python functions as Zip file with dependencies included #3321
  • Add minimum amount of resources to run setting for functions #3536
  • Fixed the behavior of Function start/stop #3477

Java client

  • Moved Pulsar v1 client API into separate artifact #3228.
    Applications that are using the Pulsar v1 API, deprecated since 2.0 release, need to update the Maven dependency to use the pulsar-client-1x artifact instead of pulsar-client. Eg. 
    <dependency>
    <groupId>org.apache.pulsar</groupId>
    <artifactId>pulsar-client-1x</artifactId>
    <version>2.3.0</version>
</dependency>
  • Fixed shading issues with Javadoc bundled in client jars by separating the API in a different Maven module #3309
  • Improve Javadocs #3592
  • Support specifying multiple hosts in pulsar service url and web url #3249
  • Automatically discover when partitions on a topic are increased #3513
  • Added Client.getPartitionsForTopic() #2972 (Javadoc)
  • Added Consumer.pauseMessageListener() and Consumer.resumeMessageListener() #2961
  • Removed shading relocations for Circe-checksum and lz4 libraries, to ensure native libraries are correctly loaded when using shaded client lib. #2191

Python client

  • Fixed Message.properties() #3595

Go client

  • Added Producer.flush() to flush all outstanding messages #3469
  • Support Consumer.Seek() #3478
  • Added Message.Topic() #3346
  • Allow to specify SubscriptionInitPos option in ConsumerOptions #3588
  • Added TLS hostname verification #3580
  • Allow to link statically against libpulsar.a#3488
  • Expose Producer.LastSequenceID() and Message.SequenceID() #3416

C++ client

  • Enable batching by default when using sendAsync() #2949
  • Allow to specify schema info in Avro format #3354
  • Added Producer.flush() to flush all outstanding messages #3020
  • Added TLS hostname verification #2475
  • Allow to specify SubscriptionInitialPosition #3567
  • Added Message.getTopicName() #3326
  • Added Cosnsumer.receiveAsync() #3389
  • Build libpulsar.a with all required dependencies #3488
  • Removed Boost from Pulsar API headers #3374

Adaptors

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/18?closed=1

https://github.com/apache/pulsar/releases/tag/v2.3.0

2.2.1 — 2018-12-31

This release includes fixes for 2.2.0 release. In particular:

  • Fixed issue when proxy HTTP admin API requests Pulsar proxy #3022

  • Fixed Consumer.unsubscribe() in Python client library #3093

  • Fixed ZLib decompression in C++ client #2854

  • Fixed Pulsar functions context publish in Python #2844

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/19?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.2.1

2.2.0 — 2018-10-24

This is the first release of Pulsar as an Apache Top Level Project

This is a feature release, including several new features, improvements and fixes for issues reported for 2.1.1-incubating.  

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/16?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.2.0

Apache incubator

2.1.1-incubating — 2018-09-17

This release fixes issues reported for 2.1.0-incubating.

  • #2473 - Downgrading ZK to stable version 3.4.13
  • #2219 - Cpp client: add PatternMultiTopicsConsumerImpl to support regex subscribe
  • #2387 - Fixed race condition during expansion of concurrent open hash maps
  • #2348 - Fix NPE when splitting and unloading bundle
  • #2223 - fix bug in FunctionRuntimeManager involving not cleaning up old invalid assignments
  • #2367 - [compaction] make topic compaction works with partitioned topic
  • #2203 - Make sure schema is initialized before the topic is loaded

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/17?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.1.1-incubating

2.1.0-incubating — 2018-08-02

This is the seventh release of Apache Pulsar since entering the ASF incubator.

It is a feature release, including several new features and major improvements:

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/13?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.1.0-incubating

2.0.1-incubating — 2018-06-18

This release fixes issues reported for 2.0.0-rc1-incubating.

  • #1893 - Fixed issues with Python packages on PyPI
  • #1797 - Proxy doesn't strip the request URL for admin requests correctly
  • #1862 - Fix REST APIs provided by Pulsar proxy

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/14?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.0.1-incubating

1.22.1-incubating — 2018-06-18

This is the sixth release of Apache Pulsar since entering the ASF incubator.

This release addresses issues reported in 1.22.0-incubating version.

  • #1660 Deadlock while closing non persistent topic
  • #1591 Deadlock while closing non shared consumer
  • #1554 Handle invalid mark delete position at managed cursor
  • #1262 Broker should not start replicator for root partitioned topic
  • #1662 NPE when cursor failed to close empty subscription
  • #1370 Relocate service files for shading pulsar-client-admin module
  • #1265 Fixed lookup redirect logic on Proxyside
  • #1428 Handle Race condition in concurrent bundle split
  • #1817 Fixed mem leak when acknowledging while disconnected from broke
  • #1851 Fixing resource leak due to open file descriptors in SecurityUtility.

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/15?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.22.1-incubating

2.0.0-rc1-incubating — 2018-05-29

This is the fifth release of Apache Pulsar since entering the ASF incubator and the first time we increase major release number.

There are several new features and major improvements:

  • Pulsar functions: Lightweight compute framework
  • New type-safe Java API for producer/consumers
  • Schema registry — Enforce schema on topics
  • Topic compaction — Out of band compaction of messages to allow consumer to fetch a snapshot with last published message for each message key.
  • Upgraded to Apache BookKeeper 4.7.0
  • Performance improvements — Up to 3x throughput improvements compared to Pulsar-1.22 and 99.9 Pct publish latencies <10ms
  • Simplified terminology and admin tools
    • Renamed "property" into "tenants"
    • Short topic names: my-topic
    • Topics independent of cluster names: my-tenant/my-namespace/my-topic

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/12?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v2.0.0-rc1-incubating

1.22.0-incubating — 2018-03-06

This is the fourth of Apache Pulsar since entering the ASF incubator.

Major changes in this release include:

Features

  • #896 PIP-7 Introduce Failure-domain and Anti-affinity-namespace group
  • #1031 Add optional key/value metadata to producers/consumers
  • #1129 Added end to end encryption in C++ client
  • #1151 Added REST handler to create a subscription on a topic
  • #1087 Add basic authentication plugin
  • #1200 Add pluggable authorization mechanism
  • #1208 Add hostname-verification at client tls connection
  • #950 Provided an DCOS Universe package for pulsar
  • #1046 Introduce config to skip non-recoverable data-ledger
  • #899 Add subscription auth mode by prefix
  • #1135 Added infinite time retention configuration option

Enhancements

  • #1094 Include BoringSSL native implementation for faster TLS
  • #1204 Reduce size of buffer used to assemble batches
  • #930 Perform async DNS resolution
  • #1124 Support Pulsar proxy from C++/Python client library
  • #1012 Made load shedding for load manager Dynamically configurable
  • #962 Raw Reader for Pulsar Topics
  • #941 Upgraded Jackson version
  • #1002, #1169, #1168 Making Pulsar Proxy more secure
  • #1029 Fix MessageRouter hash inconsistent on C++/Java client

Fixes

  • #1153 Fixed increase partitions on a partitioned topic
  • #1195 Ensure the checksum is not stripped after validation in the broker
  • #1203 Use duplicates when writing from ByteBuf pair to avoid multiple threads issues
  • #1210 Cancel keep-alive timer task after the proxy switch to TCP proxy
  • #1170 Upgrade BK version: BK-4.3.1.91-yahoo (fix: stats + DoubleByteBuf)
  • #875 Bug fixes for Websocket proxy

The complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/11?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.22.0-incubating

1.21.0-incubating — 2017-12-17

This is the third of Apache Pulsar since entering the ASF incubator.

Major changes in this release include:

  • #689 Upgrade to Netty 4.1
  • #846 Publish the shaded pulsar-client as the default dependency
  • #832 #833 #849 #852 Enhancements to Kafka API wrapper to have it work with Kafka's own benchmark tools
  • #836 Fix to C++ partitioned consumer client
  • #822 #826 Several fixes and improvements related to the namespace bundles
  • #848 Allow consumer to seek to message id from within Pulsar client
  • #903 PIP-8: Scale Pulsar beyond 1M topics
  • #824 Enable secure replication over TLS
  • #923 Upgrade to bk-4.3.1.83-yahoo to expose journalSyncData option
  • #807 Prevent message duplication when active consumer is changed

Complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/10?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.21.0-incubating

1.20.0-incubating — 2017-08-08

This is the second of Apache Pulsar since entering the ASF incubator.

Major changes in this release include:

Complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/9?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.20.0-incubating

1.19.0-incubating — 2017-08-08

This is the first of Apache Pulsar since entering the ASF incubator.

Major changes included in this release are:

  • #524 Moved APIs from com.yahoo.pulsar to org.apache.pulsar
  • #548 Added stateless Pulsar proxy
  • #538 Support for non-persistent topics
  • #587 Upgraded RocksDB to comply with ASF policy
  • #507 Instrumentation of ZooKeeper client to expose metrics
  • Various fixes for TLS auth in WebSocket proxy

Complete list of changes can be found at: https://github.com/apache/incubator-pulsar/milestone/8?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.19.0-incubating

Pre-Apache

1.18 — 2017-06-17

Main changes:

  • #325 Add Modular load manager documentation
  • #329 Add api to get list of partitioned topics
  • #296 Added spark streaming custom receiver for pulsar
  • #317 HTTP lookups for c++ client lib
  • #332 Fix: Modular load manager bug fixes
  • #352 Fix: Delete local-policies and invalidate cache when namespace is dele
  • #356 Fix: WebSocket TLS connection bug
  • #363 Use binary protocol lookup for connection between WebSocket proxy and broker
  • #375 Fix: Bug fixes on deadlock while topic loading failure
  • #376 Fix: Avoid incrementing unack-msg count for non-shared sub and not show it on stats
  • #329 Fix: Handle zkCache failures
  • #387 Pass client library version to broker and show on stats
  • #345 Add load shedding strategy
  • #393 Change default mark-delete rate limit from 10s to 1s
  • #392 Upgrade to netty-4.0.46
  • #366 NonDurable cursor for managed ledger
  • #371 Introduce topic reader in client API
  • #341 Add stats and monitoring for websocket proxy
  • #299 Add api to increase partitions of existing non-global partitioned-topic
  • #294 Add endpoint to fetch stats for Prometheus
  • #440 Enable PulsarAdmin to trust multiple certificates
  • #442 Fix: Remove broker weights for ModularLoadManager
  • #446 Fix: Recover cursor with correct readPosition and replay unackedMessages
  • #441 Set Block If queue full to false by default
  • #447 Fix: DoubleByteBuf to send large size messages in TLS mode
  • #443 Add topic termination option
  • #436 Added ZooKeeper instrumentation for enhanced stats
  • #448 WebSocket proxy should not make a consumer/producer when authorization is failed
  • #443 Add Docker images definition and instruction to deploy on Kubernetes
  • #474 Fix: message rate out with batches to count messages/s
  • #482 Allow client(producer/consumer) to check topic stats
  • #468 Pulsar Python client library
  • #386 Increment bookkeeper version to 4.3.1.69-yahoo

Full list of changes: https://github.com/yahoo/pulsar/milestone/7?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.18

1.17.5 — 2017-05-02

  • #343 Fix ModularLoadManager to select broker from current available-broker list
  • #384 Fix Send replay entries read callback from background thread, to avoid recursive stack calls
  • #390 Fix Shaded AsyncHttpClient in pulsar client
  • #374 Fix Remove Exceptionally Completed Topic Futures

https://github.com/apache/incubator-pulsar/releases/tag/v1.17.5

1.17.4 — 2017-04-25

  • #362 Fix add timeout on blocking ZookeeperCache get call
  • #375 Fix possible deal lock on topic loading if broker fails to get MLConfiguration from zk
  • #377 Fix zkCache error handling and zk-callback processing on separate dedicated thread

https://github.com/apache/incubator-pulsar/releases/tag/v1.17.4

1.17.3 — 2017-04-20

  • #367 Fix dispatcher correctly finds available consumer from list of shared-subscription consumers

https://github.com/apache/incubator-pulsar/releases/tag/v1.17.3

1.17.2 — 2017-04-06

  • #327 Create znode for dynamic configuration if not present
  • #336 Fix prevent creation of topic when bundle is disable
  • #338 Fix deserialize load report based on load-manager

https://github.com/apache/incubator-pulsar/releases/tag/v1.17.2

1.17.1 — 2017-03-30

  • #326 Fix memory leak while duplicating entry data from existing entry

https://github.com/apache/incubator-pulsar/releases/tag/v1.17.1

1.17 — 2017-03-30

Main changes:

  • #188 Pulsar Dashboard
  • #276 Broker persist individually deleted messages
  • #282 Support binary format to persist managed-ledger info in ZK
  • #292 Added REST and CLI tool to expose ManagedLedger metadata
  • #285 Add documentation in japanese
  • #178 Add Athenz authentication plugin
  • #186 Update Broker service configuration dynamically
  • #215 Fix Broker disconnects unsupported batch-consumer on batch-message topic
  • #165 Message dispatching on consumer priority-level
  • #303 Introduce new load manager implementation
  • #306 Add topic loading throttling at broker
  • #302 Update BK version to 4.3.1.60-yahoo to include: 64bit ledger-ids, fix: memory leak on read-only bookie and datasketches concurrency issue
  • #216 Binary proto api to get consumer stats
  • #225 Server lookup throttling
  • #182 Client lookup request throttling and server-error handling
  • #265 Fix client handling on http server error
  • #204 Fix discovery service redirection
  • #311 Fix netty package conflict at binary distribution
  • #221 Fixed race condition on client reconnection logic
  • #239 Fix replicator handling on closed cursor
  • #318 GC improvements: Recyclable entry and reduce collection on stats generation

Full list of changes: https://github.com/apache/incubator-pulsar/milestone/3?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.17

1.16.5 — 2017-03-10

  • #311 Exclude netty individual jars from binary distribution. This issue was causing binary distribution to have conflicting netty dependencies.

https://github.com/apache/incubator-pulsar/releases/tag/v1.16.5

1.16.4 — 2017-03-10

  • #265 Fix client closes http-connection on internal-server error
  • #283 Fix recycle keep alive command-object properly
  • #284 Reduce usage of collections in managed-ledger metrics-generation to reduce GC impact

https://github.com/apache/incubator-pulsar/releases/tag/v1.16.4

1.16.3 — 2017-03-01

  • #275 Fix for Infinite loop in PersistentReplicator.startProducer()

https://github.com/apache/incubator-pulsar/releases/tag/v1.16.3

1.16.2 — 2017-02-24

  • #250 : Disconnect consumers without closing dispatcher on cursor-reset

https://github.com/apache/incubator-pulsar/releases/tag/v1.16.2

1.16.1 — 2017-02-24

  • #221 Fixed race condition while creating client connection
  • #223 Fixed broker's direct memory usage count
  • #220 Fixed stuck replicator producer on backlog quota exception
  • #239 Fixed replicator stop reading on already closed cursor

https://github.com/apache/incubator-pulsar/releases/tag/v1.16.1

1.16 — 2017-02-02

Main changes:

  • #76 Async Zookeeper cache implementation
  • #105 Support topic lookup using pulsar binary protocol
  • #164 Fixed handling failure of unloading namespace bundle
  • #166 Support websocket proxy deployment without passing globalZK
  • #161 Fixed avoiding creation of duplicate replicator
  • #160 Add support uri encoding on broker admin rest api
  • #143 Include DataSketches metrics provider for bookie stats
  • #127 Updated BK-4.3.1.45/47-yahoo to include bookie/bookkeeper-client bug-fixes and DataSketch metrics provider
  • #124 Consumer-stats: Add blockedConsumer flag
  • #95 Consumer-stats: Add message redelivery rate
  • #123 Fixed Batch message replication
  • #106 Fixed Partitioned consumer should avoid blocking call to fill shared queue
  • #139 Support online consumer cursor reset
  • #187 Support custom advertised address in pulsar standalone

Full list of changes: https://github.com/yahoo/pulsar/milestone/2?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.16

1.15.7 — 2017-01-25

  • #174 Handling bundle unloading failure

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.7

1.15.6 — 2017-01-20

  • #171 Fix: Consumer redelivery should not wipeout availablePermits

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.6

1.15.5 — 2017-01-03

  • #159 Fix: Replicator-cleanup while closing replicator at broker.
  • #160 Fix: Http lookup for topic with special character

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.5

1.15.4 — 2016-12-14

  • #146 Fix: Partitioned consumer can consume messages with receiverQueueSize 1.

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.4

1.15.3 — 2016-12-13

  • #145 Fixed issue Partitioned-consumer aggregate messages without blocking internal listener thread

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.3

1.15.2 — 2016-11-03

  • #102 Fixed issue with message dispatching while message-replay at broker

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.2

1.15.1 — 2016-10-27

  • #89 Fixed issue with replication in a mixed environment with 1.14 and 1.15 brokers

https://github.com/apache/incubator-pulsar/releases/tag/v1.15.1

1.15 — 2016-10-18

  • #39 Updated BookKeeper version to 4.3.1.41-yahoo to include bookie storage improvements
  • #17 Fixed memory leak in stats generation buffer rollover
  • #27 Fixed issues with discovery service component when HTTPS is enabled
  • #43 Add end-to-end crc32c checksum verification on message header and payload, rather than just payload. Support for intel hardware instructions to speed up computation.
  • #26 Added ability to configure the address that the broker uses to advertise itself. Needed in cases where the public hostname/ip is different than the machine interface ip (eg: in AWS EC2 instances).
  • #38 Prevent message-replay of already acknowledged messages
  • #51 Per message unacknowledged redelivery. When ack-timeout is configured, only request redelivery of messages that effectively have the timeout expired, instead of all the messages dispatched to the consumer.
  • #48 Add unacknowledged messages threshold to stop delivery to consumer that are not acknowledging messages
  • #59 Added admin method to do a one-time messages time expiration for a given subscription (independently from the TTL configured at the namespace level)

Full list of changes: https://github.com/apache/incubator-pulsar/milestone/1?closed=1

https://github.com/apache/incubator-pulsar/releases/tag/v1.15

1.14 — 2016-08-31

First public release of Pulsar

https://github.com/apache/incubator-pulsar/releases/tag/v1.14