Skip to content

Commit

Permalink
Add config of nar extraction directory for the broker (apache#15778)
Browse files Browse the repository at this point in the history
### Motivation

Currently, some users want to change `narExtractionDirectory` config, but users may be confused, because the `broker.conf` and `standalone.conf` config files don't Include `narExtractionDirectory` config.

### Modifications

Add config of NAR extraction directory for the broker configuration
  • Loading branch information
Demogorgon314 authored Jun 16, 2022
1 parent eb2a870 commit f77e72a
Show file tree
Hide file tree
Showing 11 changed files with 23 additions and 4 deletions.
5 changes: 5 additions & 0 deletions conf/broker.conf
Original file line number Diff line number Diff line change
Expand Up @@ -1364,6 +1364,11 @@ managedLedgerOffloadDriver=
# Maximum number of thread pool threads for ledger offloading
managedLedgerOffloadMaxThreads=2

# The extraction directory of the nar package.
# Available for Protocol Handler, Additional Servlets, Entry Filter, Offloaders, Broker Interceptor.
# Default is System.getProperty("java.io.tmpdir").
narExtractionDirectory=

# Maximum prefetch rounds for ledger reading for offloading
managedLedgerOffloadPrefetchRounds=1

Expand Down
5 changes: 5 additions & 0 deletions conf/standalone.conf
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,11 @@ isSchemaValidationEnforced=false
# Available values: ALWAYS_INCOMPATIBLE, ALWAYS_COMPATIBLE, BACKWARD, FORWARD, FULL, BACKWARD_TRANSITIVE, FORWARD_TRANSITIVE, FULL_TRANSITIVE
schemaCompatibilityStrategy=FULL

# The extraction directory of the nar package.
# Available for Protocol Handler, Additional Servlets, Entry Filter, Offloaders, Broker Interceptor.
# Default is System.getProperty("java.io.tmpdir").
narExtractionDirectory=

### --- Deprecated config variables --- ###

# Deprecated. Use configurationStoreServers
Expand Down
3 changes: 2 additions & 1 deletion site2/docs/reference-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
| enableExposingBrokerEntryMetadataToClient|Whether to expose broker entry metadata to client or not.<br /><br />Available values:<li>true</li><li>false</li><br />Example<br />enableExposingBrokerEntryMetadataToClient=true | false |
| metricsBufferResponse | The configuration is for those broker which there are more than one metrics system access the `/metrics` endpoint. For the purpose of reduce `CPU` and `Memory` usage, metrics data will be generated once in the interval(`managedLedgerStatsPeriodSeconds`) and it will be cached, all `/metrics` requests in the `interval` will return same metrics | false |
| strictBookieAffinityEnabled | Enable or disable the strict bookie isolation strategy. If enabled, <br /> - `bookie-ensemble` first tries to choose bookies that belong to a namespace's affinity group. If the number of bookies is not enough, then the rest bookies are chosen. <br /> - If namespace has no affinity group, `bookie-ensemble` only chooses bookies that belong to no region. If the number of bookies is not enough, `BKNotEnoughBookiesException` is thrown.| false |
|managedCursorInfoCompressionType | The compression type of managed cursor information. <br />Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br />If this value is `NONE`, managed cursor information is not compressed. | NONE
|managedCursorInfoCompressionType | The compression type of managed cursor information. <br />Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br />If this value is `NONE`, managed cursor information is not compressed. | NONE |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Entry Filter, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

#### Configuration Override For Clients Internal to Broker

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
| brokerEntryMetadataInterceptors | Set broker entry metadata interceptors.<br /><br />Multiple interceptors should be separated by commas. <br /><br />Available values:<li>org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor</li><li>org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor</li> <br /><br />Example<br />brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendBrokerTimestampMetadataInterceptor, org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor|N/A |
| enableExposingBrokerEntryMetadataToClient|Whether to expose broker entry metadata to client or not.<br /><br />Available values:<li>true</li><li>false</li><br />Example<br />enableExposingBrokerEntryMetadataToClient=true | false |
| strictBookieAffinityEnabled | Enable or disable the strict bookie isolation strategy. If enabled, <br /> - `bookie-ensemble` first tries to choose bookies that belong to a namespace's affinity group. If the number of bookies is not enough, then the rest bookies are chosen. <br /> - If namespace has no affinity group, `bookie-ensemble` only chooses bookies that belong to no region. If the number of bookies is not enough, `BKNotEnoughBookiesException` is thrown.| false |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Entry Filter, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |


#### Deprecated parameters of Broker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
|haProxyProtocolEnabled | Enable or disable the [HAProxy](http://www.haproxy.org/) protocol. |false|
| maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 |
|subscriptionTypesEnabled| Enable all subscription types, which are exclusive, shared, failover, and key_shared. | Exclusive, Shared, Failover, Key_Shared |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
|haProxyProtocolEnabled | Enable or disable the [HAProxy](http://www.haproxy.org/) protocol. |false|
| maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 |
|subscriptionTypesEnabled| Enable all subscription types, which are exclusive, shared, failover, and key_shared. | Exclusive, Shared, Failover, Key_Shared |
/ managedLedgerInfoCompressionType / ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
| managedLedgerInfoCompressionType | ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
|haProxyProtocolEnabled | Enable or disable the [HAProxy](http://www.haproxy.org/) protocol. |false|
| maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 |
|subscriptionTypesEnabled| Enable all subscription types, which are exclusive, shared, failover, and key_shared. | Exclusive, Shared, Failover, Key_Shared |
/ managedLedgerInfoCompressionType / ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
| managedLedgerInfoCompressionType | ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,8 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
|haProxyProtocolEnabled | Enable or disable the [HAProxy](http://www.haproxy.org/) protocol. |false|
| maxTopicsPerNamespace | The maximum number of persistent topics that can be created in the namespace. When the number of topics reaches this threshold, the broker rejects the request of creating a new topic, including the auto-created topics by the producer or consumer, until the number of connected consumers decreases. The default value 0 disables the check. | 0 |
|subscriptionTypesEnabled| Enable all subscription types, which are exclusive, shared, failover, and key_shared. | Exclusive, Shared, Failover, Key_Shared |
/ managedLedgerInfoCompressionType / ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
| managedLedgerInfoCompressionType | ManagedLedgerInfo compression type, option values (NONE, LZ4, ZLIB, ZSTD, SNAPPY), if value is NONE or invalid, the managedLedgerInfo will not be compressed. Notice, after enable this configuration, if you want to degrade broker, you should change the configuration to `NONE` and make sure all ledger metadata are saved without compression. | None |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
| managedLedgerInfoCompressionType | Compression type of managed ledger information. <br /><br />Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br /><br />If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br /><br />**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
| additionalServlets | Additional servlet name. <br /><br />If you have multiple additional servlets, separate them by commas. <br /><br />For example, additionalServlet_1, additionalServlet_2 | N/A |
| additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
| managedLedgerInfoCompressionType | Compression type of managed ledger information. <br /><br />Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br /><br />If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br /><br />**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
| additionalServlets | Additional servlet name. <br /><br />If you have multiple additional servlets, separate them by commas. <br /><br />For example, additionalServlet_1, additionalServlet_2 | N/A |
| additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
| managedLedgerInfoCompressionType | Compression type of managed ledger information. <br /><br />Available options are `NONE`, `LZ4`, `ZLIB`, `ZSTD`, and `SNAPPY`). <br /><br />If this value is `NONE` or invalid, the `managedLedgerInfo` is not compressed. <br /><br />**Note** that after enabling this configuration, if you want to degrade a broker, you need to change the value to `NONE` and make sure all ledger metadata is saved without compression. | None |
| additionalServlets | Additional servlet name. <br /><br />If you have multiple additional servlets, separate them by commas. <br /><br />For example, additionalServlet_1, additionalServlet_2 | N/A |
| additionalServletDirectory | Location of broker additional servlet NAR directory | ./brokerAdditionalServlet |
|narExtractionDirectory | The extraction directory of the nar package. <br />Available for Protocol Handler, Additional Servlets, Offloaders, Broker Interceptor. | System.getProperty("java.io.tmpdir") |

## Client

Expand Down

0 comments on commit f77e72a

Please sign in to comment.