Skip to content

Commit

Permalink
[SPARK-39943][BUILD] Upgrade rocksdbjni to 7.4.4
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
This PR aims to upgrade `rocksdbjni` to 7.4.4

### Why are the changes needed?
This will bring the latest bug fixes.

- https://github.com/facebook/rocksdb/releases/tag/v7.4.4

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass the CIs.

**BEFORE**
```
Running org.apache.spark.util.kvstore.RocksDBBenchmark
                                        	count	mean	min	max	95th
dbClose                                 	4	0.439	0.364	0.508	0.508
dbCreation                              	4	307.696	210.914	582.675	582.675
naturalIndexCreateIterator              	1024	0.005	0.002	1.266	0.010
naturalIndexDescendingCreateIterator    	1024	0.005	0.004	0.047	0.006
naturalIndexDescendingIteration         	1024	0.008	0.005	0.048	0.015
naturalIndexIteration                   	1024	0.008	0.005	0.042	0.016
randomDeleteIndexed                     	1024	0.032	0.024	0.275	0.050
randomDeletesNoIndex                    	1024	0.036	0.015	16.879	0.035
randomUpdatesIndexed                    	1024	0.084	0.042	21.731	0.110
randomUpdatesNoIndex                    	1024	0.066	0.027	0.450	0.122
randomWritesIndexed                     	1024	0.133	0.042	48.340	0.176
randomWritesNoIndex                     	1024	0.069	0.033	2.031	0.103
refIndexCreateIterator                  	1024	0.003	0.003	0.026	0.005
refIndexDescendingCreateIterator        	1024	0.002	0.002	0.034	0.003
refIndexDescendingIteration             	1024	0.007	0.005	0.039	0.008
refIndexIteration                       	1024	0.009	0.006	0.287	0.011
sequentialDeleteIndexed                 	1024	0.024	0.019	0.109	0.030
sequentialDeleteNoIndex                 	1024	0.017	0.015	0.059	0.018
sequentialUpdatesIndexed                	1024	0.037	0.031	0.643	0.058
sequentialUpdatesNoIndex                	1024	0.052	0.029	0.569	0.115
sequentialWritesIndexed                 	1024	0.058	0.044	2.907	0.073
sequentialWritesNoIndex                 	1024	0.051	0.040	2.901	0.060
```
**AFTER**

```
Running org.apache.spark.util.kvstore.RocksDBBenchmark
                                        	count	mean	min	max	95th
dbClose                                 	4	0.446	0.367	0.642	0.642
dbCreation                              	4	310.345	209.803	590.420590.420
naturalIndexCreateIterator              	1024	0.005	0.002	1.295	0.007
naturalIndexDescendingCreateIterator    	1024	0.005	0.004	0.049	0.006
naturalIndexDescendingIteration         	1024	0.008	0.005	0.313	0.011
naturalIndexIteration                   	1024	0.007	0.005	0.061	0.009
randomDeleteIndexed                     	1024	0.033	0.025	0.355	0.051
randomDeletesNoIndex                    	1024	0.017	0.014	0.062	0.018
randomUpdatesIndexed                    	1024	0.089	0.044	25.477	0.103
randomUpdatesNoIndex                    	1024	0.030	0.025	0.583	0.042
randomWritesIndexed                     	1024	0.130	0.047	49.815	0.167
randomWritesNoIndex                     	1024	0.035	0.027	2.217	0.047
refIndexCreateIterator                  	1024	0.003	0.003	0.037	0.005
refIndexDescendingCreateIterator        	1024	0.002	0.002	0.034	0.004
refIndexDescendingIteration             	1024	0.007	0.006	0.042	0.008
refIndexIteration                       	1024	0.009	0.006	0.054	0.015
sequentialDeleteIndexed                 	1024	0.023	0.019	0.125	0.030
sequentialDeleteNoIndex                 	1024	0.018	0.015	0.055	0.019
sequentialUpdatesIndexed                	1024	0.038	0.033	1.057	0.055
sequentialUpdatesNoIndex                	1024	0.074	0.037	0.558	0.111
sequentialWritesIndexed                 	1024	0.058	0.039	2.804	0.074
sequentialWritesNoIndex                 	1024	0.053	0.040	2.946	0.060
```

Closes apache#37370 from williamhyun/rocksdb744.

Authored-by: William Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
williamhyun authored and dongjoon-hyun committed Aug 2, 2022
1 parent 83966e8 commit 37074c5
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion dev/deps/spark-deps-hadoop-2-hive-2.3
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ pickle/1.2//pickle-1.2.jar
protobuf-java/2.5.0//protobuf-java-2.5.0.jar
py4j/0.10.9.5//py4j-0.10.9.5.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
rocksdbjni/7.4.3//rocksdbjni-7.4.3.jar
rocksdbjni/7.4.4//rocksdbjni-7.4.4.jar
scala-collection-compat_2.12/2.2.0//scala-collection-compat_2.12-2.2.0.jar
scala-compiler/2.12.16//scala-compiler-2.12.16.jar
scala-library/2.12.16//scala-library-2.12.16.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/spark-deps-hadoop-3-hive-2.3
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ pickle/1.2//pickle-1.2.jar
protobuf-java/2.5.0//protobuf-java-2.5.0.jar
py4j/0.10.9.5//py4j-0.10.9.5.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
rocksdbjni/7.4.3//rocksdbjni-7.4.3.jar
rocksdbjni/7.4.4//rocksdbjni-7.4.4.jar
scala-collection-compat_2.12/2.2.0//scala-collection-compat_2.12-2.2.0.jar
scala-compiler/2.12.16//scala-compiler-2.12.16.jar
scala-library/2.12.16//scala-library-2.12.16.jar
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,7 @@
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
<version>7.4.3</version>
<version>7.4.4</version>
</dependency>
<dependency>
<groupId>${leveldbjni.group}</groupId>
Expand Down

0 comments on commit 37074c5

Please sign in to comment.