Skip to content

Commit

Permalink
jvm options
Browse files Browse the repository at this point in the history
  • Loading branch information
cherkavi authored Apr 22, 2022
1 parent 423b17c commit 8e5de2d
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,16 @@ spark.serializer=org.apache.spark.serializer.KryoSerializer

### Spark submit with output additional information
```sh
/opt/mapr/spark/spark-2.3.2/bin/spark-submit --verbose --master yarn --deploy-mode cluster \
...
/opt/mapr/spark/spark-2.3.2/bin/spark-submit --verbose --master yarn --deploy-mode cluster ...
```

### Spark submit with log info, logger output, spark-submit logger
```sh
export SPARK_SUBMIT_OPTS="-Dlog4j.configuration=log4j.properties"
# or
# export SPARK_SUBMIT_OPTS="-Dlog4j.debug=true -Dlog4j.configuration=log4j-local-usage-only.xml"
# or java -D option jvm options
# --driver-java-options "-Xms2G -Doracle.jdbc.Trace=true -Djava.util.logging.config.file=/opt/apache-spark/spark-2.3.0-bin-hadoop2.7/conf/oraclejdbclog.properties -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1098 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=192.168.2.120 -Dcom.sun.management.jmxremote.rmi.port=1095"

/opt/mapr/spark/spark-2.3.2/bin/spark-submit \
--conf "spark.driver.extraJavaOptions=-Dlog4jspark.root.logger=ERROR,console"
Expand Down

0 comments on commit 8e5de2d

Please sign in to comment.