Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude jackson-databind from shaded jar #50

Open
wants to merge 1 commit into
base: branch-0.7
Choose a base branch
from
Open

Exclude jackson-databind from shaded jar #50

wants to merge 1 commit into from

Conversation

jrkinley
Copy link

@jrkinley jrkinley commented Mar 1, 2021

Exclude jackson-databind from shaded .jar as it clashes with Spark's version in Cloudera Runtime 7.1.x. Looking at the dependency tree Spark includes jackson-databind:2.6.5 whereas Envelope 0.7.2 is including jackson-databind:2.3.1 as part of a kitesdk dependency.

@jrkinley
Copy link
Author

jrkinley commented Mar 1, 2021

Spark throws this exception when both versions of the jar are on the classpath:

Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;
	at org.apache.hadoop.hive.common.StatsSetupConst$ColumnStatsAccurate.<clinit>(StatsSetupConst.java:176)
	at org.apache.hadoop.hive.common.StatsSetupConst.parseStatsAcc(StatsSetupConst.java:353)
	at org.apache.hadoop.hive.common.StatsSetupConst.setBasicStatsState(StatsSetupConst.java:241)
	at org.apache.hadoop.hive.common.StatsSetupConst.setStatsStateForCreateTable(StatsSetupConst.java:345)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2443)
	at org.apache.hadoop.hive.ql.metadata.Hive.lambda$loadDynamicPartitions$4(Hive.java:2909)
	... 4 more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant