Skip to content

Commit

Permalink
[SPARK-39703][CORE][BUILD] Mima complains with Scala 2.13 for the cha…
Browse files Browse the repository at this point in the history
…nges in DeployMessages

### What changes were proposed in this pull request?

This PR is a followup of apache#36716. Mima with Scala 2.13 complains about the changes in `DeployMessages` for some reasons:

```
[error] spark-core: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.2.0! Found 6 potential problems (filtered 933)
[error]  * the type hierarchy of object org.apache.spark.deploy.DeployMessages#LaunchExecutor is different in current version. Missing types {scala.runtime.AbstractFunction7}
[error]    filter with: ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.deploy.DeployMessages$LaunchExecutor$")
[error]  * method requestedTotal()Int in class org.apache.spark.deploy.DeployMessages#RequestExecutors does not have a correspondent in current version
[error]    filter with: ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.requestedTotal")
[error]  * method copy(java.lang.String,Int)org.apache.spark.deploy.DeployMessages#RequestExecutors in class org.apache.spark.deploy.DeployMessages#RequestExecutors's type is different in current version, where it is (java.lang.String,scala.collection.immutable.Map)org.apache.spark.deploy.DeployMessages#RequestExecutors instead of (java.lang.String,Int)org.apache.spark.deploy.DeployMessages#RequestExecutors
[error]    filter with: ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.copy")
[error]  * synthetic method copy$default$2()Int in class org.apache.spark.deploy.DeployMessages#RequestExecutors has a different result type in current version, where it is scala.collection.immutable.Map rather than Int
[error]    filter with: ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.copy$default$2")
[error]  * method this(java.lang.String,Int)Unit in class org.apache.spark.deploy.DeployMessages#RequestExecutors's type is different in current version, where it is (java.lang.String,scala.collection.immutable.Map)Unit instead of (java.lang.String,Int)Unit
[error]    filter with: ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.this")
[error]  * method apply(java.lang.String,Int)org.apache.spark.deploy.DeployMessages#RequestExecutors in object org.apache.spark.deploy.DeployMessages#RequestExecutors in current version does not have a correspondent with same parameter signature among (java.lang.String,scala.collection.immutable.Map)org.apache.spark.deploy.DeployMessages#RequestExecutors, (java.lang.Object,java.lang.Object)java.lang.Object
[error]    filter with: ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.apply")
```

https://github.com/apache/spark/runs/7221231391?check_suite_focus=true

This PR adds the suggested filters.

### Why are the changes needed?

To make the scheduled build (Scala 2.13) pass in https://github.com/apache/spark/actions/workflows/build_scala213.yml

### Does this PR introduce _any_ user-facing change?

No, dev-only. The alarms are false positive.

### How was this patch tested?

CI should verify this,

Closes apache#37109 from HyukjinKwon/SPARK-39703.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
HyukjinKwon committed Jul 7, 2022
1 parent 427fbee commit 845950b
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion project/MimaExcludes.scala
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,15 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.classification.Classifier.getNumClasses"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.classification.Classifier.getNumClasses$default$2"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.classification.OneVsRest.extractInstances"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.classification.OneVsRestModel.extractInstances")
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.classification.OneVsRestModel.extractInstances"),

// [SPARK-39703][SPARK-39062] Mima complains with Scala 2.13 for the changes in DeployMessages
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.deploy.DeployMessages$LaunchExecutor$"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.requestedTotal"),
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.copy"),
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.copy$default$2"),
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.this"),
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.deploy.DeployMessages#RequestExecutors.apply")
)

// Exclude rules for 3.3.x from 3.2.0
Expand Down

0 comments on commit 845950b

Please sign in to comment.