Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-28702][SQL] Display useful error message (instead of NPE) for …
…invalid Dataset operations ### What changes were proposed in this pull request? Added proper message instead of NPE for invalid Dataset operations (e.g. calling actions inside of transformations) similar to SPARK-5063 for RDD ### Why are the changes needed? To report the user about the exact issue instead of NPE ### Does this PR introduce any user-facing change? No ### How was this patch tested? Manually tested ```scala test code snap "import spark.implicits._ val ds1 = spark.sparkContext.parallelize(1 to 100, 100).toDS() val ds2 = spark.sparkContext.parallelize(1 to 100, 100).toDS() ds1.map(x => { // scalastyle:off println(ds2.count + x) x }).collect()" ``` Closes apache#25503 from shivusondur/jira28702. Authored-by: shivusondur <[email protected]> Signed-off-by: Josh Rosen <[email protected]>
- Loading branch information