Skip to content

Commit

Permalink
[SPARK-28357][CORE][TEST] Fix Flaky Test - FileAppenderSuite.rollingf…
Browse files Browse the repository at this point in the history
…ile appender - size-based rolling compressed

## What changes were proposed in this pull request?

`SizeBasedRollingPolicy.shouldRollover` returns false when the size is equal to `rolloverSizeBytes`.
```scala
  /** Should rollover if the next set of bytes is going to exceed the size limit */
  def shouldRollover(bytesToBeWritten: Long): Boolean = {
    logDebug(s"$bytesToBeWritten + $bytesWrittenSinceRollover > $rolloverSizeBytes")
    bytesToBeWritten + bytesWrittenSinceRollover > rolloverSizeBytes
  }
```
- https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107553/testReport/org.apache.spark.util/FileAppenderSuite/rolling_file_appender___size_based_rolling__compressed_/
```
org.scalatest.exceptions.TestFailedException: 1000 was not less than 1000
```

## How was this patch tested?

Pass the Jenkins with the updated test.

Closes apache#25125 from dongjoon-hyun/SPARK-28357.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
  • Loading branch information
dongjoon-hyun authored and HyukjinKwon committed Jul 12, 2019
1 parent fe22faa commit 1c29212
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ class FileAppenderSuite extends SparkFunSuite with BeforeAndAfter with Logging {
val files = testRolling(appender, testOutputStream, textToAppend, 0, isCompressed = true)
files.foreach { file =>
logInfo(file.toString + ": " + file.length + " bytes")
assert(file.length < rolloverSize)
assert(file.length <= rolloverSize)
}
}

Expand Down

0 comments on commit 1c29212

Please sign in to comment.