[SPARK-12925] Improve HiveInspectors.unwrap for StringObjectInspector.…

Earlier fix did not copy the bytes and it is possible for higher level to reuse Text object. This was causing issues. Proposed fix now copies the bytes from Text. This still avoids the expensive encoding/decoding Author: Rajesh Balamohan <[email protected]> Closes apache#11477 from rajeshbalamohan/SPARK-12925.2.
bvkkr2808 · Mar 4, 2016 · 204b02b · 204b02b
1 parent c04dc27
commit 204b02b
Showing 1 changed file with 3 additions and 2 deletions.
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
@@ -320,9 +320,10 @@ private[hive] trait HiveInspectors {
       case hvoi: HiveCharObjectInspector =>
         UTF8String.fromString(hvoi.getPrimitiveJavaObject(data).getValue)
       case x: StringObjectInspector if x.preferWritable() =>
-        // Text is in UTF-8 already. No need to convert again via fromString
+        // Text is in UTF-8 already. No need to convert again via fromString. Copy bytes
         val wObj = x.getPrimitiveWritableObject(data)
-        UTF8String.fromBytes(wObj.getBytes, 0, wObj.getLength)
+        val result = wObj.copyBytes()
+        UTF8String.fromBytes(result, 0, result.length)
       case x: StringObjectInspector =>
         UTF8String.fromString(x.getPrimitiveJavaObject(data))
       case x: IntObjectInspector if x.preferWritable() => x.get(data)