lib/hexdump.c: truncate output in case of overflow

There is a classical off-by-one error in case when we try to place, for
example, 1+1 bytes as hex in the buffer of size 6.  The expected result is
to get an output truncated, but in the reality we get 6 bytes filed
followed by terminating NUL.

Change the logic how we fill the output in case of byte dumping into
limited space.  This will follow the snprintf() behaviour by truncating
output even on half bytes.

Fixes: 114fc1a (hexdump: make it return number of bytes placed in buffer)
Signed-off-by: Andy Shevchenko <[email protected]>
Reported-by: Aaro Koskinen <[email protected]>
Tested-by: Aaro Koskinen <[email protected]>
Cc: Al Viro <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

lib/hexdump.c

@@ -169,11 +169,15 @@ int hex_dump_to_buffer(const void *buf, size_t len, int rowsize, int groupsize,
 		}
 	} else {
 		for (j = 0; j < len; j++) {
-			if (linebuflen < lx + 3)
+			if (linebuflen < lx + 2)
 				goto overflow2;
 			ch = ptr[j];
 			linebuf[lx++] = hex_asc_hi(ch);
+			if (linebuflen < lx + 2)
+				goto overflow2;
 			linebuf[lx++] = hex_asc_lo(ch);
+			if (linebuflen < lx + 2)
+				goto overflow2;
 			linebuf[lx++] = ' ';
 		}
 		if (j)