libelf: Return error if elf_compress_gnu is used on SHF_COMPRESSED se…

…ction.

Compressing a section that is already compressed is fine, but useless.
But it isn't possible to gnu compress (or decompress) a SHF_COMPRESSED
section since there is no state kept that would tell if the section was
first GNU compressed or first gabi compressed. Calling elf_compress_gnu
on a section and then calling elf_compress on it to decompress it twice
could cause a crash (the other way around is fine). Just disallow it.

https://sourceware.org/bugzilla/show_bug.cgi?id=23528

Signed-off-by: Mark Wielaard <[email protected]>

libelf/ChangeLog

@@ -1,3 +1,10 @@
+2018-08-18  Mark Wielaard  <[email protected]>
+
+	* libelf.h (elf_compress_gnu): Add documentation about
+	interaction between SHF_COMPRESED and elf_compress_gnu.
+	* elf_compress_gnu.c (elf_compress_gnu): Return error if section
+	sh_flags has SHF_COMPRESSED set.
+
 2018-07-27  Mark Wielaard  <[email protected]>
 
 	* libelf.h (elf_getshdrstrndx): Fix documentation.

libelf/elf_compress_gnu.c

@@ -80,7 +80,9 @@ elf_compress_gnu (Elf_Scn *scn, int inflate, unsigned int flags)
       sh_addralign = shdr->sh_addralign;
     }
 
-  if ((sh_flags & SHF_ALLOC) != 0)
+  /* Allocated sections, or sections that are already are compressed
+     cannot (also) be GNU compressed.  */
+  if ((sh_flags & SHF_ALLOC) != 0 || (sh_flags & SHF_COMPRESSED))
     {
       __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
       return -1;

libelf/libelf.h

@@ -366,6 +366,11 @@ extern Elf64_Chdr *elf64_getchdr (Elf_Scn *__scn);
    It is an error to request compression for a section that already
    has SHF_COMPRESSED set, or (for elf_compress) to request
    decompression for an section that doesn't have SHF_COMPRESSED set.
+   If a section has SHF_COMPRESSED set then calling elf_compress_gnu
+   will result in an error.  The section has to be decompressed first
+   using elf_compress.  Calling elf_compress on a section compressed
+   with elf_compress_gnu is fine, but probably useless.
+
    It is always an error to call these functions on SHT_NOBITS
    sections or if the section has the SHF_ALLOC flag set.
    elf_compress_gnu will not check whether the section name starts