Rework interface for bitset-using features to use a notion of LTO vis…

…ibility.

Bitsets, and the compiler features they rely on (vtable opt, CFI),
only have visibility within the LTO'd part of the linkage unit. Therefore,
only enable these features for classes with hidden LTO visibility. This
notion is based on object file visibility or (on Windows)
dllimport/dllexport attributes.

We provide the [[clang::lto_visibility_public]] attribute to override the
compiler's LTO visibility inference in cases where the class is defined
in the non-LTO'd part of the linkage unit, or where the ABI supports
calling classes derived from abstract base classes with hidden visibility
in other linkage units (e.g. COM on Windows).

If the cross-DSO CFI mode is enabled, bitset checks are emitted even for
classes with public LTO visibility, as that mode uses a separate mechanism
to cause bitsets to be exported.

This mechanism replaces the whole-program-vtables blacklist, so remove the
-fwhole-program-vtables-blacklist flag.

Because __declspec(uuid()) now implies [[clang::lto_visibility_public]], the
support for the special attr:uuid blacklist entry is removed.

Differential Revision: http://reviews.llvm.org/D18635

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@267784 91177308-0d34-0410-b5e6-96231b3b80d8

docs/ControlFlowIntegrity.rst

@@ -25,13 +25,25 @@ As currently implemented, all schemes rely on link-time optimization (LTO);
 so it is required to specify ``-flto``, and the linker used must support LTO,
 for example via the `gold plugin`_.
 
-To allow the checks to be implemented efficiently, the program must be
-structured such that certain object files are compiled with CFI
+To allow the checks to be implemented efficiently, the program must
+be structured such that certain object files are compiled with CFI
 enabled, and are statically linked into the program. This may preclude
-the use of shared libraries in some cases. Experimental support for
-:ref:`cross-DSO control flow integrity <cfi-cross-dso>` exists that
-does not have these requirements. This cross-DSO support has unstable
-ABI at this time.
+the use of shared libraries in some cases.
+
+The compiler will only produce CFI checks for a class if it can infer hidden
+LTO visibility for that class. LTO visibility is a property of a class that
+is inferred from flags and attributes. For more details, see the documentation
+for :doc:`LTO visibility <LTOVisibility>`.
+
+The ``-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}`` flags
+require that a ``-fvisibility=`` flag also be specified. This is because the
+default visibility setting is ``-fvisibility=default``, which would disable
+CFI checks for classes without visibility attributes. Most users will want
+to specify ``-fvisibility=hidden``, which enables CFI checks for such classes.
+
+Experimental support for :ref:`cross-DSO control flow integrity
+<cfi-cross-dso>` exists that does not require classes to have hidden LTO
+visibility. This cross-DSO support has unstable ABI at this time.
 
 .. _gold plugin: http://llvm.org/docs/GoldPlugin.html
 
@@ -233,11 +245,6 @@ A :doc:`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
 source files, functions and types using the ``src``, ``fun`` and ``type``
 entity types.
 
-In addition, if a type has a ``uuid`` attribute and the blacklist contains
-the type entry ``attr:uuid``, CFI checks are suppressed for that type. This
-allows all COM types to be easily blacklisted, which is useful as COM types
-are typically defined outside of the linked program.
-
 .. code-block:: bash
 
     # Suppress checking for code in a file.
@@ -247,8 +254,6 @@ are typically defined outside of the linked program.
     fun:*MyFooBar*
     # Ignore all types in the standard library.
     type:std::*
-    # Ignore all types with a uuid attribute.
-    type:attr:uuid
 
 .. _cfi-cross-dso:
 
@@ -260,6 +265,11 @@ flow integrity mode, which allows all CFI schemes listed above to
 apply across DSO boundaries. As in the regular CFI, each DSO must be
 built with ``-flto``.
 
+Normally, CFI checks will only be performed for classes that have hidden LTO
+visibility. With this flag enabled, the compiler will emit cross-DSO CFI
+checks for all classes, except for those which appear in the CFI blacklist
+or which use a ``no_sanitize`` attribute.
+
 Design
 ======
 

docs/LTOVisibility.rst

@@ -0,0 +1,111 @@
+==============
+LTO Visibility
+==============
+
+*LTO visibility* is a property of an entity that specifies whether it can be
+referenced from outside the current LTO unit. A *linkage unit* is a set of
+translation units linked together into an executable or DSO, and a linkage
+unit's *LTO unit* is the subset of the linkage unit that is linked together
+using link-time optimization; in the case where LTO is not being used, the
+linkage unit's LTO unit is empty. Each linkage unit has only a single LTO unit.
+
+The LTO visibility of a class is used by the compiler to determine which
+classes the virtual function call optimization and control flow integrity
+features apply to. These features use whole-program information, so they
+require the entire class hierarchy to be visible in order to work correctly.
+
+If any translation unit in the program uses either of the virtual function
+call optimization or control flow integrity features, it is effectively an
+ODR violation to define a class with hidden LTO visibility in multiple linkage
+units. A class with public LTO visibility may be defined in multiple linkage
+units, but the tradeoff is that the virtual function call optimization and
+control flow integrity features can only be applied to classes with hidden LTO
+visibility. A class's LTO visibility is treated as an ODR-relevant property
+of its definition, so it must be consistent between translation units.
+
+In translation units built with LTO, LTO visibility is based on symbol
+visibility or, on the Windows platform, the dllimport and dllexport
+attributes. When targeting non-Windows platforms, classes with a visibility
+other than hidden visibility receive public LTO visibility. When targeting
+Windows, classes with dllimport or dllexport attributes receive public LTO
+visibility. All other classes receive hidden LTO visibility. Classes with
+internal linkage (e.g. classes declared in unnamed namespaces) also receive
+hidden LTO visibility.
+
+A class defined in a translation unit built without LTO receives public
+LTO visibility regardless of its object file visibility, linkage or other
+attributes.
+
+This mechanism will produce the correct result in most cases, but there are
+two cases where it may wrongly infer hidden LTO visibility.
+
+1. As a corollary of the above rules, if a linkage unit is produced from a
+   combination of LTO object files and non-LTO object files, any hidden
+   visibility class defined in both a translation unit built with LTO and
+   a translation unit built without LTO must be defined with public LTO
+   visibility in order to avoid an ODR violation.
+
+2. Some ABIs provide the ability to define an abstract base class without
+   visibility attributes in multiple linkage units and have virtual calls
+   to derived classes in other linkage units work correctly. One example of
+   this is COM on Windows platforms. If the ABI allows this, any base class
+   used in this way must be defined with public LTO visibility.
+
+Classes that fall into either of these categories can be marked up with the
+``[[clang::lto_visibility_public]]`` attribute. To specifically handle the
+COM case, classes with the ``__declspec(uuid())`` attribute receive public
+LTO visibility. On Windows platforms, clang-cl's ``/MT`` and ``/MTd``
+flags statically link the program against a prebuilt standard library;
+these flags imply public LTO visibility for every class declared in the
+``std`` and ``stdext`` namespaces.
+
+Example
+=======
+
+The following example shows how LTO visibility works in practice in several
+cases involving two linkage units, ``main`` and ``dso.so``.
+
+.. code-block:: none
+
+    +-----------------------------------------------------------+  +----------------------------------------------------+
+    | main (clang++ -fvisibility=hidden):                       |  | dso.so (clang++ -fvisibility=hidden):              |
+    |                                                           |  |                                                    |
+    |  +-----------------------------------------------------+  |  |  struct __attribute__((visibility("default"))) C { |
+    |  | LTO unit (clang++ -fvisibility=hidden -flto):       |  |  |    virtual void f();                               |
+    |  |                                                     |  |  |  }                                                 |
+    |  |  struct A { ... };                                  |  |  |  void C::f() {}                                    |
+    |  |  struct [[clang::lto_visibility_public]] B { ... }; |  |  |  struct D {                                        |
+    |  |  struct __attribute__((visibility("default"))) C {  |  |  |    virtual void g() = 0;                           |
+    |  |    virtual void f();                                |  |  |  };                                                |
+    |  |  };                                                 |  |  |  struct E : D {                                    |
+    |  |  struct [[clang::lto_visibility_public]] D {        |  |  |    virtual void g() { ... }                        |
+    |  |    virtual void g() = 0;                            |  |  |  };                                                |
+    |  |  };                                                 |  |  |  __attribute__(visibility("default"))) D *mkE() {  |
+    |  |                                                     |  |  |    return new E;                                   |
+    |  +-----------------------------------------------------+  |  |  }                                                 |
+    |                                                           |  |                                                    |
+    |  struct B { ... };                                        |  +----------------------------------------------------+
+    |                                                           |
+    +-----------------------------------------------------------+
+
+We will now describe the LTO visibility of each of the classes defined in
+these linkage units.
+
+Class ``A`` is not defined outside of ``main``'s LTO unit, so it can have
+hidden LTO visibility. This is inferred from the object file visibility
+specified on the command line.
+
+Class ``B`` is defined in ``main``, both inside and outside its LTO unit. The
+definition outside the LTO unit has public LTO visibility, so the definition
+inside the LTO unit must also have public LTO visibility in order to avoid
+an ODR violation.
+
+Class ``C`` is defined in both ``main`` and ``dso.so`` and therefore must
+have public LTO visibility. This is correctly inferred from the ``visibility``
+attribute.
+
+Class ``D`` is an abstract base class with a derived class ``E`` defined
+in ``dso.so``.  This is an example of the COM scenario; the definition of
+``D`` in ``main``'s LTO unit must have public LTO visibility in order to be
+compatible with the definition of ``D`` in ``dso.so``, which is observable
+by calling the function ``mkE``.

docs/UsersManual.rst

@@ -1056,17 +1056,8 @@ are listed below.
 .. option:: -fwhole-program-vtables
 
    Enable whole-program vtable optimizations, such as single-implementation
-   devirtualization and virtual constant propagation. Requires ``-flto``.
-
-   By default, the compiler will assume that all type hierarchies are
-   closed except those in the ``std`` namespace, the ``stdext`` namespace
-   and classes with the ``__declspec(uuid())`` attribute.
-
-.. option:: -fwhole-program-vtables-blacklist=path
-
-   Allows the user to specify the path to a list of additional classes to
-   blacklist from whole-program vtable optimizations. This list is in the
-   :ref:`CFI blacklist <cfi-blacklist>` format.
+   devirtualization and virtual constant propagation, for classes with
+   :doc:`hidden LTO visibility <LTOVisibility>`. Requires ``-flto``.
 
 .. option:: -fno-assume-sane-operator-new
 

docs/index.rst

@@ -31,6 +31,7 @@ Using Clang as a Compiler
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
+   LTOVisibility
    SafeStack
    Modules
    MSVCCompatibility

include/clang/Basic/Attr.td

@@ -1611,6 +1611,12 @@ def WeakRef : InheritableAttr {
   let Documentation = [Undocumented];
 }
 
+def LTOVisibilityPublic : InheritableAttr {
+  let Spellings = [CXX11<"clang", "lto_visibility_public">];
+  let Subjects = SubjectList<[Record]>;
+  let Documentation = [LTOVisibilityDocs];
+}
+
 def AnyX86Interrupt : InheritableAttr, TargetSpecificAttr<TargetAnyX86> {
   // NOTE: If you add any additional spellings, ARMInterrupt's,
   // MSP430Interrupt's and MipsInterrupt's spellings must match.

include/clang/Basic/AttrDocs.td

@@ -2380,3 +2380,10 @@ The ``ifunc`` attribute may only be used on a function declaration.  A function
 Not all targets support this attribute.  ELF targets support this attribute when using binutils v2.20.1 or higher and glibc v2.11.1 or higher.  Non-ELF targets currently do not support this attribute.
   }];
 }
+
+def LTOVisibilityDocs : Documentation {
+  let Category = DocCatType;
+  let Content = [{
+See :doc:`LTOVisibility`.
+  }];
+}

include/clang/Driver/CC1Options.td

@@ -282,6 +282,9 @@ def fprofile_instrument_path_EQ : Joined<["-"], "fprofile-instrument-path=">,
 def fprofile_instrument_use_path_EQ :
     Joined<["-"], "fprofile-instrument-use-path=">,
     HelpText<"Specify the profile path in PGO use compilation">;
+def flto_visibility_public_std:
+    Flag<["-"], "flto-visibility-public-std">,
+    HelpText<"Use public LTO visibility for classes in std and stdext namespaces">;
 
 //===----------------------------------------------------------------------===//
 // Dependency Output Options

include/clang/Driver/Options.td

@@ -1152,9 +1152,6 @@ def fwhole_program_vtables : Flag<["-"], "fwhole-program-vtables">, Group<f_Grou
   Flags<[CC1Option]>,
   HelpText<"Enables whole-program vtable optimization. Requires -flto">;
 def fno_whole_program_vtables : Flag<["-"], "fno-whole-program-vtables">, Group<f_Group>;
-def fwhole_program_vtables_blacklist_EQ : Joined<["-"], "fwhole-program-vtables-blacklist=">,
-  Group<f_Group>, Flags<[CC1Option]>,
-  HelpText<"Path to a blacklist file for whole-program vtable optimization">;
 def fwrapv : Flag<["-"], "fwrapv">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Treat signed integer overflow as two's complement">;
 def fwritable_strings : Flag<["-"], "fwritable-strings">, Group<f_Group>, Flags<[CC1Option]>,

include/clang/Frontend/CodeGenOptions.def

@@ -187,6 +187,10 @@ CODEGENOPT(EmitLLVMUseLists, 1, 0) ///< Control whether to serialize use-lists.
 CODEGENOPT(WholeProgramVTables, 1, 0) ///< Whether to apply whole-program
                                       ///  vtable optimization.
 
+/// Whether to use public LTO visibility for entities in std and stdext
+/// namespaces. This is enabled by clang-cl's /MT and /MTd flags.
+CODEGENOPT(LTOVisibilityPublicStd, 1, 0)
+
 /// The user specified number of registers to be used for integral arguments,
 /// or 0 if unspecified.
 VALUE_CODEGENOPT(NumRegisterParameters, 32, 0)

include/clang/Frontend/CodeGenOptions.h

@@ -199,9 +199,6 @@ class CodeGenOptions : public CodeGenOptionsBase {
   /// \brief A list of all -fno-builtin-* function names (e.g., memset).
   std::vector<std::string> NoBuiltinFuncs;
 
-  /// List of blacklist files for the whole-program vtable optimization feature.
-  std::vector<std::string> WholeProgramVTablesBlacklistFiles;
-
 public:
   // Define accessors/mutators for code generation options of enumeration type.
 #define CODEGENOPT(Name, Bits, Default)

lib/CodeGen/CGClass.cpp

@@ -2489,7 +2489,7 @@ void CodeGenFunction::EmitBitSetCodeForVCall(const CXXRecordDecl *RD,
                                              llvm::Value *VTable,
                                              SourceLocation Loc) {
   if (CGM.getCodeGenOpts().WholeProgramVTables &&
-      !CGM.IsBitSetBlacklistedRecord(RD)) {
+      CGM.HasHiddenLTOVisibility(RD)) {
     llvm::Metadata *MD =
         CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
     llvm::Value *BitSetName =
@@ -2565,7 +2565,12 @@ void CodeGenFunction::EmitVTablePtrCheck(const CXXRecordDecl *RD,
                                          llvm::Value *VTable,
                                          CFITypeCheckKind TCK,
                                          SourceLocation Loc) {
-  if (CGM.IsBitSetBlacklistedRecord(RD))
+  if (!CGM.getCodeGenOpts().SanitizeCfiCrossDso &&
+      !CGM.HasHiddenLTOVisibility(RD))
+    return;
+
+  std::string TypeName = RD->getQualifiedNameAsString();
+  if (getContext().getSanitizerBlacklist().isBlacklistedType(TypeName))
     return;
 
   SanitizerScope SanScope(this);

lib/CodeGen/CGVTables.cpp

@@ -902,34 +902,43 @@ void CodeGenModule::EmitDeferredVTables() {
   DeferredVTables.clear();
 }
 
-bool CodeGenModule::NeedVTableBitSets() {
-  return getCodeGenOpts().WholeProgramVTables ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIVCall) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFINVCall) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIDerivedCast) ||
-         getLangOpts().Sanitize.has(SanitizerKind::CFIUnrelatedCast);
-}
+bool CodeGenModule::HasHiddenLTOVisibility(const CXXRecordDecl *RD) {
+  LinkageInfo LV = RD->getLinkageAndVisibility();
+  if (!isExternallyVisible(LV.getLinkage()))
+    return true;
 
-bool CodeGenModule::IsBitSetBlacklistedRecord(const CXXRecordDecl *RD) {
-  std::string TypeName = RD->getQualifiedNameAsString();
-  auto isInBlacklist = [&](const SanitizerBlacklist &BL) {
-    if (RD->hasAttr<UuidAttr>() && BL.isBlacklistedType("attr:uuid"))
-      return true;
+  if (RD->hasAttr<LTOVisibilityPublicAttr>() || RD->hasAttr<UuidAttr>())
+    return false;
 
-    return BL.isBlacklistedType(TypeName);
-  };
+  if (getTriple().isOSBinFormatCOFF()) {
+    if (RD->hasAttr<DLLExportAttr>() || RD->hasAttr<DLLImportAttr>())
+      return false;
+  } else {
+    if (LV.getVisibility() != HiddenVisibility)
+      return false;
+  }
 
-  return isInBlacklist(WholeProgramVTablesBlacklist) ||
-         ((LangOpts.Sanitize.has(SanitizerKind::CFIVCall) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFINVCall) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFIDerivedCast) ||
-           LangOpts.Sanitize.has(SanitizerKind::CFIUnrelatedCast)) &&
-          isInBlacklist(getContext().getSanitizerBlacklist()));
+  if (getCodeGenOpts().LTOVisibilityPublicStd) {
+    const DeclContext *DC = RD;
+    while (1) {
+      auto *D = cast<Decl>(DC);
+      DC = DC->getParent();
+      if (isa<TranslationUnitDecl>(DC->getRedeclContext())) {
+        if (auto *ND = dyn_cast<NamespaceDecl>(D))
+          if (const IdentifierInfo *II = ND->getIdentifier())
+            if (II->isStr("std") || II->isStr("stdext"))
+              return false;
+        break;
+      }
+    }
+  }
+
+  return true;
 }
 
 void CodeGenModule::EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
                                             const VTableLayout &VTLayout) {
-  if (!NeedVTableBitSets())
+  if (!getCodeGenOpts().PrepareForLTO)
     return;
 
   CharUnits PointerWidth =
@@ -938,12 +947,8 @@ void CodeGenModule::EmitVTableBitSetEntries(llvm::GlobalVariable *VTable,
   typedef std::pair<const CXXRecordDecl *, unsigned> BSEntry;
   std::vector<BSEntry> BitsetEntries;
   // Create a bit set entry for each address point.
-  for (auto &&AP : VTLayout.getAddressPoints()) {
-    if (IsBitSetBlacklistedRecord(AP.first.getBase()))
-      continue;
-
+  for (auto &&AP : VTLayout.getAddressPoints())
     BitsetEntries.push_back(std::make_pair(AP.first.getBase(), AP.second));
-  }
 
   // Sort the bit set entries for determinism.
   std::sort(BitsetEntries.begin(), BitsetEntries.end(),

lib/CodeGen/CodeGenModule.cpp

@@ -88,9 +88,7 @@ CodeGenModule::CodeGenModule(ASTContext &C, const HeaderSearchOptions &HSO,
       PreprocessorOpts(PPO), CodeGenOpts(CGO), TheModule(M), Diags(diags),
       Target(C.getTargetInfo()), ABI(createCXXABI(*this)),
       VMContext(M.getContext()), Types(*this), VTables(*this),
-      SanitizerMD(new SanitizerMetadata(*this)),
-      WholeProgramVTablesBlacklist(CGO.WholeProgramVTablesBlacklistFiles,
-                                   C.getSourceManager()) {
+      SanitizerMD(new SanitizerMetadata(*this)) {
 
   // Initialize the type cache.
   llvm::LLVMContext &LLVMContext = M.getContext();