dynamic_cast won't work across dylib boundaries on macOS

[ProfFan]

Currently the python wrapper will fail for some dynamic_casts as the type_info for the same type changes between dylibs. Now it is mitigated by compiling GTSAM statically, but still need to resolve sooner or later.

See #217

[ProfFan]

So note for everyone: After some careful thought, I think the best way of handling RTTI-based dynamic dispatch with header-only libraries is to NOT make their objects cross shared library boundaries. For example, an Eigen::Matrix initialized in some library should stay in the same library's code region, and dynamically dispatched within that library before being handed over to another library for processing.

[dellaert]

I don't understand this :-)

[ProfFan]

The idea is that the same type may have different type_info generated by RTTI if they are compiled in different libraries (a core flaw of "header-only"), so when dynamic_cast wants to check if a generic (Base) can be converted to another type (Derived), the check would fail because type_info::hash_code() of the same derived type will be different.

[ProfFan]

So I have a solution here:

quoting Apple documentation:

---

Command Line Options for Visibility

The visibility command line options are useful for setting the visibility defaults. This can be especially helpful when dealing with source code which is third party, or otherwise for some reason not practical to decorate with attributes or pragmas. If you do not specify a command line visibility option then symbols will be marked visible (unless overridden by one of the other three techniques). You can make this choice of default visibility explicit by placing the following on your command line:

 -fvisibility=default

Specifying -fvisibility=hidden on the command line marks all symbols as hidden (unless overridden by pragmas or attributes - export lists can not make a hidden symbol visible).

There are two other visibility-related command options one may find useful:

1. Use -fvisibility-inlines-hidden to declare that there will be no attempt to compare pointers to inlined member functions where the addresses of the two member functions were taken in different linkage units.

   For many applications, -fvisibility-inlines-hidden is a safe and easy way to hide many symbols which in turn improves load time. The dynamic linker (at load time) does much more work for visible symbols than for hidden ones.

   The behavior of this switch is not quite the same as marking the member function as hidden directly, because it does not affect static variables local to the function or cause the compiler to deduce that the function is defined in only one linkage unit. Any local statics will assume whatever visibility setting is in effect independent of this command line switch.

   • -fvisibility-inlines-hidden has no effect when -fvisibility=hidden is also on the command line.

   • -fvisibility-inlines-hidden has no effect on non-member inline functions.

   • Do not use -fvisibility-inlines-hidden if you compare pointers to inlined member functions across shared library boundaries.

2. -fvisibility-ms-compat sets the default visibility to hidden, except for the type_info data associated with types. The impact of this is to emulate the linkage model of Microsoft Visual Studio. Comparing types is important for catching exceptions, dynamic_cast, and comparing type_info's obtained by a typeid expression.

   • -fvisibility=hidden has no effect when -fvisibility-ms-compat is also on the command line.

   • -fvisibility-inlines-hidden has no effect when -fvisibility-ms-compat is also on the command line.

   • Use of -fvisibility-ms-compat can cause two implementation detail classes which were independently developed within different shared libraries to accidently be mistaken for the same type.

   • The -fvisibility-ms-compat flag will hide any static data members of types, giving each linkage unit private copies (unless said data is otherwise marked visible).

---

This will solve the typeid issue on macOS (matching the Linux behavior). I think it is also worthwhile to add this to the docs as there may be future projects dynamically linking to GTSAM on macOS.

[dellaert]

@ProfFan that's a lot of good info, but what is concisely the proposed solution ? :-)

[ProfFan]

@dellaert ProfFan@0b7d140

  if(APPLE)
    target_compile_options(${target} PRIVATE "-fvisibility-ms-compat")
  endif()

[dellaert]

If you can verify it solves our issue on Mac, then, yes, let's adopt and document this solution. Great find!

[dellaert]

Let's close after PR is merged?

[ProfFan]

Copy :)