[question] Integrate conan into development tool chain, e.g. standard make files and IDEs

[langeseb]

Dear conan-developers,

I'm currently using conan 1.1.1. with RHEL and try to evaluate how to integrate conan in our tool chain. Thus, I want to create packages, use packages, CI them, and adapt my tools to work with conan.
The documentation is really good, but I'm missing a whole picture/tutorial how to integrate conan, see also:
#32
To be more specific: I'm using Netbeans IDE which generates makefiles for my c/c++ projects.
What is the suggested way to integrate conan into the following example:
MPI-Helloworld:
I've successfully created a conan package for openmpi and a new C-Application project, which basically uses MPI to say helloworld from multiple processes. Netbeans generates the makefiles for the building process (as well as tests, which I'll ignore here).
Now, I want to use the conan-openmpi package I've build. The examples mostly shows how to integrate it into cmake, only mentioning, that make is also supported. What's the recommended way to use the include-/libdirs in my standard makefile? Shall I use the conan txt-generator and put the paths into my makefile? If so: a manual process would not be feasible for more complex projects. Thus, should I script this process or is it better to use conan to build my mpi-helloworld?
The next step is to create also a package for my mpi-helloworld. Then I'll have to adapt my IDE, i.e. adding the includes and libs into my IDE-properties (for sake of tracing code and navigate also into my openmpi library). At this point, I could of course work with symlinks or is there better way, like replacing the make command with a conan create and using the Autotools with conan to resolve my include/lib-path?
Finally, after the local integration, I want upload the code to a remote git-repo which triggers Jenkins and publish the artefact into artifactory (e.g. Jenkinsfile).
A lot of the pieces are described in the docs, but it is - at least for me - hard to get a full picture. A How-To article would be great!
I understand, that you can't cover all the topics, like specific IDEs, but maybe you can give an idea how conan would be used in such a environment?
Thank you & kind regards!

[memsharded]

Hi @langeseb !

Lets start step by step. You mention that:

The examples mostly shows how to integrate it into cmake, only mentioning, that make is also supported.

I am not sure if you know about the how-to regarding Makefiles, I think it can be very useful for your use case. It also contains links to a github repo with some code: http://docs.conan.io/en/latest/howtos/makefiles.html

So, no you shouldn't need using and parsing the txt generator :)

I think you might also find relevant this section of the reference: http://docs.conan.io/en/latest/reference/build_helpers/autotools.html

Please have a look at those and tell me if they help. Thanks!

[langeseb]

Hi @memsharded!
Thank you for the quick response!
I've already read the how-to article concerning the makefile. As far as I've understood it aimes more on integrating make into conan, whereas I try to figure out how to integrate conan into make.
I already succeeded using a makefile based build system from conan ( \o/ ), see the example given above; I've created a conan package for openmpi. Further I also managed to build a second conan package (hdf5) using the conan-openmpi. So far so good.
Now I want to integrate conan in our development chain.
If conan takes the role of the build system (i.e. calling conan install, instead of make), then my IDE (Netbeans, Eclipse, etc.) might need a plugin to understand conan (read the cached files for IDE features), which does not exist, yet.
So my idea was to integrate conan into my makefile (which is generated and modified by Netbeans). In that case I would aim for the include- and lib-dirs provided by conan install, which I can use to call make.

Maybe I also don't get the idea of conan straight and this is not the way it should be used?

Have a nice weekend!

[memsharded]

Oh, so you are trying to achieve the equivalent to cmake-conan in Makefiles?
If you haven't checked yet, this is the repo: https://github.com/conan-io/cmake-conan

I am no expert in Makefiles, don't know the hackery required to do something equivalent, but we could try to help you if you want to give it a try. I think the best could be to create a github repo with a very simple proof of concept that we could use to sync.

Lets talk about it. Have a nice weekend too!

[langeseb]

Correct - The cmake-conan project is what I'm looking for, only for Makefiles. As far as I've understood there is no recent approach for Make. I will test a little bit, what I can do with the combination of calling conan from Make and integrate this to Netbeans and come back to you, when I'll find something.
The weak spot of this approach is, that Netbeans won't be able to integrate/link the include dirs from the conan cache, since there is no explicit -I [includedir] when calling conan install.
I will create a repo with the openmpi - hello-world example the next days.
Do you use any IDE for coding or do you stick to terminal hacking ;) ?

[memsharded]

The weak spot of this approach is, that Netbeans won't be able to integrate/link the include dirs from the conan cache, since there is no explicit -I [includedir] when calling conan install.

I am not sure what you mean, if you are driving the process from Make, then you will have access to all include-dirs, etc. why would you like to provide some includedirs to conan install? The includedirs are the output of the conan install, not the input, right? What I am missing?

Regarding IDE, I use some combinations:

• Vim or VSCode for lightweight editing, some demos, etc.
• CLion for portable C and C++ dev across platforms
• VS in Windows, when need to, and for some debugging is possible to do it in Win

But otherwise, yes, I like the terminal much, e.g. I always use git on the terminal, I don't use any IDE plugin or addon. I like very much the explicit flow under my control: git clone + conan install + cmake (to configure project) + Open IDE, rather than the opposite.

[langeseb]

I agree - terminal would be great. Unfortunately I'm integrated into a project where we have to use the given toolchain. It's not easy to introduce new tools, when other colleagues prefer the IDE way.

To the problem with the include dirs; Netbeans also just calls Make and don't parse its content. It uses an xml-config file, to create the Makefile structure, where the top level Makefile can be edited (where I can put conan install) whereas the lower level Makefiles will be generated if the config changes. E.g. Makefile includes Makefile-Debug.mk, which holds the config for a specific build, in that case 'Debug'. That means, when I set an include-dir within the IDE (GUI), Netbeans adds the corresponding line to the config.xml and re-generates the Makefile-Debug.mk, which then contains the line -I[myincludedir]. Thus calling make --config=Debug will find the required files. Same is true for linker paths. Only from the config.xml Netbeans knows, where to find the included codes, i.e. /myincludedir/myincludefile.h will be known by Netbeans and can be edited, etc. If I use conan install and the connected environment variables e.g. $CONAN_INCLUDES within the top-level Makefile, Netbeans won't recognise the includes pointing to the conan cache and specific SHA-1 notaded directories. Make would nevertheless work, but I lose the integration into Netbeans and make my colleagues unhappy. Anyway, this is not a conan problem, but an IDE-related topic. I'll see how far I will come with the integration. I think, this is solvable by writing a conan-plugin for Netbeans (and I guess same is true for Eclipse). At least Java build tools like gradle or maven are integrated in those IDEs and they do dependency management.

[memsharded]

Ah! Now I get it. It is not pure Makefile based, but Makefile-generated based.

Can that xml-config file add extra information from another xml file? Maybe the best integration could be writing a conan-generator, that outputs the info in something that Netbeans can consume. It is not very difficult, if you know the output format, etc, it can be developed in about 1 hr.

The docs to check would be: http://docs.conan.io/en/latest/howtos/custom_generators.html#dyn-generators

Please tell me if this could make sense, thanks!

[langeseb]

That could be a solution - I'll ask the Netbeans-pros around here.
Thank you - I'll come back to you, soon!

[langeseb]

Hey @memsharded!
Back again. I've started a github repo, with a small example, where we can start from:
https://github.com/langeseb/conan-make-netbeans-integration
First there is the OpenMPI library to be build. I've uploaded a corresponding conan package built (created with my actual poor conan skills ;) )
This package shall be used by the MPI hello world project, created with Netbeans (8.2).
So far so good. Some background infos:

• Netbeans generates a Makefilestructure. The Makefile in the top directory integrates those within the nbproject dir, where different build configurations are generated (by default: Release and Debug).
• The project settings itselves are controlled by the configurations.xml, which is permanently observed by Netbeans and reacts to changes, e.g. adding a library path. Further, any changes made within the GUI will be added in the configurations.xml accordingly
• The top level Makefile can be used for general settings, e.g. calling conan install, the lower level Makefiles can be edited for specific configs
• The Makefiles are used by Jenkins for CI, thus they must be indenpendent from any individual local tailoring.

Some ideas:

• It's favourable to just involve Netbeans for keeping the dependencies linked into the project > your idea to generate an xml would be great. Maybe it's best whether to edit the configurations.xml directly or copy the content, add paths, copy it back. This aims only on supporting the Netbeans IDE.
• Netbeans calls the Makefiles to build the project. Therefore, conan has to be called by the Makefile itself to provide the depencies i.e. the environment vars, like paths to libs and includes.
• The Makefiles should not be modified with the conan build SHA1 hashes, since they can change and won't be the same for CI/Jenkins
• The most easiest way would be a plugin for Netbeans, which enables the handling of conan instead of make. But that's something more complex, which may be a future topic for IDEs