forked from robotology/yarp
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathyarp_data_dirs.dox
188 lines (116 loc) · 10.7 KB
/
yarp_data_dirs.dox
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
/*
* Copyright (C) 2013 iCub Facility Istituto Italiano di Tecnologia
* CopyPolicy: Released under the terms of the LGPLv2.1 or later, see LGPL.TXT
*
*/
/**
* @page yarp_data_dirs YARP data directories
\author Elena Ceseracciu and Lorenzo Natale
YARP utilities and modules written by users need to find and access data and configuration files at run-time. The former use generic configuration files and manifest files for modules and plugins. The latter often require configuration and data files. YARP has an OS-dependent set of default locations where to look for files, and a set of rules that define the search priorities. The default locations can be overridden or extended by modifying the environment variables that YARP relies on. To simplify finding and accessing these files YARP provides a helper class called ResourceFinder.
A full description of the directories and environment variables used by the ResourceFinder is too detailed for most users. We provide here a short and simplified description of the files layout of typical YARP installations for Windows, Linux and MacOs.
If you are confused by this description you can jump directly to the tutorials \ref yarp_resource_finder_tutorials.
\section datafiles_rationale Rationale
The key points to keep in mind are:
\li The software installation should be able to provide reasonable defaults for configuration files so that applications can run out-of-the-box;
\li Installation directories are generally non-writable, users without root privileges cannot edit installed configuration files unless they first copy them inside their own private directories, the latter must take precedence and hide the others;
\li Therefore it is normal that configuration files can be in multiple places, inside user-specific, private or shared installation directories;
\li External packages can install files so that YARP can find them (see \ref datafiles_3rdparty "here");
\li Type of files are divided in families that are placed in specific sub-directories. Current allowed sub-directories are listed \ref datafiles_datadirs "here".
File customization can happen by copying files or contexts from the shared installation directories to the ones private to the user. The \ref yarp_yarp-config utility simplifies this process.
\section datafiles_datadirs Data directories contents
Files are normally organized in the following directories:
<UL>
<LI> \ref datafiles_contextsrobots "contexts" </LI>
<LI> \ref datafiles_contextsrobots "robots" </LI>
<LI> \ref datafiles_plugins "plugins" </LI>
<LI> \ref datafiles_applications "applications" </LI>
<LI> \ref datafiles_applications "modules" </LI>
<LI> \ref datafiles_applications "resources" </LI>
<LI> \ref datafiles_applications "templates" </LI>
<UL>
<LI> applications </LI>
<LI> modules </LI>
</UL>
</UL>
These directories can be stored in a directory private for the \e user (usually in its home) or \e installed in a shared directory.
\subsection datafiles_lin Linux
The private directories for the user are in <EM> $HOME/.local/share/yarp </EM>
The shared directories for installation are <EM> usr/local/share/yarp </EM> and <EM> /usr/share/yarp </EM>
The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through \e path.d (see \ref datafiles_extendsearch "below").
\subsection datafiles_win Windows
The private directories for the user are in <EM> \%APPDATA\%\\yarp\\ </EM>
The shared directory for installation are in <EM> \%YARP_DIR\%\\share\\yarp\\ </EM>
The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through \e path.d (see \ref datafiles_extendsearch "below").
\subsection datafiles_macos MacOS
The private directory for the user are in <EM> $HOME/Library/Application Support/yarp </EM>
The shared directory for installation are in <EM>/usr/local/share/yarp </EM> and <EM> /usr/share/yarp </EM>
The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through \e path.d (see \ref datafiles_extendsearch "below").
\section datafiles_searchpolicy ResourceFinder Search policy
The resource finder searchers for files in the following order of precedence:
<UL>
<LI> Looking in the current working directory </LI>
<LI> Searches in the \e robot directory </LI>
<UL>
<LI> In the user private directory </LI>
<LI> In the shared, installation directory(ies) </LI>
</UL>
<LI> Searches in the \e context directory </LI>
<UL>
<LI> In the user private directory </LI>
<LI> In the shared, installation directory(ies) </LI>
</UL>
</UL>
When searching for files and directories, the ResourceFinder looks inside these directories in the above order, so that files modified by the user take precedence over installed ones.
\subsection datafiles_contextsrobots Contexts and Robots files
The \e contexts folders contain multiple sub-folders. Each subfolder represents a context, i.e., a collection of configuration files and data that may be used to configure modules at runtime and read/write any type of files. When the resource finder is configured with a specific context-name (either a default, hard-coded one, or provided as command line parameter to the module), contexts/<context-name> is added to the search path in which the initial configuration file and any additional files are sought.
The \e robots folders contain multiple sub-folders as well. Each subfolder has the name of a robot, and contains robot-specific files. The ResourceFinder adds robots/$YARP_ROBOT_NAME (default value for this environment variable: default) to the search path, before the context-specific paths.
\subsubsection datafiles_shadow Shadowing
Search of files proceeds from the user private directories to shared installation directories. The same <context-name> directory can appear in multiple places and likely contain files with the same name. In this case the files that are found first take precedence and \e hide files in the other locations. This \e shadowing or \e masking mechanism is useful when the user needs to customize only a subset of the files of a specific context.
\subsection datafiles_plugins Plugins Manifest files
Plugin manifest files are INI files that describe which class implements the plugin, the library it is included in, and how to find such library. These files are needed when loading plugins (e.g. by the yarpdev executable).
\subsection datafiles_applications Application Deployment files
\e Applications, \e modules and \e resources folders contain XML files that describe programs (modules), the resources available to run the programs (resources), and collections of interconnected modules (applications). The \ref yarpmanager "yarpmanager and gyarpmanager" executables allow to run and monitor applications, while the gyarpbuilder executables allows to design new ones.
Templates are XML files with a .template extension. Typical templates are applications that need to be configured (i.e., assigning the node where each module should be run) before they can be executed.
\section managing_data_files Managing data files: the yarp-config utility
Managing configuration and data files requires nothing special, you can copy, edit and remove them using the shell. However YARP provides an utility that simplifies this, see \ref yarp_yarp-config.
\section datafiles_3rdparty 3rd party packages (for developers)
<EM> If you are not a developer you can skip this section. </EM>
Other packages need to contribute configuration files for their modules, applications, plugins etc. This can be done in two ways.
\li By installing the above files in the YARP directories (YARP exports a set of CMake variables that can be used to this aim);
\li By extending the ResourceFinder search path.
YARP provides cmake variables and macros to simplify both the above tasks.
If you are not interested in the details you can skip the following sections and go directly to \ref yarp_external_build.
\subsection datafiles_installedpkgs Installed packages
Yarp exports the following CMake variables:
\li YARP_INSTALL_PREFIX
\li YARP_CONFIG_INSTALL_DIR
\li YARP_PLUGIN_MANIFESTS_INSTALL_DIR
\li YARP_MODULES_INSTALL_DIR
\li YARP_APPLICATIONS_INSTALL_DIR
\li YARP_TEMPLATES_INSTALL_DIR
\li YARP_APPLICATIONS_TEMPLATES_INSTALL_DIR
\li YARP_MODULES_TEMPLATES_INSTALL_DIR
\li YARP_CONTEXTS_INSTALL_DIR
\li YARP_ROBOTS_INSTALL_DIR
Using the correct CMake variables when installing files will ensure that those files will be in the correct place on every platform, provided CMake is instructed to use the same CMAKE_INSTALL_PREFIX (in Linux this is the case by default).
Also non-root installation is supported, but both YARP and 3rd party software will require to be configured with the same CMAKE_INSTALL_PREFIX, and the environment variables (YARP_DATA_DIRS) will require to be tweaked accordingly (see next section).
\subsection datafiles_extendsearch Extending the search path
For packages that are not installed, the search path followed by the ResourceFinder can be extended in two ways:
\li Through the YARP_DATA_DIRS variable: this is a list of locations, each is used by the ResourceFinder when looking for shared installation directories
<EM> Notice: the following should be considered experimental and could be removed in the future if not required) </EM>
\li By extending the directory \e path.d
This solution should be adopted for packages that are installed.
\e path.d is a special folder used by YARP to provide an easy way to install 3rd party applications in separate folders. This folder contains a list of text files that contains extra search paths. The ResourceFinder searches for the directory \e path.d in the same locations as "Installed" configuration files before checking the same directory for other files. Each file inside \e path.d indicates an additional path to a directory; the latter is appended to the list of directories used by the ResourceFinder. This mechanisms works similarly to YARP_DATA_DIRS; however it is handy because it allows 3rd party applications to extend the search path without affecting the user environment.
To do so a package should install a .ini file in $YARP_PREFIX/yarp/config/path.d .
This can be done using the \e yarp_configure_external_installation macro (see \ref yarp_external_build)
Example: the iCub package in Linux.
The iCub package contributes a set of application, modules, contexts and robot files to YARP. It does so by installing them in /usr/local/share/iCub to keep them separate from the ones in YARP (/usr/local/share/yarp). To make these files visible to the ResourceFinder \e make \e install adds the file \e iCub.ini in:
/usr/local/share/yarp/config/path.d
The content of the file iCub.ini tells YARP where to look for additional resource files:
\verbatim
###### This file is automatically generated by CMake.
[search iCub]
path "/usr/local/share/iCub"
\endverbatim
*
*/