The goal of the packaging solution is to generate native debian packages for Symfony2 applications. This allows leveraging the debian dependency system and package repository solutions. Questions asked during the installation take away the guess work what server specific settings need to be set. Furthermore having all packages installed via the debian system enables the creation of tools that determine what version is installed on what server, which is useful for example to determine the impact of backward compatibility breaks or what server needs to be updated when security fixes are made available.
The packaging system using a similar approach to the Moodle/Mahara/Drupal packaging used at Liip based on the work done at Catalyst. Nothing is committed to the Symfony2 project except metadata, and the package is built from templates inside the packaging "submodule" (normally a git submodule, in this case a vendor). Multiple binary packages are built from a single source package, defined by packagemanifest.yml in the root of the symfony2 directory.
The logic for templating all the files from the packaging "submodule" into the debian/ directory before calling dpkg-buildpackage, is all handed in packaging/maketime.pl. This file parses the packagemanifest.yml, and for each application in there, starts building up the binary package definitions and templates everything into debian/.
The best way to understand it is to read it, but the most important thing to understand, is that this is the script that combines the declarations in packagemanifest.yml, the templates in packaging/debian_templates and writes the output into debian/.
It also calls dpkg-buildpackage to actually build the debian packages, and then clears out the debian/ directory again afterwards.
To setup the packaging scripts you can use git to either clone the sitepackaging repository in to your Symfony2 projects root directory or set it up as a submobule at your own discretion:
git clone git://github.com/mjollnir/sf2debpkg.git packaging
Make sure the following debian packages are installed
sudo apt-get install dpkg-dev
sudo apt-get install debhelper
sudo apt-get install devscripts
sudo apt-get install libconfig-yaml-perl
sudo apt-get install libfile-slurp-perl
To generate skeleton packagemanifest.yml, Makefile and debian/changelog files run the following command: ./packaging/maketime.pl
The packager assumes that a custom directory structure is used for the application kernels. Specifically each application kernel is stored inside a subdirectory of an "app" directory. With this structure its possible for the installer to generate multiple separate packages for each kernel. This requires some minimal changes to the frontcontroller and kernel files. Note this also requires some changes to the createKernel() method in the WebTestCase class, which are implemented in FunctionalTestBundle.
web/main.php (will be installed as web/index.php)
<?php
require_once __DIR__.'/../app/bootstrap.php';
require_once __DIR__.'/../app/main/MainKernel.php';
use Symfony\Component\HttpFoundation\Request;
$kernel = new MainKernel('prod', false);
//$kernel = new MainCache(new MainKernel('prod', false));
$kernel->handle(Request::createFromGlobals())->send();
app/main/MainKernel.php
<?php
use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\DependencyInjection\Loader\LoaderInterface;
class MainKernel extends Kernel
{
public function registerRootDir()
{
return __DIR__;
}
public function registerBundles()
{
$bundles = array(
new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
// enable third-party bundles
);
if ($this->isDebug()) {
$bundles[] = new Symfony\Bundle\WebProfilerBundle\WebProfilerBundle();
}
return $bundles;
}
public function registerContainerConfiguration(LoaderInterface $loader)
{
// use YAML for configuration
// comment to use another configuration format
$loader->load(__DIR__.'/config/config_'.$this->getEnvironment().'.yml');
}
}
This file defines the multiple applications that are built by the packaging procedure. In case it needs stating, you can quote single quotes like so ' something ''quoted'' '. The rough structure is so:
packagemanifest.yml
constants:
database: &database
'php5-pgsql': ~
'postgresql-client': ~
web: &web
'php5': '>=5.3.0'
'libapache2-mod-php5': ~
'php5-curl': ~
project:
projectname: yourproject
maintainername: 'Your Team Name'
maintaineremail: '[email protected]'
apps:
main:
frontend: main.php
description: 'Main website'
dependencies:
<< : [ *database, *web ]
conflicts: ~
recommends: ~
suggests: ~
predepends: ~
cron:
#'some:console:command': "0-5/2 * * * *"
bundles:
- 'vendor/symfony'
- 'vendor/zend/library/Zend/Log'
- 'vendor/swiftmailer'
- 'vendor/doctrine'
- 'vendor/doctrine-migrations'
- 'vendor/doctrine-dbal'
- 'vendor/doctrine-common'
- 'vendor/twig'
- 'src/Acme/YourBundle'
assets: ~
profiles:
- 'web'
- 'db'
dbconfig:
dbtypes: pgsql
create: false
postinst: ~
debianpostinst: 'shell commands to run "as is" (you can use _-_WWWROOT_-_ and it will be replaced)
installfiles:
- 'app/main'
- 'src/autoload.php'
- 'app/boostrap.php.cache'
dbmigration:
description: 'DB Schema and Migrations package'
dependencies:
'php5-cli': ~
bundles:
- 'vendor/doctrine'
- 'vendor/symfony'
- 'vendor/doctrine-migrations'
- 'vendor/doctrine-data-fixtures'
- 'vendor/doctrine-dbal'
- 'vendor/doctrine-common'
- 'src/Acme/YourBundle'
assets: ~
profiles:
- 'db'
dbconfig:
dbtypes: pgsql
create: false
postinst:
- 'doctrine:migrations:migrate -n -q'
#- 'doctrine:fixtures:load --fixtures=src/Acme/YourBundle/Resources/data/fixtures --append=true'
installfiles:
- 'app/dbmigration'
- 'app/autoload.php'
- 'app/bootstrap.php.bin'
static:
description: 'All static website content'
prebuild:
- 'cp app/main/config/dynamic.yml.dist app/main/config/dynamic.yml'
- 'app/main/console assets:install web'
- 'app/main/console asetic:dump web'
- 'rm app/main/config/dynamic.yml'
dependencies:
<<: *web
bundles: ~
assets:
- 'web/bundles/'
- 'web/css/'
- 'web/js/'
profiles:
- 'web'
- 'static'
installfiles:
- 'app/static'
The dynamic.yml should be included and used (rather than config.yml) for all dynamic config options that are dependant on which machine symfony2 is installed upon. For example, database connection details or urls. During the installation a dynamic.yml.dist is parsed if it exists. For each line the in the config the installer will ask for a value to generate the dynamic.yml. The dynamic.doctrine.dbal.default values are built from a template if the 'db' profile is required, and other parameters can be added into it with debconf during package installation using key/value pairs.
app/main/config/dynamic.yml.dist
parameters:
dynamic.bar: 123
dynamic.doctrine.dbal.default:
driver: pdo_pgsql
dbname: XXXXXXXX
user: XXXXXXXX
password: ~
The dynamic.yml can then be imported into config.yml.
app/main/config/config.yml
imports:
- { resource: dynamic.yml }
doctrine.dbal:
connections:
default:
%dynamic.doctrine.dbal.default%
foo.config:
bar: %dynamic.bar%
Note: Right now there is a "hack" for handling writing the proper PDO driver when using the "dbconfig" settings. As a result the dynamic.yml is manipulated after Debian has created the file. As a result Debian will think that the user has done local changes to the config file during the installation. As a result when asked say yes to the question of to "OVERWRITE WITH THE PACKAGE MAINTAINER'S VERSION".
By default a mod_rewrite setting will be installed to point every non existent file in the web root to index.php (note that webappname.php will be installed as index.php). However additional configurations can be done with an apachesettings file.
Hint: the installation paths are defined at make time and are derived from the package name. So it is safe to put a hardcoded path value into the apachesettings file since the installation paths are known (example: /var/www/<package name>/web for the document root).
app/main/config/apachesettings
# directives can be placed here, to be included into the virtualhost
__APACHE_DEFAULT_REWRITE__
The following table lists the files are used in the packaging (in the build directory, "debian/" during make time):
| filename | purpose |
|---|---|
| changelog | global changelog - used to define the debian version |
| compat | debian compatibilitiy version |
| control | source and binary packages declaration including dependencies |
| rules | makefile for building the package - uses dh_ (debhelper) mostly |
| PACKAGENAME.apache.conf.httpsonly.template | apache config for pure https sites |
| PACKAGENAME.apache.conf.http.template | apache config for pure https sites |
| PACKAGENAME.apache.conf.mixed.template | apache config for mixed http/https sites |
| PACKAGENAME.apache.conf.redirect.template | apache config for redirects (eg wwww.SITE -> SITE) |
| PACKAGENAME.apachedir.template | apache config for the document root |
| PACKAGENAME.config | debconf questions to ask during package installation |
| PACKAGENAME.confmodule | helper for sites that use apache - does graceful restarting and ssl checking |
| PACKAGENAME.cron.d | file to be dropped into /etc/cron.d - contains multiple lines - one for each cronjob defined in packagemanifest.yml |
| PACKAGENAME.dynamic.yml.template | used for the questions during install to generate the dynamic.yml |
| PACKAGENAME.dirs | list of directories on the filesystem thought to be "owned" by the package (will be created) |
| PACKAGENAME.install | list of files to install on the filesystem that the package considers to "own" |
| PACKAGENAME.logrotate | file to be dropped into /etc/logrotate.d - web profiles will contain a snippet to rotate the apache logs |
| PACKAGENAME.postinst | main logic during installation happens here, like templating the config files and trying to connect to the database |
| PACKAGENAME.postrm | script to handle logic after the package is removed |
| PACKAGENAME.preinst | script to handle logic before the package is installed (for example to early exit) |
| PACKAGENAME.prerm | script to handle logic before the package is removed |
| PACKAGENAME.templates | language files and datatype declarations for debconf (see PACKAGENAME.config file) |
The following table lists the template files in the packaging "submodule" packaging/debian_templates. These are templated into debian/ during make time.
| filename | purpose |
|---|---|
| apache.conf.httpsonly.template | see above section in debian/ directory |
| apache.conf.http.template | see above section in debian/ directory |
| apache.conf.mixed.template | see above section in debian/ directory |
| apache.conf.redirect.template | see above section in debian/ directory |
| apachedir.template | see above section in debian/ directory |
| compat | see above section in debian/ directory |
| config | see above section in debian/ directory |
| config.db | questions for apps using the "db" profile, included in main 'config' |
| config.web | questions for apps using the "web" profile, included in main 'config' |
| confmodule | see above section in debian/ directory |
| control | source package declaration |
| control.binarytemplate | template file for each binary package defined in the package manifest, appended to control |
| dirs | see above section in debian/ directory |
| dirs.web | additional dirs for apps using the "web" profile |
| install | see above section in debian/ directory |
| logrotate | see above section in debian/ directory |
| postinst | see above section in debian/ directory |
| postinst.constants.db | top of postinst defining variables for "db" profile apps |
| postinst.constants.web | top of postinst defining variables for "web" profile apps |
| postinst.configure.db | main logic during "configure" phase of postinst for "db" apps |
| postinst.configure.web | main logic during "configure" phase of postinst for "web" apps |
| postinst.configure.end.web | final logic during "configure" phase of postinst for "web" apps |
| postrm | see above section in debian/ directory |
| postrm.db | extra postrm for "db" apps - just calls dbconfig-common |
| preinst | see above section in debian/ directory |
| preinst.web | extra postrm for "web" apps - validates hostname |
| prerm | see above section in debian/ directory |
| prerm.db | extra prerm logic for "db" apps |
| prerm.web | prerm logic for "web" apps |
| rules | see above section in debian/ directory |
| templates | see above section in debian/ directory |
First bump the debian version. This can be done with dch -i --no-auto-nmu, which will increment the changelog file and open it in your editor to type a commit message into. Just follow the existing format. Then commit the changelog "git commit debian/changelog; git push", and if necessary, create a tag.
The packages should be built in a clean checkout of master, not in your working directory. Make sure all external dependencies that need to be bundled have been loaded (submodule's, subversion repositories etc.).
Type "make" in the main directory of your Symfony2 project. This will output the following files:
../symfony2-site-foo_0.0.1.dsc
../symfony2-site-foo_0.0.1_i386.changes
../symfony2-site-foo_0.0.1.tar.gz
../symfony2-site-foo-main_0.0.1_all.deb
../symfony2-site-foo-static_0.0.1_all.deb
Notice that there is one deb per directory in app/, as well as some extra files that are shared across all packages.
The version number will be slightly different each time, relative to the value in debian/changelog.
If you're installing a package that's going to use a database on a remote machine, make sure that the debian priority is low:
dpkg-reconfigure debconf
Frontend can be set to anything - dialogue is fine. Priority should be low. This is because of a limitation in dbconfig-common (See: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607171)
Before trying to install a package that wants a database (for example symfony2-site-foo-main), the database server must be able to be connected to. The package will not create the database for you.
Note: Theoretically this works with dbconfig-common, and indeed we're using it for other projects, but it hasn't been tested with Symony2. It may be enough to change the apps.yourapp.dbconfig.create parameter to true in packagemanifest.yml to enable this, but it's not yet tested.
Finally its recommended to setup a debian package repository to distribute the packages. This way on the target servers you just need to setup the custom repository via the following line in its apt sources:
deb http://packages.example.com/debian/ sid main
and simply type the following command to either install or upgrade:
apt-get install symfony2-site-foo-APPNAME
All the values in all dynamic.yml files can be changed by doing:
dpkg-reconfigure symfony2-site-foo-APPNAME
This work is based on the original sitepackaging.git found at http://git.catalyst.net.nz/gw?p=sitepackaging.git;a=summary which was originally written by Pete Bulmer at Catalyst IT. The Symfony2 implementation doesn't have a common ancestor with this, but re-uses some code and is definitely based on the same idea.