Leverages ISLE to provide a local development environment for the IDC stack, with particular focus on development/testing of the Drupal site.
- Our Drupal site is in
codebase.- Use
composerto add, remove, or update dependencies incodebase/composer.jsonandcodebase/composer.lockwhen developing - Dependencies are not vendored, so you need to do a composer install. This is included in
make bootstrap
- Use
- IDC development-specific environment variables are in
.envanddocker-compose.env.yml - An idc-specific Makefile
idc.Makefiledefines additional make targets available for `make
To start the IDC development environment, run
make up
This will build a docker-compose file, run composer install to locally install all dependencies for our site (which will
take a few minutes when done the first time, but will be much quicker subsequent times), and start the stack. The stack will
start from a known snapshot state, which currently is an entirely empty (but initialized) Drupal.
To reset to a known Drupal state, run
make reset
This will remove all content from volumes that you may have added, remove all php files downloaded by composer,
launch using the snapshot as its initial state, then import any config from config/sync.
composer install will run automatically as part of the startup process.
A slightly quicker way to reset to a known state is:
docker-compose down -v
make up
This removes all content from volumes, restores from the last snapshot, pulls in config from config/sync, but does not remove composer-managed PHP files. composer install will still run upon startup, but it needs to do much less.
To dump the site's configuration so that it can be committed to git, do
make config-export
There are several Make targets in the Makefile, and its idc-specific companion idc.Makefile (which are included by default,
so no need to do anything special other than make to invoke them). A few useful targets are as follows:
- make reset Burn everything down and create a fresh installation from the snapshot image. composer-installed modules and dependencies do not survive; they will be deleted, then installed from scratch via
composer installwhen the drupal container starts. Once up, configuration will fromconfig/syncwill be imported. - make composer-install Use the Drupal container to run a
composer install. This avoids having to install composer on your local system. Note: the Drupal image startup process implicitly runscomposer installalready, so this make target is used for on-demand composer installs. - make cache-rebuild Uses Drush inside the Drupal container to rebuild Drupal's cache.
- make config-export Exports all current active Drupal config to the
codebase/config/syncdirectory, so that it can be committed to git. - make up Brings up the development environment, including running
composer install. If there is no pre-existing state (e.g. after adocker compose down -v), this will load initial state from the current snapshot, and pull in config fromconfig/sync. Otherwise, if there is pre-existing state (i.e. after adocker stop), docker will start,composer installwill run, but configuration fromconfig/syncwill not be imported. - make test Runs all tests. Optionally, takes a
test=parameter for running a particular test suite. NOTE: Running all tests without atest=argument will result in all local state being wiped out - make dev-up Launches the stack with a Drupal image configured with XDebug for IDE-based debugging. Updates the environment requiring
make dev-downto be invoked at the conclusion of a development session. - make dev-down Stops the Drupal development image, and resets the environment to using production.
A few specialized targets are:
- make bootstrap Burn everything down and create a fresh installation from scratch, deleting any pre-existing data, and starting from a completely empty state. Only the list of modules in
composer.json(and dependencies incomposer.lock) survives the process. - make minio-bucket Create a new bucket in minio based on the environment variables present in
.env - make snapshot Create a snapshot of the current Drupal state (db, content files, etc), so that you can reset to this state at will, or push it so that others can.
- make snapshot-push Push the current snapshot image to the container registry
- make static-docker-compose.yml Make a docker-compose.yml based off non-development "static" environment. Notably:
- A
drupal-staticimage is built or pulled, which hascodebasebaked in, and is used in place of the normaldrupalimage codebaseis no longer bind mounted
- A
- make static-drupal-image builds (or pulls, if published) a "static" drupal image suitable for deployment in the cloud. This image:
- Has the contents of
codebasebaked into it, as well as all dependencies viacomposer install - Will load its config from
config/syncupon startup - Is named
drupal-staticand is tagged based ongit describe --tags.
- Has the contents of
- make minio_bucket Creates a new S3 bucket in minio for IDC. The empty bucket should be part of the snapshot, so this is really only useful for bootstrapping new snapshots from scratch.
To run all tests, with the environment (configuration, modules, content) reset between each test, run make test. This
is the make target used to run tests during CI. To run an individual test, run make test test=<name of test>, where <name of test> is the name of a test script in the tests directory, e.g. make test test=01-end-to-end.sh.
When a single test is specified with the test= argument, the environment is not reset for the execution of the test.
That is, the existing configuration, content, and modules present in the current codebase are used. This is the recommended (and really only supported way) of running your tests while iterating.
Some additional notes about the test execution environment:
- Tests are run in a subshell given the environment from
.envand functions fromtests/.includes.sh.- To share common test-related variables or functions between tests, add them to
tests/.includes.sh. - Variables added to
.envare automatically available to the test script.
- To share common test-related variables or functions between tests, add them to
- The
test=argument tomake testaccepts the name of the test suite, minus the.shending;make test test=01-end-to-end.shandmake test test=01-end-to-end(no.shextension) will both work. - Directly executing the test script (e.g.
cding intotestsand invoking./01-end-to-end.shon its own) will no longer work reliably.- Instead, invoke the test using
make test=<test name> - This is because test scripts are provided the environment. If their environment is not properly set up, they won't be able to execute properly. The test controller (
run-tests.sh) insures that the test environment is properly set up.
- Instead, invoke the test using
Drupal is configured to store media on the private:// filesystem, backed by S3. The s3fs module provides this s3-backed implementation of the private:// filesystem. For the idc-isle-dc development environment, minio ([https://minio-idc.traefik.me][https://minio-idc.traefik.me]) provides the S3 API. Cloud/production instances will use Amazon S3.
Configuration is via environment variables
| Env Var | Default Value | Description |
|---|---|---|
| DRUPAL_DEFAULT_MIGRATIONS_VALIDATE | true | Determines whether migrations will perform validation |
| DRUPAL_DEFAULT_S3_ACCESS_KEY | - | S3 access key |
| DRUPAL_DEFAULT_S3_SECRET_KEY | - | S3 secret key |
| DRUPAL_DEFAULT_S3_BUCKET | idc | S3 bucket name |
| DRUPAL_DEFAULT_S3_USE_CUSTOMHOST | false | Connect to an S3-compatible storage service other than Amazon. |
| DRUPAL_DEFAULT_S3_HOSTNAME | - | S3 hostname; use only if DRUPAL_DEFAULT_USE_CUSTOMHOST is true |
| DRUPAL_DEFAULT_S3_USE_CNAME | false | Serve files from a custom domain by using an appropriately named bucket, e.g. "mybucket.mydomain.com". |
| DRUPAL_DEFAULT_S3_USE_PATH_STYLE_ENDPOINT | false | Send requests to a path-style endpoint, instead of a virtual-hosted-style endpoint. For example, http://s3.amazonaws.com/bucket, insead of http://bucket.s3.amazonaws.com. |
- Install browser extension
- make dev-up
- set breakpoint
- load page, trip breakpoint, repeat
- make dev-down
| Env Var | Default Value | Description |
|---|---|---|
| XDEBUG_CONFIG | n/a | Allows for multiple XDebug config params to be set in a string. Used for CLI debugging. |
| XDEBUG_SESSION | n/a | Enables XDebug when present. Used for CLI debugging. |
| XDEBUG_MODE | develop,debug,trace |
Supported XDebug modes. Used for browser-based debugging |
| XDEBUG_LOG | /var/www/drupal/xdebug.log |
XDebug logfile. Used for browser-based debugging |
| XDEBUG_LOGLEVEL | 7 |
XDebug log level. Used for browser-based debuggging. |
| XDEBUG_DISCOVERCLIENTHOST | false |
If XDebug should attempt to discover the debugging client host. Used for browser-based debugging. |
| XDEBUG_CLIENTHOST | host.docker.internal |
The address of the debugging client host. Used for browser-based debugging. |
Allows a developer to launch Drupal with step-wise debugging enabled. The three supported use cases are:
- Debugging human-mediated interactions with Drupal (i.e. what most developers would consider to be "debugging")
- Debugging CLI commands (e.g.
drush) and/or PHP code executed on container startup - Debugging service-mediate interactions with Drupal
Each use case requires setting a different combination of XDebug settings. By default, the Drupal development image is configured to support the primary use-case, step-enabled debugging of PHP from a browser. Enabling all the use cases at once would be counter-productive from a performance standpoint.
Debugging Drupal interactively from a browser requires the installation of an XDebug helper plugin, so if you don't have one installed, you'll need to do so.
To run the debugging container:
- Determine which use case applies, and set the environment accordingly in
docker-compose.drupal-dev.yml. If you want to debug Drupal with your browser and an IDE, you don't need to change anything. - Run
make dev-up
When your debugging session is over, it is important to remember to run
make dev-down
In your IDE, configure a PHP/XDebug development session. Your IDE must listen for debugging requests on port 9003. Set a breakpoint. In your browser, activate the XDebug helper by selecting "Debug", then make a request to Drupal which will activate your breakpoint. Your browser should prompt you to map the file from the remote server to your local filesystem, and then you can perform step-by-step debugging.
There are a few ways to determine whether you've successfully started the development environment correctly:
- First,
docker psshould list a running container named<repo>/drupal-dev:<tag>, and there should be no other containers named<repo>/drupalrunning. - Second, create a file named
xdebug.phpin codebase root:echo "<?php xdebug_info();" > codebase/xdebug.php, then in your browser visithttps://islandora-idc.traefik.me/xdebug.php. Information about XDebug should appear, including its settings. This is a handy way to double-check XDebug's runtime configuration, especially for more complex debugging scenarios.
.gitignore is set up to ignore any file that is prefixed with
xdebug. If you need to create files to support your debug session, or store output from your debug session, name themxdebug<something>to avoid committing debug artifacts.
- A third check, which should be unnecessary, is to
execinto thedrupal-devcontainer and runphp -v; the output should reference XDebug version 3.
If first two conditions are met, you should be able to start a debug session. If you can't, then check for the presence of xdebug.log in the codebase directory for clues, double check your IDE settings, and check the output of php -v in the drupal-dev container. The IDE must accept connections on port 9003, and XDebug must know the IP address of your machine. By default, XDebug connects to host.docker.internal, which is a special DNS name for the host computer.
To switch back from the debugging container:
- Run
make dev-down
Snapshots are Docker images that contain Drupal state (content files, database, SOLR indexes, Fedora files, etc). When Docker starts, all Docker volumes will be populated with files from the snapshot image. The net result is that an environment will start quickly, from a known state, with pre-populated content.
After Docker starts from a snapshot, data subsequent in Docker's volumes is ephemeral. It will persist across stop and down, but can be wiped out by
docker-compose down -v
When docker subsequently starts, it will start from the known snapshot state. You are free to take a snapshot whenever you want a checkpoint you can reliably reset Drupal to.
The image used for the snapshot is specified via environment variables in .env. For example:
# Docker image and tag for snapshot image
SNAPSHOT_IMAGE=birkland/snapshot
SNAPSHOT_TAG=upstream-20201007-739693ae-12-ga409e4d8.1602146397
When the docker-compose.yml make target is run, that image and tag will be specified in the docker-compose file. The images contain data that are
copied to Docker volumes upon initial startup of the stack (i.e. snapshots are deployed only once, until all volumes are wiped out via docker-compose down -v). Because they are just regular docker images, they can be pushed and puled from container registry as usual.
To take a snapshot, run
make snapshot
This will do the following:
- stop the docker-compose stack
- dump the contents of the volumes
- create a new image from the contents of the volume
- give the image a unique tag based on the current git commit, and the date
- update the
.envfile to specify the just-takenSNAPSHOT_TAG - rebuild the
docker-compose.ymlfile to specify that tag - start docker-compose
If you want to commit that snapshot so that others can use it, you need to commit .env (which contains the tag of the snapshot image),
and publish the snapshot image to a Docker registry via
make snapshot-push
Make sure you do both steps! You need to push the image (so others can pull it), and push .env (so others can check out and run it).
Drupal incorporates its own SAML Service Provider using SimpleSAMLphp; an Apache-based SP proxy (c.f. PASS SAML configuration) is not used, and would be incompatible with Drupal. The SimpleSAMLphp SP will be used in development and production environments (including all cloud-based instances).
The developer environment includes a mock Shibboleth Identity Provider (IdP) and LDAP user store to serve as a backend for user attributes. These services would also be used when running integration tests, e.g. as a part of a PR or locally-executed tests. The production environment would not deploy these mock services.
SimpleSAMLphp has an administrative web interface which supports:
- diagnostics and configuration testing
- testing logins, including viewing attribute assertions by the IdP
- SP metadata generation (used to generate SP
EntityDescriptorXML used by Shibboleth IdPs) - IdP XML metadata conversion (used to transform Shibboleth IdP metadata for use by SimpleSAMLphp)
Clicking on the Federation tab, and then click on the link labeled Show metadata under the heading “SAML 2.0 SP Metadata” will generate the SAML metadata to provide to the IdP. Testing authentication and attributes presented by the IdP is best performed using this admin interface.
Runtime parameterization of the IdP and SP is accomplished using a mixture of environment variables and secrets. Two containers use these secrets: the idp and drupal containers. Secrets are defined in docker-compose.yml from the consitutient service definitions in docker-compose.saml.yml and docker-compose.local.yml.
For a deeper understanding of SAML certificate roles, this primer may help.
| Allocation1 | SAML 12 | SAML 23 |
|---|---|---|
| eduPersonAffiliation | urn:mace:dir:attribute-def:eduPersonAffiliation | 1.3.6.1.4.1.5923.1.1.1.1 |
| eduPersonUniqueId | urn:oid:1.3.6.1.4.1.5923.1.1.1.13 | 1.3.6.1.4.1.5923.1.1.1.13 |
| eduPersonPrincipalName | urn:mace:dir:attribute-def:eduPersonPrincipalName | 1.3.6.1.4.1.5923.1.1.1.6 |
| eduPersonScopedAffiliation | urn:mace:dir:attribute-def:eduPersonScopedAffiliation | 1.3.6.1.4.1.5923.1.1.1.9 |
| eduPersonAffiliation | urn:mace:dir:attribute-def:eduPersonAffiliation | 1.3.6.1.4.1.5923.1.1.1.1 |
| employeeNumber | urn:mace:dir:attribute-def:employeeNumber | 2.16.840.1.113730.3.1.3 |
| displayName | urn:mace:dir:attribute-def:displayName | 2.16.840.1.113730.3.1.241 |
| givenName | urn:mace:dir:attribute-def:givenName | 2.5.4.42 |
| departmentNumber | urn:mace:dir:attribute-def:departmentNumber | 2.16.840.1.113730.3.1.2 |
| urn:mace:dir:attribute-def:mail | urn:oid:0.9.2342.19200300.100.1.3 |
1 urn:oasis:names:tc:SAML:2.0:attrname-format:basic
2 urn:mace:shibboleth:1.0:attributeNamespace:uri
3 urn:oasis:names:tc:SAML:2.0:attrname-format:uri
The authsources.php file contains the requested attributes. These are hardcoded, and are not parameterized.
See the SimpleSAMLphp User Sync page for how these attributes may be mapped to Drupal user attributes or role selection.
Note: 'mail' and 'eduPersonPrincipleName' are required attributes
These are truly environment variables (in the sense that they are not secrets). Cloud-based installations will need to update these to expected values.
| Variable Name | Default Value | Location | Description |
|---|---|---|---|
| SP_BASEURL | https://islandora-idc.traefik.me | .env file |
Base URL for SP services. Cloud-based installations will need to update these to expected values. |
| SP_ENTITYID | https://islandora-idc.traefik.me/sp/shibboleth | .env file |
SP entity ID (a URI). Cloud-based installations will need to update these to expected values. |
| IDP_BASEURL | https://islandora-idp.traefik.me:4443 | .env file |
Base URL for IdP services. Cloud-based installations will need to update these to expected values. |
| IDP_ENTITYID | https://islandora-idp.traefik.me/idp/shibboleth | .env file |
IdP entity ID (a URI). Cloud-based installations will need to update these to expected values. |
| DRUPAL_DEFAULT_SITE_URL | https://islandora-idc.traefik.me | docker-compose.env.yml |
Use to derive the SimpleSAMLphp baseurlpath |
| DRUPAL_SP_TECHCONTACTNAME | Moo Cow | docker-compose.env.yml |
Some information about the technical persons running this installation. The email address will be used as the recipient address for error reports, and also as the technical contact in generated metadata. |
| DRUPAL_SP_TECHCONTACTEMAIL | [email protected] | docker-compose.env.yml |
Some information about the technical persons running this installation. The email address will be used as the recipient address for error reports, and also as the technical contact in generated metadata. |
| DRUPAL_SP_PROTECTINDEXPAGE | false | docker-compose.env.yml |
Set this options to true if you want to require administrator password to access the web interface or the metadata pages, respectively. |
| DRUPAL_SP_PROTECTMETADATAPAGE | false | docker-compose.env.yml |
Set this options to true if you want to require administrator password to access the web interface or the metadata pages, respectively. |
| DRUPAL_SP_CHECKFORUPDATES | true | docker-compose.env.yml |
Set this option to false if you don't want SimpleSAMLphp to check for new stable releases when visiting the configuration tab in the web interface. |
| DRUPAL_SP_ASSERTIONALLOWEDCLOCKSKEW | 180 | docker-compose.env.yml |
Set the allowed clock skew between encrypting/decrypting assertions |
| DRUPAL_SP_SESSION_DURATIONSECONDS | 28800 | docker-compose.env.yml |
This value is the duration of the session in seconds. Make sure that the time duration of cookies both at the SP and the IdP exceeds this duration. |
| DRUPAL_SP_SESSION_DATASTORETIMEOUTSECONDS | 14400 | docker-compose.env.yml |
Sets the duration, in seconds, data should be stored in the datastore. As the data store is used for login and logout requests, this option will control the maximum time these operations can take. |
| DRUPAL_SP_SESSION_STATETIMEOUTSECONDS | 3600 | docker-compose.env.yml |
Sets the duration, in seconds, auth state should be stored. |
| DRUPAL_SP_SESSION_COOKIENAME | SimpleSAMLSessionID | docker-compose.env.yml |
Option to override the default settings for the session cookie name |
| DRUPAL_SP_SESSION_COOKIELIFETIMESECONDS | 0 | docker-compose.env.yml |
Expiration time for the session cookie, in seconds. Defaults to 0, which means that the cookie expires when the browser is closed. |
| DRUPAL_SP_SESSION_COOKIEPATH | / | docker-compose.env.yml |
Limit the path of the cookies. Can be used to limit the path of the cookies to a specific subdirectory. |
| DRUPAL_SP_SESSION_COOKIESECURE | false | docker-compose.env.yml |
Set the secure flag in the cookie. Set this to TRUE if the user only accesses your service through https. If the user can access the service through both http and https, this must be set to FALSE. |
| DRUPAL_SP_LOGGING_LEVEL | INFO | simplesaml_config.patch, <vendor dir>/simplesamlphp/simplesamlphp/config/config.php |
Set the log level used by the SimpleSAMLphp SP. Valid values are 'ERR', 'WARNING', 'NOTICE', 'INFO', 'DEBUG' |
| DRUPAL_SP_LOGGING_HANDLER | errorlog | simplesaml_config.patch, <vendor dir>/simplesamlphp/simplesamlphp/config/config.php |
Sets the log handler used by the SimpleSAMLphp SP. Valid values are 'syslog', 'file', 'errorlog', 'stderr' |
It should be said immediately that not every key/value pair in the saml-secrets.yml file is strictly a secret. For example, public keys (in the form of X509 certs) would not be considered a secret. However, to facilitate uniform processing of private/public key pairs and other values, using a single secrets file together with the confd file backend makes sense. saml-secrets.yml uses a structure well-known to the containers that process its contents. Renaming, removing, or adding keys may impact the processing of its content by the idp or drupal containers.
In the Docker environment, there are unfortunate layers of indirection when it comes to the use of secrets.
First and foremost, secrets are defined in docker-compose.yml, which is formed by constituent service files including docker-compose.local.yml (which defines the drupal service) and docker-compose.saml.yml (which defines the idp service). The first indirection decouples the location of the secret from the name of the secret: the secrets are exposed to the containers by their name, independent of their location in the repository.
Generally, a named secret defines a single value. The exception is the secret named saml_secrets; it contains multiple secrets. This is the second indirection, whereby a single named secret can securely expose multiple secret values. (This is in order to leverage the file backend of confd) Values in saml-secrets.yml may themselves be environment variables, which must be processed before they are used. This is the third indirection.
Finally, some values from saml_secrets may be written out to files. This depends on the capabilities of the consumer: some read configuration values inline, others read them from files.
| Composer Secret Name | Composer Location | Confd Key | Container(s) | Used By | Referenced As | References | Notes |
|---|---|---|---|---|---|---|---|
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/signing-key | idp | Shibboleth IdP | file | idp.properties |
Ends with: 'opm/rxS83hCrTsIX3Il3T8Fpb97kdF+unCiWEaxrPEurjW8lB506' |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/signing-cert | idp,drupal | Shibboleth IdP, SimpleSAMLphp | file, inline | idp.properties, idp-metadata.xml, shib13-idp-remote.php |
Ends with: 's00xrv14zLifcc8oj5DYzOhYRifRXgHX' |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/encryption-key | idp | Shibboleth IdP | file | idp.properties |
Ends with: 'qX7ZsBuOT72RwVEa8fpT6IZ6IpOOEPmUid/f2VM2aAcXgaF//vMjxA==' |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/encryption-cert | idp,drupal | Shibboleth IdP, SimpleSAMLphp | file, inline | idp.properties, idp-metadata.xml, shib13-idp-remote.php |
Ends with: 'p+tGUbGS2l873J5PrsbpeKEVR/IIoKo=' |
| idp_sealer | /secrets/idp/sealer.jks | - | idp | Shibboleth IdP | file | idp.properties |
- |
| idp_backchannel | /secrets/idp/idp-backchannel.p12 | - | idp | Jetty | file | backchannel.ini |
Runs on port 8443, may not be used |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/backchannel-signing-cert | idp,drupal | Shibboleth IdP, SimpleSAMLphp | inline | idp-metadata.xml, shib13-idp-remote.php |
PEM version of the PKCS 12 encoded idp_backchannel; ends with: 't6Lf23Kb8yD6ZR7dihMZAGHnYQ/hlhM=' |
| idp_browser | /secrets/idp/idp-browser.p12 | - | idp | Jetty | file | ssl.ini |
Runs on port 4443; this is the cert presented to HTTPS clients for IdP requests |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/idp/browser-cert | - | - | - | - | PEM version of the PKCS 12 encoded idp_browser, seems unused; ends with '7vawjZs0YP5qGifhos34g2GKW81m6sjoxpstLMK7pNQRy/pR/kv/jiXEn8xHRE6s' |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/sp/admin-pw | drupal | SimpleSAMLphp | inline | config.php |
Contains the administrative password for the SimpleSAMLphp interface |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/sp/signing-cert | idp,drupal | Shibboleth IdP, SimpleSAMLphp | inline | sp-metadata.xml, authsources.php |
Ends with: 'lQQUhxyEXTBJx3luLlpIjoloFKIute9K7pE5qAENjg==' |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/sp/hashing-salt | drupal | SimpleSAMLphp | inline | config.php |
Used by SimpleSAMLphp to create secure hashes |
| saml_secrets | /secrets/saml-secrets.yml | /saml-secrets/sp/signing-key | drupal | SimpleSAMLphp | file | authsources.php |
Ends with: 'HPxmTTEX5graPtXeDM3hz5A=' |
Two containers use SAML env vars and secrets: the idp container (the mock Shibboleth Identity Provider) and the drupal container (SimpleSAMLphp Service Provider). Secrets are defined in their accompanying docker-compose files, docker-compose.local.yml and docker-compose.saml.yml.
There are a number of files which contribute to the SAML environment, distributed across the idc-isle-dc and idc-isle-buildkit repositories, and multiple methods used for processing. The complexity is high for a couple of reasons:
- multiple applications with different runtimes (java, nginx/php) consume the same SAML parameters (e.g. the cert used to sign SAML assertions is referenced by both the SP and IdP)
- a single application will use the same parameter inline as well as consume it from a file (so if you want to change a value, and only want to change it in one place, some hoops need to be jumped through)
An example may help: you wish to know how/where the IdP signing key is used. Using the table above, a starting point would be to search the idc-isle-buildkit repo for occurrences of idp/signing-key key. The only hits will be confd configuration, where you see that it is written out to /tmp/idp_signing. Perform a second search for /tmp/idp_signing, and you'll see it used in the idp.properties confd template (which in turn writes idp.properties to /opt/shibboleth-idp/conf/idp.properties).
A second, more complex example: you wish to know how/where the IdP signing cert is used. Using the table, start by searching the idc-isle-buildkit repo for occurrences of idp/signing-cert key. You'll see it referenced by a couple of confd templates which in turn will lead you to idp.properties (references the value in the file /tmp/idp_signing.crt) and idp-metadata.xml (the value is used inline). You will also want to perform a similar search of the idc-isle-dc repository. To cast a wide net, search for the string signing-cert (you'll get some hits from both the SP and IdP). You'll note that the file shib13-idp-remote.php uses PHP to parse the value from the saml_secrets YAML structure (essentially inline use). As things evolve, I do not expect the table below to be kept up-to-date, so it is worth understanding how to discover where and how various secrets are used.
Tips:
- Search in both repositories
- Cast a wide net: searching for
/saml-secrets/idp/signing-certis not going to find hits in PHP code that reference[saml-secrets][idp][signing-cert] - If
confdwrites the value to a file, perform a subsequent search for references to the file. - Having a general understanding of the SAML protocol helps: for example, encryption or signing keys shouldn't be shared between two different containers, but public keys (i.e. those used for verifying signatures) would be.
The IdP entrypoint uses confd to interpolate a number of configuration files that dictate SAML request processing. Because saml_secrets can use environment variables as secret values, the configuration files are subsequently processed through envsubst.
Note that the IdP configuration files may refer to any named secrets that are exposed by Docker, not just the secrets present in saml_secrets.
Finally, processing logic occurs in the idp entrypoint which resides in the idc-isle-buildkit repository. Secrets are defined in the idc-isle-dc repository. The names of secrets are shared between the two repositories; if the name of a secret changes, it must be changed in both.
The Service Provider (SimpleSAMLphp) runs in the drupal container, installed as a vendored composer dependency. A number of PHP configuration files have been modified to reference secrets and environment variables where it makes sense. For example, in config.php 'assertion.allowed_clock_skew' => 180 has been changed to 'assertion.allowed_clock_skew' => intval(getenv('DRUPAL_SP_ASSERTIONALLOWEDCLOCKSKEW')). Instead of inlining certificates, shib-13-idp-remote.php parses saml_secrets. These modifications reside in a patch file managed in the idc-isle-dc repository.
The PHP interpreter executes under PHP-FPM, therefore environment variables must be explicitly passed from NGINX. Confd is used to process SP-related environment variables (using the env backend) and expose them to PHP-FPM via the include file drupal_sp_fastcgi_params. Confd is executed on container start, and is managed in the idc-isle-buildkit repository.
======= Note: In order for the push to be successful, you need to be "logged in" to the container registry via
docker login ghcr.io
See the guide to the github container registry for more information.