Implementation of the UK NHS dictionary of medicines and devices (dm+d).
This is a lightweight library and microservice for UK dm+d data.
It is a thin wrapper around the dm+d data; dmd creates a SQLite file-based database and a set of APIs
including for the JVM (e.g. Clojure) and via HTTP (via other programming languages). As SQLite databases
use a very stable file format, they are extremely good as acting as containers for rich data, such as dm+d,
as well as for archival purposes. The data can be used directly from a range of programming languages
as most modern systems have SQLite bindings.
The test harness contains an extract of dm+d data that represents public sector information licensed under the Open Government Licence v3.0. These data are reproduced in order to support automated software code tests. The results of these automated tests are shown above.
dmd is designed to be composable with other services, including hermes. Fortunately,
dm+d codes are actually SNOMED identifiers, which means that as long as you run a terminology service that includes the
UK drug extension, you will be able to seamlessly process data across both hermes and dmd. They are naturally
composable and yet can be developed in parallel.
SNOMED releases occur at three-monthly intervals. The NHS Business Services Authority (BSA) release dm+d data weekly.
That means there may be products within dm+d that are not yet within the UK SNOMED CT drug extension. That was one of
the reasons I decided to create dmd as a separate service.
dmd creates what you should regard as an immutable file-based database.
That database can then be used by dmd whether operating as a library or a
microservice. The data is stored in a SQLite database.
You can use dmd to run a server providing an easy to use HTTP API, or use
in-process on the JVM (Java Virtual Machine) from, for example, Clojure,
or directly use the created SQLite from your programming language of choice,
such as Python.
You can either install Clojure and run from source code, or download a runnable jar file.
I suggest installing Clojure for the easiest experience. If you prefer, download a runnable jar file from the published releases.
You will need to get a dm+d distribution via TRUD, which requires registration. The dm+d data is published by the NHS Business Services Authority under an Open Government Licence, but you will need an NHS digital TRUD account in order to download a distribution.
If you want to use automatic downloads, you will also need to download an API key from the service. Copy and paste that key into a file on your local filesystem, if required.
e.g.
echo >/var/local/trud/api-key.txt XXXXXXXXXYou can choose to download manually, or automatically. I'd recommend automatic downloading.
You can download a dm+d distribution from the NHS Digital's TRUD service, on behalf of the NHS BSA. Download the main dm+d distribution; you can optionally also download the 'extras' which include mapping from dm+d to ATC BNF codes. Once downloaded, unzip one or both into a common directory and run:
clj -M:run install --db dmd-2021-07-01.db ~/Downloads/dmd-2021-07-01 or
java -jar dmd-server-0.6.139.jar --db dmd-2021-07-01.db ~/Downloads/dmd-2021-07-01 This will look for files in the directory specified and create a new file-based
database at dmd-2021-07-01.db.
You can also specify multiple directories from which to import.
dmd has automatic functionality to download and install the latest distribution. The process takes a few
minutes.
Automatic downloads need a cache directory. This stops automated tools like this from overloading NHS Digital's infrastructure. If the right file with the right checksum is already available in your local cache, download will be avoided.
Here I use `/var/local/trud/cache' as my cache directory. I share this among a number of tools.
clj -M:run latest --api-key /var/local/trud/api-key.txt --cache-dir /var/local/trud/cacheor
java -jar dmd-server-0.6.139.jar latest --api-key /var/local/trud/api-key.txt --cache-dir /var/local/trud/cacheThis will create a file-based database named dmd-2021-04-12.db (or whatever release date is appropriate). Alternatively, you can manually specify a name:
clj -M:run latest --api-key /var/local/trud/api-key.txt --cache-dir /var/local/trud/cache --db dmd.dbor
java -jar dmd-server-0.6.139.jar latest --api-key /var/local/trud/api-key.txt --cache-dir /var/local/trud/cache --db dmd.dbNote: this does not support updating an existing datafile. It is much better to build a new datafile for each release of the dm+d distribution. You should regard that file as immutable after creation. This is reference data; do not update in place.
You can find available releases on TRUD by looking at their portal or running:
clj -M:run list --api-key /var/local/trud/api-key.txtor
java -jar dmd-server-0.6.139.jar list --api-key /var/local/trud/api-key.txtResult:
➜ dmd git:(main) ✗ clj -M:run list --api-key /var/local/trud/api-key.txt
| :id | :releaseDate | :name |
|--------------------------------------+--------------+----------------|
| nhsbsa_dmd_4.1.0_20210412000001.zip | 2021-04-12 | Release 4.1.0 |
| nhsbsa_dmd_4.0.0_20210405000001.zip | 2021-04-05 | Release 4.0.0 |
| nhsbsa_dmd_3.4.0_20210329000001.zip | 2021-03-29 | Release 3.4.0 |
| nhsbsa_dmd_3.3.0_20210322000001.zip | 2021-03-22 | Release 3.3.0 |To download a specific edition, you can download manually, unzip and then directly import from that distribution.
clj -M:run install --db dmd.db /tmp/downloads/trudOr you can specify the version and let dmd download for you.
Specify which release you want by date according to ISO-8601 standard:
clj -M:run --api-key /var/local/trud/api-key.txt download 2021-04-05Once you have downloaded a distribution, you can use it to run a very fast HTTP server.
clj -M:run serve --db dmd-2021-04-12.db --port 8080or
java -jar dmd-server-0.6.139.jar --db dmd-2021-04-12.db --port 8080As it is very likely that the complete dm+d dataset will fit into the memory of a small server, this service can scale easily. It is entirely reasonable to run behind an API gateway and share a read-only filesystem across multiple instances should it be necessary. It is also entirely reasonable to keep older instances running to provide access to data from a particular time point should that be necessary. Clients can then choose to use a specific version or simply the latest dataset.
The HTTP server is under active development. There are a number of endpoints available that simplify obtaining and making sense of dm+d data.
These endpoints are complete:
- Get denormalized information for a given product.
- Map between BNF or ATC codes and dm+d products.
- Get lookup definitions
- Get associated products for any given product.
Endpoints are currently under development:
- Convert between product-based prescribing models to dose-based models and back.
Further documentation will be forthcoming.
Example usage:
$ http -j http://127.0.0.1:8080/dmd/v1/product/12797611000001109 Result will be the denormalized product. Each product will include important references such as ingredients, excipients and the associated products. A VMP will include its VTM. An AMP or VMPP will include its VMP, and the VTM, and an AMPP will include all of the others!
{
"BASIS": {
"CD": 1,
"DESC": "rINN - Recommended International Non-proprietary"
},
"BASISCD": 1,
"BASIS_PREVCD": 1,
"BNF_DETAILS": {
"ATC": "C08CA01",
"DDD": 5.0,
"DDD_UOM": {
"CD": 258684004,
"DESC": "mg"
},
"DDD_UOMCD": 258684004,
"VPID": 12797611000001109
},
"CONTROL_DRUG_INFO": {
"CD": 0,
"DESC": "No Controlled Drug Status"
},
"DF_IND": {
"CD": 2,
"DESC": "Continuous"
},
"DF_INDCD": 2,
"DRUG_FORM": {
"CD": 385023001,
"DESC": "Oral solution"
},
"DRUG_ROUTES": [
{
"CD": 26643006,
"DESC": "Oral"
}
],
"ID": 12797611000001109,
"INGREDIENTS": [
{
"BASIS_STRNT": {
"CD": 1,
"DESC": "Based on Ingredient Substance"
},
"BASIS_STRNTCD": 1,
"IS": {
"ISID": 386864001,
"ISIDDT": "2005-07-27",
"ISIDPREV": 3512211000001104,
"NM": "Amlodipine"
},
"ISID": 386864001,
"STRNT_DNMTR_UOMCD": 258773002,
"STRNT_DNMTR_VAL": 1.0,
"STRNT_NMRTR_UOMCD": 258684004,
"STRNT_NMRTR_VAL": 1.0,
"VPID": 12797611000001109
}
],
"INVALID": true,
"NM": "Amlodipine 5mg/5ml oral solution sugar-free",
"NMDT": "2015-06-08",
"NMPREV": "Amlodipine 5mg/5ml oral solution sugar free",
"NON_AVAIL": {
"CD": 1,
"DESC": "Actual Products not Available"
},
"NON_AVAILCD": 1,
"NON_AVAILDT": "2016-12-06",
"ONT_DRUG_FORMS": [
{
"CD": 5,
"DESC": "solution.oral"
}
],
"PRES_STAT": {
"CD": 1,
"DESC": "Valid as a prescribable product"
},
"PRES_STATCD": 1,
"SUG_F": true,
"TYPE": "VMP",
"VPID": 12797611000001109,
"VTM": {
"ID": 108537001,
"NM": "Amlodipine",
"TYPE": "VTM",
"VTMID": 108537001
},
"VTMID": 108537001
}You can search by exact name, if you have data without dm+d codes, but derived from the dm+d dataset.
This is a case-sensitive search.
http -j 'http://127.0.0.1:8080/dmd/v1/search?s=Amlodipine 5mg/5ml oral solution sugar-free'Result:
{
"BASIS": {
"CD": 1,
"DESC": "rINN - Recommended International Non-proprietary"
},
"BASISCD": 1,
"BNF_DETAILS": {
"ATC": "C08CA01",
"BNF": "02060200",
"DDD": 5.0,
"DDD_UOM": {
"CD": 258684004,
"DESC": "mg"
},
"DDD_UOMCD": 258684004,
"VPID": 39732011000001102
},
"CONTROL_DRUG_INFO": {
...Here I look for ATC codes beginning with L04AX.
Anatomical Therapeutic Chemical (ATC) codes are a useful way of categorising drugs,
and so you can use dmd to generate codelists for operational, realtime or
offline analytics.
The standard dm+d includes ATC codes for VMPs only, but this service can easily map between dm+d products and give results for VTMs and AMPs as well.
http -j 127.0.0.1:8080/dmd/v1/atc/L04AX/vmpsResult:
[
{
"NM": "Thalidomide 100mg tablets",
"VPID": 10194111000001109
},
{
"NM": "Azathioprine 30mg/5ml oral suspension",
"VPID": 8308811000001100
},
{
"NM": "Azathioprine 15mg/5ml oral suspension",
"VPID": 8283211000001104
},
{
"NM": "Methotrexate 750micrograms/5ml oral suspension",
"VPID": 12815511000001103
},
{
"NM": "Azathioprine 30mg capsules",
"VPID": 8791411000001107
},
...http 127.0.0.1:8080/dmd/v1/atc/L03AX13/productsresult:
[
{
"ID": 29828011000001100,
"NM": "Glatiramer acetate 40mg/1ml solution for injection pre-filled syringes",
"TYPE": "VMP"
},
{
"ID": 400446007,
"NM": "Glatiramer acetate 20mg powder and solvent for solution for injection vials",
"TYPE": "VMP"
},
{
"ID": 36051411000001106,
"NM": "Glatiramer acetate 20mg/1ml solution for injection pre-filled syringes",
"TYPE": "VMP"
},
{
"ID": 108754007,
"NM": "Glatiramer",
"TYPE": "VTM"
},
{
"ID": 8261711000001107,
"NM": "Copaxone 20mg/1ml solution for injection pre-filled syringes",
"TYPE": "AMP"
},
{
"ID": 29821511000001103,
"NM": "Copaxone 40mg/1ml solution for injection pre-filled syringes",
"TYPE": "AMP"
},
{
"ID": 4880311000001105,
"NM": "Copaxone 20mg powder and solvent for solution for injection vials",
"TYPE": "AMP"
},
{
"ID": 34807711000001106,
"NM": "Brabio 20mg/1ml solution for injection pre-filled syringes",
"TYPE": "AMP"
},
{
"ID": 35041211000001107,
"NM": "Brabio 40mg/1ml solution for injection pre-filled syringes",
"TYPE": "AMP"
}
]As the dm+d dataset from the NHS BSA does not include products representing
trade families, a codelist generated purely from dm+d will not include all
device or medicine products potentially recorded in electronic health and
care systems. As such, dmd can generate a SNOMED expression that can be used
in a terminology server to generate a codelist, or check that a medication is
a match for that constraint. I recommend hermes
to do this, as it is lightweight and fast, and can expand those expressions
easily.
http -j 127.0.0.1:8080/dmd/v1/atc/L03AX13/eclResult
{
"atc": "L03AX13",
"ecl": "<<29828011000001100 OR <<400446007 OR <<36051411000001106 OR <<108754007 OR <<(>8261711000001107 AND <9191801000001103) OR <<(>29821511000001103 AND <9191801000001103) OR <<(>4880311000001105 AND <9191801000001103) OR <<(>34807711000001106 AND <9191801000001103) OR <<(>35041211000001107 AND <9191801000001103)"
}Many products are associated with different LOOKUPS, such as BASIS_OF_NAME.
dmd can provide these values, although the other endpoints provide
denormalized versions of products providing lookup information in-place so
it is less likely you will need this endpoint.
For a full list of supported lookups, see the dm+d technical specification.
Example:
http -j '127.0.0.1:8080/dmd/v1/lookup/BASIS_OF_NAME' [
{
"CD": 3,
"DESC": "BANM - British Approved Name (Modified)"
},
{
"CD": 6,
"DESC": "USAN - United States Adopted Name"
},
{
"CD": 2,
"DESC": "BAN - British Approved Name"
},
{
"CD": 5,
"DESC": "pINN - Proposed International Non-proprietary Name"
},
{
"CD": 4,
"DESC": "INNM - International Non-proprietary Name (Modified)"
},
{
"CD": 1,
"DESC": "rINN - Recommended International Non-proprietary"
},
{
"CD": 7,
"DESC": "Other"
}
]You can obtain all of the VMPs of a particular product, such as a VTM:
This just works for different products e.g. /vtms /vmps /amps
Examples:
http -j 127.0.0.1:8080/dmd/v1/product/109143003/vmps
http -j 127.0.0.1:8080/dmd/v1/product/109143003/vtms
http -j 127.0.0.1:8080/dmd/v1/product/109143003/vtm
http -j 127.0.0.1:8080/dmd/v1/product/109143003/amps[
{
"ID": 330057007,
"NM": "Leflunomide 10mg tablets",
"TYPE": "VMP"
},
{
"ID": 330058002,
"NM": "Leflunomide 20mg tablets",
"TYPE": "VMP"
},
{
"ID": 330059005,
"NM": "Leflunomide 100mg tablets",
"TYPE": "VMP"
},
{
"ID": 24408011000001101,
"NM": "Leflunomide 15mg tablets",
"TYPE": "VMP"
}
]While dmd can be run as an independent microservice, it can also be embedded into a larger application or service as a
library.
eg. use a git coordinate in deps.edn:
com.eldrix/dmd {:git/url "https://github.com/wardle/dmd.git"
:sha "XXX"}dmd provides a simple library, and optionally a microservice, to help you make use of the UK dm+d.
A library can be embedded into your application; this is easy using Clojure or Java. You make calls using the dmd API just as you'd use any regular library.
A microservice runs independently and you make use of the data and software by making an API call over the network.
Like all PatientCare components, you can use dmd in either way. Usually, when you're starting out, it's best to use
as a library but larger projects and larger installations will want to run their software components independently,
optimising for usage patterns, resilience, reliability and rate of change.
The current focus is on making dm+d data available in your applications.
The next focus will be on providing functionality to support more sophisticated processing of prescribing data, in particular a way to round-trip between dose-based and product-based prescribing. Actually, all of this code has been designed to support building that functionality!
Previously, I implemented dm+d within an EPR and later within a SNOMED terminology server. However, the UK drug extension lacks some of the data that is included in the NHS BSA dm+d distribution, such as the concrete data that are needed in order to safely map from dose-based prescribing to product-based prescribing. It is difficult to build software that processes prescribing data without those concrete values documenting how much of each ingredient is in each product.
dmd provides data and computing services for the full UK dm+d dataset. It's there if you need to supplement the UK
SNOMED drug extension with additional data. That will almost certainly be the case if you're writing an e-prescribing
system.
If the SNOMED-based UK drug extension provides all of the information you need in your user-facing applications, you
won't need to use dm+d; simply use hermes instead.
Small modules of functionality are easier to develop, easier to understand, easier to test and easier to maintain. I design modules to be composable so that I can stitch different components together in order to solve problems.
In larger systems, it is easy to see code rotting. Dependencies become outdated and the software becomes difficult to change easily because of software that depend on it. Small, well-defined modules are much easier to build and are less likely to need ongoing changes over time; my goal is to need to update modules only in response to changes in domain not software itself. Accretion of functionality.
It is very difficult to 'prove' software is working as designed when there are lots of moving parts.
One of the core abtractions used across the PatientCare components is identifier resolution and mapping. It might be possible to use a single terminology server to import a range of what HL7 FHIR regards as a terminology, or a value set, but that couldn't work for me.
With a federatable model, I can easily add new terminologies and value sets, such as resolving dm+d identifiers or OMIM references, to a loosely-coupled terminology server.
I am building multiple discrete but composable data and computing services - they are building blocks.
Yes. It is better to start with data and build computing services around those data and map to higher-level abstractions at run-time. I used to import and map at the time of import, but that presupposes that I know what I will need both now and in the future, and experience has taught me that I know neither!
I am building user-facing applications by composing these building blocks together. The shiny applications are under development, but we need to start with the basics.
I have spent a lot of time trying to influence how we build software within the public sector, and while I've made progress, most decision-makers are neither technical nor clinical.
Instead, I think the best way to effect change is to demonstrate how we can build health and care software in a new way, focusing on data-centricity, openness and adopting a standards-based approach. Open-source is a critical component of that.
There is too much incidental complexity in health and care software, and too many moving parts. The domains of health and care are too complex to add to that burden by using the wrong technical and architectural approaches. These tools show a new approach based on data and its flow between loosely-coupled components.
The COCOMO model provides an estimate of the cost of software. I don't think it is accurate, at all, but when you look at open-source software it is important to recognise that open-source work is not free. There is a cost and it is sensible to remind people of that.
There is good information available online. The latest documentation is from 2017 from the NHSBSA - https://www.nhsbsa.nhs.uk/sites/default/files/2017-02/Technical_Specification_of_data_files_R2_v3.1_May_2015.pdf.
There are five main dm+d classes:
- VTM
- VMP
- AMP
- VMPP
- AMPP
Each class has associated properties. The cardinalities of the different classes and properties are shown on page 5 of that document.
In order to be safe, I have automated reports that process dm+d data to confirm assumptions. For example, here is an automated report on dm+d data confirming the cardinalities:
| :TYPE | :MAX-CARDINALITY | :CARDINALITY |
|------------------------------------+------------------+--------------|
| [:VMP :VIRTUAL_PRODUCT_INGREDIENT] | 29 | :TO-MANY |
| [:VMP :ONT_DRUG_FORM] | 9 | :TO-MANY |
| [:VMP :DRUG_FORM] | 1 | :TO-ONE |
| [:VMP :DRUG_ROUTE] | 7 | :TO-MANY |
| [:VMP :CONTROL_DRUG_INFO] | 1 | :TO-ONE |
| :TYPE | :MAX-CARDINALITY | :CARDINALITY |
|------------------------+------------------+--------------|
| [:AMP :AP_INGREDIENT] | 16 | :TO-MANY |
| [:AMP :LICENSED_ROUTE] | 6 | :TO-MANY |
| [:AMP :AP_INFORMATION] | 1 | :TO-ONE |
| :TYPE | :MAX-CARDINALITY | :CARDINALITY |
|---------------------------+------------------+--------------|
| [:VMPP :DRUG_TARIFF_INFO] | 1 | :TO-ONE |
| [:VMPP :COMB_CONTENT] | 875 | :TO-MANY |
| :TYPE | :MAX-CARDINALITY | :CARDINALITY |
|-------------------------------------+------------------+--------------|
| [:AMPP :APPLIANCE_PACK_INFO] | 1 | :TO-ONE |
| [:AMPP :DRUG_PRODUCT_PRESCRIB_INFO] | 1 | :TO-ONE |
| [:AMPP :MEDICINAL_PRODUCT_PRICE] | 1 | :TO-ONE |
| [:AMPP :REIMBURSEMENT_INFO] | 1 | :TO-ONE |
| [:AMPP :COMB_CONTENT] | 1085 | :TO-MANY |
| :TYPE | :MAX-CARDINALITY | :CARDINALITY |
|---------------+------------------+--------------|
| [:GTIN :AMPP] | 1 | :TO-ONE |
From the documentation, it is not clear that would only be a 1:1 map from AMPP to GTIN, as it looks as if GTINs can expire. This data quality report confirms the to-one relationship.
It's easy to run this from a clojure REPL in order to test another release, or you can run it from the command line:
clj -X:cardinalities :dir '"/Users/mark/Downloads/nhsbsa_dmd_3.4.0_20210329000001/"'clj -X:deps treeclj -M:outdatedclj -M:checkclj -X:testclj -M:lint/eastwood
clj -M:lint/kondoclj -T:build jarclj -T:build jar
java -jar target/dmd-server-0.6.139.jar --help