pkg-repo.8

.\"
.\" FreeBSD pkg - a next generation package for the installation and maintenance
.\" of non-core utilities.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\"
.\"     @(#)pkg.8
.\"
.Dd January 17, 2021
.Dt PKG-REPO 8
.Os
.Sh NAME
.Nm "pkg repo"
.Nd create a package repository catalogue
.Sh SYNOPSIS
.Nm
.Op Fl lq
.Op Fl m Ar meta-file
.Op Fl o Ar output-dir
.Ao Ar repo-path Ac Op rsa: Ns Ao Ar rsa-key Ac | signing_command: Ao Ar the command Ac
.Pp
.Nm
.Op Cm --{list-files,quiet}
.Op Cm --meta-file Ar meta-file
.Op Cm --output-dir Ar output-dir
.Ao Ar repo-path Ac Op rsa: Ns Ao Ar rsa-key Ac | signing_command: Ao Ar the command Ac
.Sh DESCRIPTION
.Nm
is used to create a catalogue of the available
packages in a repository.
.Nm
catalogues are necessary for sharing your package repository,
and are intrinsic to the operation of
.Nm "pkg install"
or
.Nm "pkg upgrade" .
.Pp
The repository files created by
.Nm
consist of a number of compressed tar archives stored typically at the
top level of the repository filesystem.
Of these,
.Pa meta.txz
must exist at the apex of the repository filesystem.
This is a well-known name that is hard-wired into
.Xr pkg 8 .
.Pp
.Pa meta.txz
contains at least one file:
.Pa meta
which contains a key to the location and format of the other files
comprising the catalogue information.
Other files may have arbitrary names as defined in
.Pa meta ,
but conventionally the following names are used.
.Pp
.Pa digests.txz
contains
.Pa digests
which lists the cryptographic checksums for each of the packages in
the repository.
This is downloaded when
.Cm SIGNATURE_TYPE
is set to
.Ar FINGERPRINTS
in the repository configuration.
.Pp
.Pa filesite.txz
contains
.Pa filesite.yaml
which is a database of all of the files present in all of the packages in
the repository, containing filenames, file sizes and checksums.
Generating
.Pa filesite.txz
involves significant additional system resources and is not usually done.
.Pp
.Pa packagesite.txz
similarly contains at least one file
.Pa packagesite.yaml ,
which is a YAML document listing selected metadata for each of the
packages in the repository.
This is the key file containing the working data used by
.Xr pkg 8
and includes the run-time dependencies for each package,
plus shared library dependencies and similar data that are used by
.Xr pkg 8
to solve package dependency problems.
.Pp
In addition to the files already mentioned, the
.Pa .txz
archives may also contain cryptographic signatures.
These will be produced when the internal signature mechanism of
.Nm
is enabled.
.Pp
Repository users download these files to their local machines, where
they are processed into per-repository sqlite databases for fast
lookup of available packages by programs such as
.Xr pkg-install 8 .
.Pp
To create a package repository catalogue, specify the top-level
directory beneath which all the packages are stored as
.Ar repo-path .
.Nm
will search the filesystem beneath
.Ar repo-path
to find all the packages it contains.
Directories starting with
.Sq \&.
or
named
.Pa Latest
are not traversed.
.Pp
The repository files will be created in the top-level repository directory
unless relocated by specifying
.Fl o Ar output-dir
or
.Cm --output-dir Ar output-dir .
.Pp
Optionally, the repository catalogue may be cryptographically signed.
This is enabled either by specifying the path to an RSA private key as the
.Ar rsa-key
argument or by using an external command.
.Pp
If
.Ar rsa-key
is used, the SHA256 of the repository is signed using the provided key.
The signature is added into the repository catalogue.
The client side should use
.Sy SIGNATURE_TYPE
set to
.Dv PUBKEY
and
.Sy PUBKEY
set to a local path of the public key in its repository configuration file.
.Pp
An external command can be useful to create a signing server to keep the
private key separate from the repository.
The external command is passed the SHA256 of the repository
catalogue on its stdin.
It should output the following format:
.Bd -literal -offset indent
SIGNATURE
signature data here
CERT
public key data here
END
.Ed
.Pp
When using an external command, the client's
.Pa pkg.conf
must have
.Sy SIGNATURE_TYPE
set to
.Dv FINGERPRINTS
and
.Sy FINGERPRINTS
set to a directory having a
.Pa trusted/myrepo
containing a fingerprint style representation of the public key:
.Bd -literal -offset indent
function: sha256
fingerprint: sha256_representation_of_the_public_key
.Ed
.Pp
See the
.Sx EXAMPLES
section and
.Xr pkg.conf 5
for more information.
.Pp
Signing the catalogue is strongly recommended.
.Sh OPTIONS
The following options are supported by
.Nm :
.Bl -tag -width quiet
.It Fl l , Cm --list-files
Generate list of all files in repo as filesite.txz archive.
.It Fl m Ar meta-file , Cm --meta-file Ar meta-file
Use the specified file as repository meta file instead of the default settings.
.It Fl o Ar output-dir , Cm --output-dir Ar output-dir
Create the repository in the specified directory instead of the package directory.
.It Fl q , Cm --quiet
Force quiet output.
.El
.Sh FILES
See
.Xr pkg.conf 5 .
.Sh ENVIRONMENT
.Bl -tag -width "PKG_REPO_SYMLINK"
.It Ev PKG_REPO_HASH
When set, rename packages with the short hash of contents appended to the
filename.
.It Ev PKG_REPO_SYMLINK
When set, create a symlink between the short hash filename and the regular
filename.
.El
.Sh SEE ALSO
.Xr pkg_printf 3 ,
.Xr pkg_repos 3 ,
.Xr pkg-lua-script 5 ,
.Xr pkg-repository 5 ,
.Xr pkg-script 5 ,
.Xr pkg.conf 5 ,
.Xr pkg 8 ,
.Xr pkg-add 8 ,
.Xr pkg-alias 8 ,
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
.Xr pkg-create 8 ,
.Xr pkg-delete 8 ,
.Xr pkg-fetch 8 ,
.Xr pkg-info 8 ,
.Xr pkg-install 8 ,
.Xr pkg-lock 8 ,
.Xr pkg-query 8 ,
.Xr pkg-register 8 ,
.Xr pkg-rquery 8 ,
.Xr pkg-search 8 ,
.Xr pkg-set 8 ,
.Xr pkg-shell 8 ,
.Xr pkg-shlib 8 ,
.Xr pkg-ssh 8 ,
.Xr pkg-stats 8 ,
.Xr pkg-update 8 ,
.Xr pkg-updating 8 ,
.Xr pkg-upgrade 8 ,
.Xr pkg-version 8 ,
.Xr pkg-which 8
.Sh EXAMPLES
Create an RSA key pair:
.Bd -literal -offset indent
% openssl genrsa -out repo.key 2048
% chmod 0400 repo.key
% openssl rsa -in repo.key -out repo.pub -pubout
.Ed
.Pp
Create a repository and sign it with a local RSA key.
The public key would be shared on all client servers with
.Sy SIGNATURE_TYPE
set to
.Dv PUBKEY
and its path set via
.Sy PUBKEY
setting in the repository configuration file:
.Pp
.Dl pkg repo /usr/ports/packages repo.key
.Pp
Create a repository and sign it with an external command.
The client should set, via the repository configuration file,
.Sy SIGNATURE_TYPE
to
.Dv FINGERPRINTS
and
.Sy FINGERPRINTS
to a path containing a file with the SHA256 of the public key:
.Bd -literal -offset indent
# On signing server:
% cat > sign.sh << EOF
#!/bin/sh
read -t 2 sum
[ -z "$sum" ] && exit 1
echo SIGNATURE
echo -n $sum | /usr/bin/openssl dgst -sign repo.key -sha256 -binary
echo
echo CERT
cat repo.pub
echo END
EOF

# On package server:
% pkg repo /usr/ports/packages signing_command: ssh signing-server sign.sh
# Generate fingerprint for sharing with clients
% sh -c '( echo "function: sha256"; echo "fingerprint: $(sha256 -q repo.pub)"; ) > fingerprint'
# The 'fingerprint' file should be distributed to all clients.

# On clients with FINGERPRINTS: /usr/local/etc/pkg/fingerprints/myrepo:
$ mkdir -p /usr/local/etc/pkg/fingerprints/myrepo/trusted
# Add 'fingerprint' into /usr/local/etc/pkg/fingerprints/myrepo/trusted

.Ed