matrix.secrets

manage secrets in bash - https://github.com/shadowbq/matrix.dot.files (extracted from mono.repo)

We should never store unecrypted secrets on our machines. Storing un-encrypted Secret ENVs in a file is bad idea®.

Table of Contents

• Methodology
• Usage
• Implementation
• Setup
  • Install GPG and Init
  • Set your pin entry method
• New Secrets - Creation Securely using RAMDisks
• Appendix
  • Loading of Secrets - Manual Implementation
  • GnuPG
    • Creating a new GPG Key
    • Working with Existing GPG Keys in more than one location.
      • Extract private key and import on different machine
      • Register an Existing Key
      • Trust the newly Imported key

Methodology

Store secrets as an RSA 2048 encrypted Base64 bash ENV file that gets decrypted in memory via GPG and sourced as needed into Bash.

Usage

$> env |grep -i SECRET_TOKEN
$> secrets_load
Please enter the passphrase to unlock the OpenPGP secret key:
"scott macgregor <[email protected]>"
2048-bit RSA key, ID 0123456789ABCDEF,
created 2019-12-28 (main key ID 0123456789ABCDEF).

Passphrase:
$> env |grep -i SECRET_TOKEN
SECRET_TOKEN=abcdefg12345678ZYXWV

Implementation

Store secrets as ENVs on a file (.bash_secrets) that can be sourced from Bash, but don't write the file to the hard-drive. Instead write it RSA 2048 encrypted then Base64 as ( ~/.bash_encrypted) and decrypt in memory and source it as needed.

It is implemented as an alias secrets_load which evals bash function _secrets_decrypt on the ~/.bash_encrypted file using gpg keys that are pin secured.

Setup

Install GPG and Init

Install the GPG client

• NOTE: that on MacOS the command isn't gpg2 but rather just gpg.

$(macOS)>  brew install gpg
$(deb/ubuntu)> apt install gnupg
$(linux/rhel)> yum install gnupg
$(bsd) > gpg

Init GPG

gpg --list-keys
gpg: directory '/Users/smacgregor/.gnupg' created
gpg: keybox '/Users/smacgregor/.gnupg/pubring.kbx' created
gpg: /Users/smacgregor/.gnupg/trustdb.gpg: trustdb created

Set your pin entry method

You are required to have a pin entry application that works(looking at you mac)

• $(macOS)> brew install pinentry-mac

• $(deb/ubuntu)> apt install pinentry-tty

• $(linux/rhel)> yum install pinentry-tty

• SO - Install Help

Under mac homebrew

ls -la /usr/*bin/pinentry*
lrwxr-xr-x  1 smacgregor  admin  39 Nov  5 09:15 /usr/local/bin/pinentry -> ../Cellar/pinentry/1.1.0_1/bin/pinentry
lrwxr-xr-x  1 smacgregor  admin  46 Nov  5 09:15 /usr/local/bin/pinentry-curses -> ../Cellar/pinentry/1.1.0_1/bin/pinentry-curses
lrwxr-xr-x  1 smacgregor  admin  45 Nov  5 09:31 /usr/local/bin/pinentry-mac -> ../Cellar/pinentry-mac/0.9.4/bin/pinentry-mac
lrwxr-xr-x  1 smacgregor  admin  43 Nov  5 09:15 /usr/local/bin/pinentry-tty -> ../Cellar/pinentry/1.1.0_1/bin/pinentry-tty

Under ubuntu/debian

ls -la /usr/*bin/pinentry*
lrwxrwxrwx 1 root root    26 Sep 12 19:34 /usr/bin/pinentry -> /etc/alternatives/pinentry
-rwxr-xr-x 1 root root 68240 Mar 22  2020 /usr/bin/pinentry-curses
-rwxr-xr-x 1 root root 60048 Mar 22  2020 /usr/bin/pinentry-tty

A pure cli experience on servers or terminal

echo "pinentry-program /usr/bin/pinentry-tty" >> ~/.gnupg/gpg-agent.conf

For debian/ubuntu you MUST update the alternatives

$(deb/ubuntu)> sudo update-alternatives --install /usr/bin/pinentry pinentry /usr/bin/pinentry-tty 200

For macos/OSX you can ALTERNATIVELY use a GUI/popup which also works with keychain

$(macOS)> brew install pinentry-mac
$(macOS)> echo "pinentry-program /usr/local/bin/pinentry-mac" >> ~/.gnupg/gpg-agent.conf

Reload the GPG Agent (or kill it to force a restart)

gpg-connect-agent reloadagent /bye
# `killall gpg-agent` if that not working correctly

Note (This was seen as required at least on macos):

env |grep GPG
GPG_TTY=/dev/ttys001

else

echo 'export GPG_TTY=$(tty)' >> ~/.bash_profile

Download and include .matrix.secrets in your .bashrc

curl -O -J -L "https://raw.githubusercontent.com/shadowbq/matrix.secrets/main/bash/.matrix.secrets" > ~/.matrix.secrets
echo 'Load in this session: "source ~/.matrix.secrets"'
echo 'source ~/.matrix.secrets' >> ~/.bashrc

New Secrets - Creation Securely using RAMDisks

Note: If you don't have a key yet see the GnuPG primer.

Given that you have a SECRET GPG KEY 0123456789ABCDEF0123456789ABCDEF listed as ultimate.

$> gpg --list-secret-keys
/Users/scottmacgregor/.gnupg/pubring.kbx
----------------------------------------
pub   rsa2048 2019-12-28 [SC] [expires: 2021-12-27]
      0123456789ABCDEF0123456789ABCDEF
uid           [ultimate] scott macgregor <[email protected]>
sub   rsa2048 2019-12-28 [E] [expires: 2021-12-27]

Encrypt a bash script with contents: export SECRET_TOKEN=abcdefg12345678ZYXWV securely into .bash_encrypted

Option 1) 'Ensured Security'

• https://unix.stackexchange.com/a/271870/104660)
• tmpfs + luks\encryption + ext4 + immediate destruction

Option 2) 'Good Enough' new ramDisk + immediate destruction

# Make your Linux secrets securely 
$(linux)> mkdir -p $HOME/tmpfs
$(linux)> sudo mount -t tmpfs -o size=128m ramfs $HOME/tmpfs

# Make your MacOS secrets securely (macos_ramdisk is in .matrix/Darwin/bin)
$(macOS)> sudo macos_ramdisk mount

Example UNENCRYPTED .bash_secrets file

export SECRET_TOKEN=abcdefg12345678ZYXWV
export SECRET_CLIENT_ID=abcdefg12345678ZYXWV
export SECRET_FOO_CLIENT=abcdefg12345678ZYXWV

Create the .bash_secrets in the tmpfs

vi $HOME/tmpfs/.bash_secrets
[..Write.Secrets.here..]

Encrypt it with the keyid 0123456789ABCDEF0123456789ABCDEF

cat $HOME/tmpfs/.bash_secrets | gpg --encrypt -r 0123456789ABCDEF0123456789ABCDEF --armor |base64 > ~/.bash_encrypted

Test the Encrypted Text

cat ~/.bash_encrypted |base64 -d |gpg --decrypt

'Good enough' Destruction of Secrets

$(linux)> sudo umount $HOME/tmpfs
-or-
$(macOS)> sudo macos_ramdisk umount $HOME/tmpfs

Appendix

Loading of Secrets - Manual Implementation

As an alternative to secrets_load alias, you can manually decrypt and load into current tty ENV.

$> eval $(cat ~/.bash_encrypted |base64 -d |gpg --decrypt 2> /dev/null)

GnuPG

This section will assist your in creating new keys, or use existing keys GPG across multiple machines.

Creating a new GPG Key

gpg --full-generate-key

1. At the prompt, specify the kind of key you want, or press Enter to accept the default.
2. At the prompt, specify the key size you want, or press Enter to accept the default. Your key must be at least 4096 bits.
3. Enter the length of time the key should be valid. Press Enter to specify the default selection, indicating that the key doesn't expire.
4. Verify that your selections are correct.
5. Enter your user ID information.
6. Type a secure passphrase.

gpg (GnuPG) 2.3.2; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
   (9) ECC (sign and encrypt) *default*
  (10) ECC (sign only)
  (14) Existing key from card
Your selection? <default>
Please select which elliptic curve you want:
   (1) Curve 25519 *default*
   (4) NIST P-384
   (6) Brainpool P-256
Your selection? <default>
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y

Validate the Key is Ultimate

$> gpg --list-secret-keys --keyid-format=long
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
/Users/shadowbq/.gnupg/pubring.kbx
------------------------------------
sec   ed25519/0123456789ABCDEF 2021-10-21 [SC]
      0123456789ABCDEF0123456789ABCDEF
uid                 [ultimate] scott macgregor <[email protected]>
ssb   cv25519/0123456789ABCDEF 2021-10-21 [E]

Working with Existing GPG Keys in more than one location.

Extract private key and import on different machine

Identify your private key by running

gpg --list-secret-keys --keyid-format=LONG.

You need the ID of your private key (second column)

0123456789ABCDEF

Run this command to export your key:

gpg --export-secret-keys $ID > ~/.ssh/my-gpg-private-key.asc.

Copy the key to the other machine ( scp is your friend)

scp ~/.ssh/my-gpg-private-key.asc target:~/.ssh/..

Register an Existing Key

To import the key on the target-server, run

$> gpg --import ~/my-gpg-private-key.asc

Please enter the passphrase to import the OpenPGP secret key:
"scott macgregor <[email protected]>"
256-bit EDDSA key, ID 0123456789ABCDEF,
created 2021-10-21.

Passphrase:
gpg: key 0123456789ABCDEF: secret key imported
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg:       secret keys read: 1
gpg:   secret keys imported: 1

Trust the newly Imported key

Ensure the keys are correct by observing the ID with LONG format:

gpg --list-secret-keys --keyid-format=LONG gpg --list-keys --keyid-format 0xLONG

Everything showed up as normal except for the uid which now reads [unknown]:

uid [ unknown ] User < [email protected] >

Bump that trust, because its yours!

$> gpg --edit-key [email protected]
gpg> trust

Please decide how far you trust this user to correctly verify other users\' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y
gpg> save

Validate it is now ultimate trust.

gpg --list-keys --keyid-format 0xLONG

uid [ ultimate ] User < [email protected] >