SHIELD v8.4.1

Improvements

• Added a /v2/mbus/status API endpoint that returns metrics about the state of
  the event message bus. This allows for some introspection into what websocket
  connections may be doing at that moment.
• The UI now has less wasted space when displaying a bunch of cards for data
  systems.
• Implemented a configurable timeout when making SSH connections to agents. A
  lower default now also keeps dead agents from taking up large amounts of
  scheduler worker time.
• SHIELD now tracks when the last time an agent erred was.
• Hidden agents are now sorted under a separate header in the web UI.
• Hiding, showing, and deleting agents can now be done from the CLI.

Bug Fixes

• We no longer leak the file descriptors and goroutines for detached websocket
  clients
• Workers can no longer be starved out when sending events to the message bus
  if the receiver of the message bus is misbehaving because these event sends
  are now asynchronous.
• Fixed a bug where a worker could derefence a nil pointer when certain
  database selects returned no rows.
• The database layer now has more stringent locking, which both avoids certain
  threads locking each other out in SQLite, and also makes certain series of
  database operations effectively atomic.
• A couple of fixups would deadlock themselves out of the database and prevent
  fixups from actually running. Now they don't.
• Fixups now only run once instead of on every startup, like nature intended.
• The agent "Last Checked At" timestamp was being updated when the task was
  pulled off the scheduler, whether or not the agent was actually checked
  (due to other potential errors).
• Named a fixup without a name.
• Agents that failed their status checks are now once again marked as such.

SHIELD v8.4.0

New Features

• SHIELD now features a new etcd plugin for backing up and restoring your etcd key-value stores. It supports single- and multi-node clusters and can authenticate via roles and X.509 certificates. If you want, you can restrict the backup to a subset of the etcd tree (via a prefix setting). It also supports additive restore for situations that need it. You can also do a clean restore of the etcd cluster.

Improvements

• The cancel, task, restore-archive and purge-archive
  commands in the SHIELD CLI now properly support short UUIDs,
  like all other commands.

SHIELD v8.3.0

Improvements

• Submit buttons on forms now (a) disable themselves when clicked
  and (b) change their text to indicate an ongoing operation.
  This greatly increases the usability of the web UI. See #505

• The web UI for rekeying SHIELD Core now correctly identifies
  when the operator would like to rotate the fixed key. Also, the
  error messaging for an incorrect current master password is
  better now, and by default, the "rotate fixed key" checkbox on
  the rekeying form is off. See #546

• The default password for the failsafe account has been changed
  from shield to password, for more continuity across various
  packaging formats. See #531

• The shield tasks command (and the backing API) can now filter
  tasks based on their task type (i.e. "backup", or "restore")
  See #523

• The Encryption column of the system detail page's backup jobs
  table now always shows something. For jobs that do not used
  the fixed key, the new tag is randomized. See #536

• SHIELD now tracks when it last checked each agent separately
  from when it last "saw" the agent. Last Seen now means the
  point in time when the agent last connected to the SHIELD core,
  and Last Checked is when the core last connected to the agent
  for metadata retrieval.

• SHIELD now allows agents to change their IP address; only the
  agent name is unchangeable. Previously, attempts to change an
  agents registered IP address (without changing its name) would
  fail.

• Both the SHIELD Agent and the SHIELD CLI now trust the system
  X.509 CA Certificate Stores. See #555 and #556

Bug Fixes

• The MotD separator no longer displays if the MotD is empty
  or not specified. See #530

• The Ad Hoc Backup and Restore wizards now handle the "empty"
  state more gracefully, and instead of showing an empty table
  when there are no data systems, they warn you that you have
  no systems to backup or restore. See #532 and #533

• Stores (global and tenant-specific) can now be properly deleted
  via the web UI and CLI.

• When editing targets and stores on the webui changes are now
  persisted when editing again without a refresh.

• The "Agents of SHIELD" admin page no longer gets stuck in a
  loading loop whenever websocket events are seen.

• Global Storage Configuration details are now properly loaded
  when accessing a global store detail page from the admin panel.
  Previously, the details would only load if you visited the
  global store from the "Storage Systems" (non-admin) top-level,
  which only works if you actually have a tenant. See #535

• shield annotate-archive now works with short uuids

• Orphaned archives from a target deletion no longer prevent users
  from loggin into SHIELD

• The mongo plugin now handles hosts with embedded ports,
  allowing operators to configure multi-node mongo across
  different ports on the same node, without --port getting in
  the way.

• The mongo plugin now allows you to authenticate against a
  database other than "admin" (which remains the default).

SHIELD v8.2.1

Improvements

• The swift plugin now supports the Openstack v3 Identity API

• The vault plugin now supports k/v v2 endpoints, which are
  now the default in contemporary versions of Vault.

SHIELD v8.2.0

Bug Fixes

• The shieldd binary now properly reports its release version in
  both CLI (-v) and web UI contexts.

• The archives list on the system page now no longer gives you the
  option of restoring invalid archives (i.e. purged stuff).
  Thanks @thomasmitchell for finding and reporting in #506.

• System-initiated archive purges now properly set the store agent
  for purgation, so that the purge task has someone to talk to for
  removal of the archive from cloud storage. See #514.

• The core scheduler now immediately fails any task for which the
  remote SHIELD agent does not signal a successful (rc=0) exit
  status. This should clean up some task logs, and remove red
  herring issues like JSON unmarshal failures, while
  simultaneously ensuring that failed purge tasks are re-tried.
  See #518.

• Purge tasks are now being properly supplied with the restore key
  necessary for deleting the archive blob. See #516.

• Agent Status tasks (op agent-status) were not previously being
  created with proper global tenant association. This prohibited
  operators from viewing the details of those tasks. We fixed
  this, and added a data fixup created to re-associate existing
  tasks. See #522.

• The HUD now always registers the global cloud storage in its
  health data, so operators are aware of all issues with storaage
  systems that they might be using, global or tenant-private.
  See #504.

• Jobs created via the Web UI now properly set their "KeepN"
  attribute, which was missing from the ingestion / insertion.
  Accompanying this is a new data fixup that should re-calculate
  the keep_n database field wherever it is zero. See #460.

• Unlocking SHIELD was not properly giving feedback when a wrong
  password was given or no password at all. This is now working.
  See #524.

• Previously we had issues doing a fs backup and restore of SHIELD
  itself. Improvments to the fs plugin, and fixing permissions issues
  within core now allow SHIELD to properly backup and restore itself.

Improvements

• All -v handlers in CLI utilities now properly handle the 'dev'
  version as analogous to the empty ('') version, and revert to
  reporting the version of the binary as '(development)'. This is
  mainly for packaing Docker images properly.

• All shield* CLI utilities, include the shield CLI itself,
  the shieldd daemon, and all helper binaries now sport options
  for getting their usage (--help) and versions (--versions).

• The s3 plugin now accepts a URL as its s3_host endpoint
  parameter, affording operators more flexibility.
  The alternative was confusion! See #509.

• When purging archives manually, you can now supply
  human-friendly reasons for the purge. For example, if the data
  is known to be bad in that particular vintage of the target
  system, you can purge the archives containing it, and explain
  that. See #520

• Archives can now be annotated from the command-line, with the
  annotate-archive command.

• Manually purged archives now track their reason for purge as
  "manually purged", instead of "expired". See #517.

• All system- and tenant-level objects can now be searched for,
  and referenced by short UUIDs. This is huge (though short),
  going a long way to making the CLI easier to work with.

Release Engineering

• Docker images can now be built with embedded release versions,
  for non-dev distribution as a container image.

SHIELD v8.1.0

New Features

• The SHIELD Web UI now allows you to download the SHIELD CLI
  directly, for both MacOS (Darwin) and Linux. From now on,
  SHIELD releases will include the paired version of the CLI.

• We now support minutely backups, but only from the CLI.

• New shield op pry for decrypting and inspecting the contents
  of a SHIELD Vault Crypt.

Improvements

• SHIELD now cleans up the Vault when archives are marked as
  expired (for purgation).

• Scheduled jobs no longer "stack" in the queue. If SHIELD goes
  to schedule a backup and an existing task is in-flight for the
  same job, an already-cancelled task is stored in the database,
  as a placeholder to the task that should have run.

• Storage Health Check Tasks no longer stack. SHIELD only allows
  one in-flight task for a given Cloud Storage System, at a time.

• The shield CLI now handles API endpoints with any number of
  trailing forward slash (/) characters.

Bug Fixes

• Web UI page dispatch logic now properly cancels all outstanding
  AJAX requests, to avoid a rather annoying lag/delay UX issue
  where pages would flip "back" to a previous node in the history,
  because a delayed AJAX request was still working away in the
  background.

• shield import now honors the paused setting on jobs that it
  creates on your behalf.

SHIELD v8.0.19

Improvements

• Updated postgres plugin to allow for optional flags to be supplied to pg_dump

SHIELD v8.0.18

Improvements

• syncing shield release version with bosh release

SHIELD v8.0.17

New Features

• The SHIELD Web UI now allows you to download the SHIELD CLI
  directly, for both MacOS (Darwin) and Linux. From now on,
  SHIELD releases will include the paired version of the CLI.

• We now support minutely backups, but only from the CLI.

• New shield op pry for decrypting and inspecting the contents
  of a SHIELD Vault Crypt.

• New shield cli command 'delete-tenant' which will delete a tenant and clean up it's underlying configs with a -r

Improvements

• SHIELD now cleans up the Vault when archives are marked as
  expired (for purgation).

• Scheduled jobs no longer "stack" in the queue. If SHIELD goes
  to schedule a backup and an existing task is in-flight for the
  same job, an already-cancelled task is stored in the database,
  as a placeholder to the task that should have run.

• Storage Health Check Tasks no longer stack. SHIELD only allows
  one in-flight task for a given Cloud Storage System, at a time.

• The shield CLI now handles API endpoints with any number of
  trailing forward slash (/) characters.

• Update --help page on import to reflect correct roles

Bug Fixes

• Web UI page dispatch logic now properly cancels all outstanding
  AJAX requests, to avoid a rather annoying lag/delay UX issue
  where pages would flip "back" to a previous node in the history,
  because a delayed AJAX request was still working away in the
  background.

• Updated go-s3 to help fix connection closing issue

SHIELD v8.0.16

New Features

• We now have a BackBlaze B2 storage plugin!

Improvements

• Ordinals are now optional in monthly schedule specs (via the web
  UI), allowing front-end users to type '3' or '3rd', per their
  strongly-held personal preference.

• The token field of the vault plugins is now marked as a
  password, so that autocompletion in the browser gets turned off.
  Otherwise, Chrome/FF keeps wanting to leak your Vault tokens to
  people.

• The data directory and web UI root configurations are now
  properly validated by the SHIELD core. If they do not exist,
  core startup is halted. That way, you find out sooner if you've
  misconfigured something. Wheee.

• shield import can now properly import fixed-key backup jobs.
  Just what the doctor ordered for BOSH and SHIELD backup and
  recovery.

Bug Fixes

• Errors with hourly schedules are now properly handled and give a
  readable error message to the front-end.

• The mysql plugin can now properly restore a single database.

• Some silly typos (some copy-pasta, some bad whitespace, some
  we-don't-know-what-we-were-thinking) have been fixed in SHIELD
  CLI --help output.