Migration Aides Second Release

[bcarld]

This PR radically changes the migration aids.
It has the following:

1. a single script that controls all aspects of the migration.

This script is self contained so that it can be copied onto a standalone server and used to manage a migration. It supports starting, checking status, and cleaning up a migration.
This script will create the deployment definition file and deploy it so that the user does not have to know anything about
deployment definitions.
However, if some customization of the deployment is needed, the deployment definition file is output so that it can be edited and deployed manually.

2. More fine grained migrations a supported

With these changes, migration of an entire user or a project group is supported. It is possible to specify multiple users or project groups on a single invocation. Or, multiple migrations can be done; either sequentially or at the same time.

When multiple users (or project groups) are given on a single invocation, each user (or project group) is migrated to completion before the next one in the list is started.

Note that you cannot mix project groups and users in a single migration invocation.

[bcarld]

Testing/Validation Notes

Migration Validation

Current validation of a migration is done by hand. There are ideas/plans to automate the testing.
Each migration is validated by using vision tools to examine the JSON objects. The same command is used in each terminal and is just copy/pasted between terminals.
The following items are validated

1. dataset counts are correct.
2. file counts are correct for each dataset.
3. object label counts are correct for the dataset.
4. object label counts are correct for each file in the dataset.

Random objects are selected and json content is inspected to be the same (this step should be automated to ensure every resource object is the same, but that will be done later).

Lastly, I exec into the Task Analysis container on the destination and examine dataset directory to ensure that the file count there matches the file count from the DB (e.g. ls -1 MOUNT_POINT/USER/datasets/DSID/files | wc -l).

Migration scenarios

Ensure project group migration works

This is a test of migrating a single project group. The target project group must have at least 1 dataset and 1 trained-model.

Ensure multiple project groups are migrated

This is a test of multiple project groups in a single migration operation. One of the target project groups has the same
characteristic requirements as the test for a single project group. The other project group should have different characteristics (in this case, the 2nd project group that I had access to only had 1 dataset).

Ensure user migration works

This is a test to ensure that all resources associated with a single user is migrated correctly. The user should have multiple
datasets and multiple trained models.

Ensure multiple users are migrated

This is a test to ensure that all resources associated with a single user is migrated correctly. On of the users should have
at least 1 project group and multiple datasets (at least 1 outside of the project group) and at least 1 trained model.

Ensure that whole server migration still works

This test is to ensure that the previous capability to migrate an entire server is still possible.

Results

All tests have been performed and all tests have run successfully.

Caveats

During testing in a customer environment the following issues were discovered and corrected.

1. Migration failed because MVI versions could not be validated

This problem occurred because there was no route from the test source server to the destination MVI API end point.
We could run open shift CLI commands, but we could not reach the destination MVI server API.

At the present time, the purpose of the version check is to ensure that we are not migrating from a higher version to a lower version. With the current support, migration only works from a standalone server to a MAS based cloud server. By definition that means the destination version will be higher.

If either for the mviUrl attributes are missing from the source or destination objects in the cluster config json file, the version validation will not be performed.