Async tenant init OKAPI-875 (#1053)

Co-authored-by: julianladisch <[email protected]>

.gitignore

@@ -10,6 +10,7 @@ nbproject/
 .settings/
 .classpath
 .idea
+*.iml
 okapi.log
 */bin/
 x

doc/guide.md

@@ -48,7 +48,6 @@ managing and running microservices.
     * [Instrumentation](#instrumentation)
 * [Module Reference](#module-reference)
     * [Life cycle of a module](#life-cycle-of-a-module)
-    * [Tenant Interface](#tenant-interface)
     * [HTTP](#http)
 
 ## Introduction
@@ -2972,7 +2971,7 @@ kind of housekeeping it needs.
 
 For the [specifics](#web-service), see under
 `.../okapi/okapi-core/src/main/raml/raml-util` the files
-`ramls/tenant.raml` and `schemas/moduleInfo.schema`.  The
+`ramls/tenant.raml` and `schemas/tenantAttributes.schema`.  The
 okapi-test-module has a very trivial implementation of this, and the
 moduleTest shows a module Descriptor that defines this interface.
 
@@ -2999,10 +2998,7 @@ then insert those it received in the request. That way it will clean
 up permissions that may have been introduced in some older version of
 the module, and are no longer used.
 
-For the [specifics](#web-service), see under
-`.../okapi/okapi-core/src/main/raml/raml-util` the files
-`ramls/tenant.raml` and `schemas/moduleInfo.schema`.  The
-okapi-test-header-module has a very trivial implementation of this,
+The okapi-test-header-module has a very trivial implementation of this,
 and the moduleTest shows a module Descriptor that defines this
 interface.
 
@@ -3142,9 +3138,23 @@ during the operation.
 
 Add 'loadSample=true` to parameters, to load sample data as well.
 
+If the module supports version 2 of the `_tenant` interface it should return status 201 with
+a
+
+    Location: /_/tenant/6da99bac-457b-499f-89a4-34f4da8e9be8
+
+header. This signals that the tenant job has started. Use a GET request on the
+Location path returned to poll for the completion of the tenant job with the
+
+    curl -H "X-Okapi-url: http://localhost:8081" -H "X-Okapi-Tenant: testlib" \
+      -H "Content-Type: application/json" -H "Accept: */*" \
+      http://localhost:8081/_/tenant/6da99bac-457b-499f-89a4-34f4da8e9be8
+
+(substitute above path with the Location returned)
+
 #### Upgrading
 
-When a module gets upgraded to a new version, it happens separately
+When a module is upgraded to a new version, it happens separately
 for each tenant.  Some tenants may not wish to upgrade in the middle
 of a busy season, others may want to have everything in the latest
 version. The process starts by Okapi deploying the new version of the
@@ -3157,29 +3167,31 @@ request with path `/_/tenant` if version 1.0 or later of interface
 `_tenant` is provided. With the POST request, a JSON object is passed:
 member `module_from` being the module ID that we are upgrading 'from'
 and member `module_to` being the module ID that we are upgrading
-'to'. Note that thqe Module Descriptor of the target module
+'to'. Note that the Module Descriptor of the target module
 (module_to) is being used for the call.
 
-Upgrading large amounts of data to a newer schema can be slow. We are
-thinking about a way to make it happen asynchronously, but that is not
-even designed yet.  (TODO).
-
 We are using semantic versioning, see [Versioning and
 Dependencies](#versioning-and-dependencies)
 
 #### Disabling
 
-When a module is disabled for a tenant, Okapi makes a POST request
-with path `/_/tenant/disable` if version 1.1 and later of interface
-`_tenant` is provided. With the POST request a JSON object is passed:
-member `module_from` being the module ID that is being disabled.
+For tenant interface 1.1/1.2, when a module is disabled for a tenant, Okapi
+makes a POST request with path `/_/tenant/disable`. The request body is a
+JSON object where property `module_from` is the module ID that is being disabled.
+
+For tenant interface 2.0, when a module is disabled, Okapi
+makes a POST request with path `/_/tenant/` with `module_from` property
+being the module that is disabled and `module_to` omitted.
 
 #### Purge
 
-When a module is purged for a tenant, it disables the tenant for the
-module but also removes persistent content. A module may implement
-this by providing `_tenant` interface 1.0 and later with a DELETE
-method.
+Purge is like disable, but purges persistent content too for a module.
+
+For tenant interfaces 1.1 and 1.2, this is performed by a DELETE request
+to the module.
+
+For tenant interface 2.0, this is performed exactly like disable, but with
+`purge` property being `true` of the JSON content posted.
 
 #### Tenant Parameters
 
@@ -3188,29 +3200,51 @@ etc.  also load sets of reference data. This can be controlled by
 supplying tenant parameters. These are properties (key-value pairs)
 that are passed to the module when enabled or upgraded. Passing those
 are only performed when tenantParameters is specified for install and
-when the tenant interface is version 1.2.
+when the tenant interface is version 1.2 and later.
 
 In FOLIO two such parameters are widely recognized:
 
  * `loadReference` with value `true` loads reference data.
  * `loadSample` with value `true` loads sample data.
 
-### Tenant Interface
+#### Tenant Interface definitions
 
-The full `_tenant` interface version 1.1/1.2 portion:
+A module supporting 1.1/1.2 of the `_tenant` interface should use this
+snippet in the module descriptor:
 
 ```
    "id" : "_tenant",
    "version" : "1.2",
    "interfaceType" : "system",
    "handlers" : [ {
-     "methods" : [ "POST", "DELETE" ],
-     "pathPattern" : "/_/tenant"
-    }, {
-     "methods" : [ "POST" ],
-     "pathPattern" : "/_/tenant/disable"
-    } ]
+       "methods" : [ "POST", "DELETE" ],
+       "pathPattern" : "/_/tenant"
+      }, {
+       "methods" : [ "POST" ],
+       "pathPattern" : "/_/tenant/disable"
+      }
+   ]
+```
+The corresponding RAML definition:
+[tenant.raml](https://github.com/folio-org/raml/blob/tenant_interface_1_2/ramls/tenant.raml)
+
+Snippet of version 2.0 of the `_tenant` interface:
+
+```
+   "id" : "_tenant",
+   "version" : "2.0",
+   "interfaceType" : "system",
+   "handlers" : [ {
+       "methods" : [ "POST" ],
+       "pathPattern" : "/_/tenant"
+      }, {
+       "methods" : [ "GET", "DELETE" ],
+       "pathPattern" : "/_/tenant/{id}"
+      }
+   ]
 ```
+The corresponding RAML definition:
+[tenant.raml](https://github.com/folio-org/raml/blob/raml1.0/ramls/tenant.raml)
 
 #### Closing down
 

okapi-core/src/main/java/org/folio/okapi/bean/InterfaceDescriptor.java

@@ -217,8 +217,8 @@ public String validate(Logger logger, String section, String mod) {
           prefix, version);
     }
 
-    if ("_tenant".equals(this.id) && !"1.0 1.1 1.2".contains(version)) {
-      logger.warn("{} is '{}'. Should be '1.0/1.1/1.2'", prefix, version);
+    if ("_tenant".equals(this.id) && !"1.0 1.1 1.2 2.0".contains(version)) {
+      logger.warn("{} is '{}'. Should be '1.0/1.1/1.2/2.0'", prefix, version);
     }
     if ("_tenantPermissions".equals(this.id) && !"1.0 1.1".contains(version)) {
       logger.warn("{} is '{}'. should be '1.0/1.1'", prefix, version);

okapi-core/src/main/java/org/folio/okapi/bean/ModuleInstance.java

@@ -14,7 +14,7 @@ public class ModuleInstance {
   private String authToken;
   private String userId;
   private String permissions;
-  private final String path; // The relative URI from the proxy request
+  private String path; // The relative URI from the proxy request
   private final HttpMethod method;
   private final boolean handler;  // is true if handler; false otherwise (filter)
   private boolean withRetry;
@@ -44,6 +44,14 @@ public ModuleInstance(ModuleDescriptor md, RoutingEntry re,
     this.withRetry = false;
   }
 
+  /**
+   * Substitute {id} in path.
+   * @param id identifier
+   */
+  public void substPathId(String id) {
+    path = path.replace("{id}", id);
+  }
+
   public ModuleDescriptor getModuleDescriptor() {
     return md;
   }

okapi-core/src/main/java/org/folio/okapi/managers/ProxyService.java

@@ -1193,6 +1193,40 @@ private Future<OkapiClient> authForSystemInterface(
         });
   }
 
+  private Future<OkapiClient> doCallSystemInterface2(
+      MultiMap headersIn, String tenantId, String authToken,
+      ModuleInstance inst, String modPerms, String request) {
+
+    Map<String, String> headers = sysReqHeaders(headersIn, tenantId, authToken, inst, modPerms);
+    headers.put(XOkapiHeaders.URL_TO, inst.getUrl());
+    logger.debug("syscall begin {} {}{}", inst.getMethod(), inst.getUrl(), inst.getPath());
+    OkapiClient cli = new OkapiClient(this.httpClient, inst.getUrl(), vertx, headers);
+    String reqId = inst.getPath().replaceFirst("^[/_]*([^/]+).*", "$1");
+    cli.newReqId(reqId); // "tenant" or "tenantpermissions"
+    cli.enableInfoLog();
+    if (inst.isWithRetry()) {
+      cli.setClosedRetry(40000);
+    }
+    final Timer.Sample sample = MetricsHelper.getTimerSample();
+    Promise<OkapiClient> promise = Promise.promise();
+    cli.request(inst.getMethod(), inst.getPath(), request, cres -> {
+      logger.debug("syscall return {} {}{}", inst.getMethod(), inst.getUrl(), inst.getPath());
+      if (cres.failed()) {
+        String msg = messages.getMessage("11101", inst.getMethod(),
+            inst.getModuleDescriptor().getId(), inst.getPath(), cres.cause().getMessage());
+        logger.warn(msg, cres.cause());
+        MetricsHelper.recordHttpClientError(tenantId, inst.getMethod().name(), inst.getPath());
+        promise.fail(new OkapiError(ErrorType.USER, msg));
+        return;
+      }
+      MetricsHelper.recordHttpClientResponse(sample, tenantId, cli.getStatusCode(),
+          inst.getMethod().name(), inst);
+      // Pass response headers - needed for unit test, if nothing else
+      promise.complete(cli);
+    });
+    return promise.future();
+  }
+
   /**
    * Actually make a request to a system interface, like _tenant. Assumes we are
    * operating as the correct tenant.
@@ -1201,45 +1235,21 @@ Future<OkapiClient> doCallSystemInterface(
       MultiMap headersIn, String tenantId, String authToken,
       ModuleInstance inst, String modPerms, String request) {
 
-    return discoveryManager.get(inst.getModuleDescriptor().getId()).compose(gres -> {
-      DeploymentDescriptor instance = null;
-      if (gres != null) {
-        instance = pickInstance(gres);
-      }
-      if (instance == null) {
-        return Future.failedFuture(new OkapiError(ErrorType.USER, messages.getMessage("11100",
-            inst.getModuleDescriptor().getId(), inst.getPath())));
-      }
-      String baseurl = instance.getUrl();
-      Map<String, String> headers = sysReqHeaders(headersIn, tenantId, authToken, inst, modPerms);
-      headers.put(XOkapiHeaders.URL_TO, baseurl);
-      logger.debug("syscall begin {} {}{}", inst.getMethod(), baseurl, inst.getPath());
-      OkapiClient cli = new OkapiClient(this.httpClient, baseurl, vertx, headers);
-      String reqId = inst.getPath().replaceFirst("^[/_]*([^/]+).*", "$1");
-      cli.newReqId(reqId); // "tenant" or "tenantpermissions"
-      cli.enableInfoLog();
-      if (inst.isWithRetry()) {
-        cli.setClosedRetry(40000);
-      }
-      final Timer.Sample sample = MetricsHelper.getTimerSample();
-      Promise<OkapiClient> promise = Promise.promise();
-      cli.request(inst.getMethod(), inst.getPath(), request, cres -> {
-        logger.debug("syscall return {} {}{}", inst.getMethod(), baseurl, inst.getPath());
-        if (cres.failed()) {
-          String msg = messages.getMessage("11101", inst.getMethod(),
-              inst.getModuleDescriptor().getId(), inst.getPath(), cres.cause().getMessage());
-          logger.warn(msg, cres.cause());
-          MetricsHelper.recordHttpClientError(tenantId, inst.getMethod().name(), inst.getPath());
-          promise.fail(new OkapiError(ErrorType.USER, msg));
-          return;
-        }
-        MetricsHelper.recordHttpClientResponse(sample, tenantId, cli.getStatusCode(),
-            inst.getMethod().name(), inst);
-        // Pass response headers - needed for unit test, if nothing else
-        promise.complete(cli);
-      });
-      return promise.future();
-    });
+    Future<Void> future = Future.succeededFuture();
+    if (inst.getUrl() == null) {
+      future = discoveryManager.get(inst.getModuleDescriptor().getId())
+          .compose(gres -> {
+            DeploymentDescriptor instance = pickInstance(gres);
+            if (instance == null) {
+              return Future.failedFuture(new OkapiError(ErrorType.USER, messages.getMessage("11100",
+                  inst.getModuleDescriptor().getId(), inst.getPath())));
+            }
+            inst.setUrl(instance.getUrl());
+            return Future.succeededFuture();
+          });
+    }
+    return future.compose(x -> doCallSystemInterface2(
+        headersIn, tenantId, authToken, inst, modPerms, request));
   }
 
   /**