kazoo_speech: ASR Billing Features and kazoo_asr Refactor (2600hz#6055)

This PR enhances the kazoo_speech application by augmenting the `kazoo_asr` abstraction as well as introduce new billing features. - Add `transcribe` field to the mailbox schema - Add and export callbacks to`gen_asr_provider` for content-types - Enhance the `kazoo_asr` behavior to handle default and accepted content types on behalf of the specific providers - Create an abstraction for ASR requests - Create account ledger entries on successful transcription as well as the impact the reseller ledgers. - Create `asr_flat_rate` module to handle flat rate billing of ASR requests. - Create quantifiers in `kz_services` for asr transcriptions. ``` "asr": { "google": { "rate": 1, "name": "Google ASR" }, "ispeech": { "rate": 1, "name": "ispeech ASR" } }, "plan": { .... "voicemails": { "mailbox": { "name": "Voicemail Box", "rate": 1.99, "cascade": true }, "transcription": { "cascade": true, "rate": 1, "name": "VMBox Transcription MRC" } }, .... ``` The only configuration change in this PR is adding the `transcribe` flag to the JSON schema. Two new callbacks have been added to `gen_asr_provider` to help facilitate a more generic approach to handling the preferred content-types for a provider as well as act as a gatekeeper and help identify if conversion is required or even currently supported for the submitted media. - preferred_content_type/0 - accepted_content_types/0 This callback is designed to return the ASR provider's preferred content-type for requests. This callback is designed to return the accepted content-types and assist in determining if the media payload will require conversion. The content-type callbacks have been added to `kazoo_asr` as well any `kapps_config` calls from the current providers in favor of keeping with the abstraction. Originally the ASR logic was hardcoded into the voicemail notify and save logic in `kvm_util`. I've removed that and replaced it with an asr_request. The `asr_request` module is designed to be a generic request type to handle creating and servicing ASR requests which are proxied with `kazoo_asr` to the configured provider.. Additionally it also handles the billing and services logic and introduces a `asr_flat_rate` module that can consult an ASR service plan item for rates. The long term vision is to create a primitive that can in the future could be augmented to handle multiple types of ASR requests apart from file conversion (i.e. streaming from an active call). I've added some placeholder fields in `#asr_req` record should metered billing be desired. `asr_flat_rate` is the only and default `billing_method` for an `asr_request`. It is patterned after jonny5 and aims to perform the following actions on an `asr_req`: - **authorize** *account and reseller have funds* - **debit** *ledger entries created for account and reseller if configured* Some ASR primitives have been gently introduced into `kz_services`: - `asr` getter methods have been added to `kzd_service_plans` and `kz_service_plans` - default quantifiers for transcribe enabled maiboxes in `services.hrl` - `kz_services_asr` module for interacting with the transcription usage ledgers
ronexvoip · Dec 3, 2019 · 86ba2d2 · 86ba2d2
1 parent b3a3657
commit 86ba2d2
Show file tree

Hide file tree

Showing 27 changed files with 1,258 additions and 77 deletions.
diff --git a/applications/crossbar/doc/ref/vmboxes.md b/applications/crossbar/doc/ref/vmboxes.md
@@ -37,6 +37,7 @@ Key | Description | Type | Default | Required | Support Level
 `skip_greeting` | Determines if the greeting should be skipped | `boolean()` | `false` | `false` | `supported`
 `skip_instructions` | Determines if the instructions after the greeting and prior to composing a message should be played | `boolean()` | `false` | `false` | `supported`
 `timezone` | The default timezone | `string(5..32)` |   | `false` | `supported`
+`transcribe` | Transcribe voicemail using ASR engine | `boolean()` | `false` | `false` | `supported`
 
 ### notify.callback
 

diff --git a/applications/crossbar/doc/voicemail.md b/applications/crossbar/doc/voicemail.md
@@ -48,6 +48,7 @@ Key | Description | Type | Default | Required | Support Level
 `skip_greeting` | Determines if the greeting should be skipped | `boolean()` | `false` | `false` | `supported`
 `skip_instructions` | Determines if the instructions after the greeting and prior to composing a message should be played | `boolean()` | `false` | `false` | `supported`
 `timezone` | The default timezone | `string(5..32)` |   | `false` | `supported`
+`transcribe` | Transcribe voicemail using ASR engine | `boolean()` | `false` | `false` | `supported`
 
 ### notify.callback
 

diff --git a/applications/crossbar/priv/api/swagger.json b/applications/crossbar/priv/api/swagger.json
@@ -289,6 +289,10 @@
                         {
                             "friendly_name": "Support",
                             "name": "support"
+                        },
+                        {
+                            "friendly_name": "ASR Transcriptions",
+                            "name": "asr-transcriptions"
                         }
                     ],
                     "description": "ledgers registered_ledgers",
@@ -32785,6 +32789,10 @@
                         {
                             "friendly_name": "Support",
                             "name": "support"
+                        },
+                        {
+                            "friendly_name": "ASR Transcriptions",
+                            "name": "asr-transcriptions"
                         }
                     ],
                     "description": "ledgers registered_ledgers",
@@ -36795,6 +36803,11 @@
                     "maxLength": 32,
                     "minLength": 5,
                     "type": "string"
+                },
+                "transcribe": {
+                    "default": false,
+                    "description": "Transcribe voicemail using ASR engine",
+                    "type": "boolean"
                 }
             },
             "required": [

diff --git a/applications/crossbar/priv/couchdb/schemas/account_config.ledgers.json b/applications/crossbar/priv/couchdb/schemas/account_config.ledgers.json
@@ -16,6 +16,10 @@
                 {
                     "friendly_name": "Support",
                     "name": "support"
+                },
+                {
+                    "friendly_name": "ASR Transcriptions",
+                    "name": "asr-transcriptions"
                 }
             ],
             "description": "ledgers registered_ledgers",

diff --git a/applications/crossbar/priv/couchdb/schemas/system_config.ledgers.json b/applications/crossbar/priv/couchdb/schemas/system_config.ledgers.json
@@ -16,6 +16,10 @@
                 {
                     "friendly_name": "Support",
                     "name": "support"
+                },
+                {
+                    "friendly_name": "ASR Transcriptions",
+                    "name": "asr-transcriptions"
                 }
             ],
             "description": "ledgers registered_ledgers",

diff --git a/applications/crossbar/priv/couchdb/schemas/vmboxes.json b/applications/crossbar/priv/couchdb/schemas/vmboxes.json
@@ -172,6 +172,12 @@
             "minLength": 5,
             "support_level": "supported",
             "type": "string"
+        },
+        "transcribe": {
+            "default": false,
+            "description": "Transcribe voicemail using ASR engine",
+            "support_level": "supported",
+            "type": "boolean"
         }
     },
     "required": [

diff --git a/applications/crossbar/priv/oas3/oas3-schemas.yml b/applications/crossbar/priv/oas3/oas3-schemas.yml
@@ -212,6 +212,8 @@
           'name': mobile_data
         - 'friendly_name': Support
           'name': support
+        - 'friendly_name': ASR Transcriptions
+          'name': asr-transcriptions
       'description': ledgers registered_ledgers
       'items':
         'type': object
@@ -11407,6 +11409,8 @@
           'name': mobile_data
         - 'friendly_name': Support
           'name': support
+        - 'friendly_name': ASR Transcriptions
+          'name': asr-transcriptions
       'description': ledgers registered_ledgers
       'items':
         'type': object
@@ -14613,6 +14617,11 @@
       'minLength': 5
       'type': string
       'x-support_level': supported
+    'transcribe':
+      'default': false
+      'description': Transcribe voicemail using ASR engine
+      'type': boolean
+      'x-support_level': supported
   'required':
     - mailbox
     - name

diff --git a/core/kazoo_documents/src/kzd_box_message.erl b/core/kazoo_documents/src/kzd_box_message.erl
@@ -25,6 +25,7 @@
 
         ,change_message_name/2, change_to_sip_field/3
 
+        ,length/1
         ,media_id/1, set_media_id/2, update_media_id/2
         ,metadata/1, metadata/2, set_metadata/2
         ,source_id/1, set_source_id/2
@@ -267,6 +268,10 @@ message_history(JObj) ->
 add_message_history(History, JObj) ->
     kz_json:set_value(?KEY_HISTORY, message_history(JObj) ++ [History], JObj).
 
+-spec length(doc()) -> integer().
+length(JObj) ->
+    kz_json:get_value(?KEY_META_LENGTH, JObj).
+
 -spec message_name(doc()) -> kz_term:api_binary().
 message_name(JObj) ->
     message_name(JObj, 'undefined').

diff --git a/core/kazoo_documents/src/kzd_service_plan.erl b/core/kazoo_documents/src/kzd_service_plan.erl
@@ -41,6 +41,9 @@
         ,ratedeck_name/2
         ,set_ratedeck_name/2
         ]).
+-export([asr/1
+        ,asr/2
+        ]).
 -export([applications/1
         ,applications/2
         ,set_applications/2
@@ -283,6 +286,19 @@ applications(JObj, Default) ->
 set_applications(JObj, Applications) ->
     kz_json:set_value(?APPLICATIONS, Applications, JObj).
 
+%%------------------------------------------------------------------------------
+%% @doc
+%% @end
+%%------------------------------------------------------------------------------
+-spec asr(doc()) -> kz_json:object().
+asr(JObj) ->
+    asr(JObj, kz_json:new()).
+
+-spec asr(doc(), Default) -> Default | kz_json:object().
+asr(JObj, Default) ->
+    kz_json:get_ne_json_value(<<"asr">>, JObj, Default).
+
+
 %%------------------------------------------------------------------------------
 %% @doc
 %% @end

diff --git a/core/kazoo_documents/src/kzd_vmboxes.erl.src b/core/kazoo_documents/src/kzd_vmboxes.erl.src
@@ -31,6 +31,7 @@
 -export([skip_greeting/1, skip_greeting/2, set_skip_greeting/2]).
 -export([skip_instructions/1, skip_instructions/2, set_skip_instructions/2]).
 -export([timezone/1, timezone/2, set_timezone/2]).
+-export([transcribe/1, transcribe/2, set_transcribe/2]).
 
 
 -include("kz_documents.hrl").
@@ -343,3 +344,15 @@ timezone(Doc, Default) ->
 -spec set_timezone(doc(), kz_term:ne_binary()) -> doc().
 set_timezone(Doc, Timezone) ->
     kz_json:set_value([<<"timezone">>], Timezone, Doc).
+
+-spec transcribe(doc()) -> boolean().
+transcribe(Doc) ->
+    transcribe(Doc, false).
+
+-spec transcribe(doc(), Default) -> boolean() | Default.
+transcribe(Doc, Default) ->
+    kz_json:get_boolean_value([<<"transcribe">>], Doc, Default).
+
+-spec set_transcribe(doc(), boolean()) -> doc().
+set_transcribe(Doc, Transcribe) ->
+    kz_json:set_value([<<"transcribe">>], Transcribe, Doc).
diff --git a/core/kazoo_services/src/kazoo_services_maintenance.erl b/core/kazoo_services/src/kazoo_services_maintenance.erl
@@ -215,6 +215,9 @@ js_quantify_map() ->
        "            break;"
        "        case 'vmbox':"
        "            emit(['voicemails', 'mailbox'], 1);"
+       "            if (doc.transcribe) {"
+       "                emit(['voicemails', 'transcription'], 1);"
+       "            }"
        "            break;"
        "        case 'faxbox':"
        "            emit(['faxes', 'mailbox'], 1);"

diff --git a/core/kazoo_services/src/kz_services_plan.erl b/core/kazoo_services/src/kz_services_plan.erl
@@ -24,6 +24,7 @@
 -export([ratedeck_id/1]).
 -export([ratedeck_name/1]).
 -export([applications/1]).
+-export([asr/1]).
 -export([limits/1]).
 -export([jobj/1
         ,set_jobj/2
@@ -173,6 +174,14 @@ ratedeck_name(Plan) ->
 applications(Plan) ->
     kzd_service_plan:applications(jobj(Plan)).
 
+%%------------------------------------------------------------------------------
+%% @doc
+%% @end
+%%------------------------------------------------------------------------------
+-spec asr(plan()) -> kz_json:object().
+asr(Plan) ->
+    kzd_service_plan:asr(jobj(Plan)).
+
 %%------------------------------------------------------------------------------
 %% @doc
 %% @end

diff --git a/core/kazoo_services/src/kz_services_quantities.erl b/core/kazoo_services/src/kz_services_quantities.erl
@@ -556,8 +556,21 @@ calculate_vmbox_updates(JObj, Updates) ->
     case kz_doc:type(JObj) =:= <<"vmbox">> of
         'false' -> Updates;
         'true' ->
-            Key = [<<"voicemails">>, <<"mailbox">>],
-            [{Key, 1} | Updates]
+            Keys = [<<"mailbox">>, <<"transcription">>],
+            Fun = calculate_vmbox_updates_foldl(JObj),
+            lists:foldl(Fun, Updates, Keys)
+    end.
+
+-spec calculate_vmbox_updates_foldl(kz_json:object()) -> fun((kz_term:ne_binary(), kz_term:proplist()) -> kz_term:proplist()).
+calculate_vmbox_updates_foldl(JObj) ->
+    fun(<<"transcription">> = Key, Updates) ->
+            case kz_json:get_boolean_value(<<"transcribe">>, JObj, 'false') of
+                'true' ->
+                    [{[<<"voicemails">>, Key], 1} | Updates];
+                _ -> Updates
+            end;
+       (Key, Updates) ->
+            [{[<<"voicemails">>, Key], 1} | Updates]
     end.
 
 -spec calculate_faxbox_updates(kz_json:object(), kz_term:proplist()) -> kz_term:proplist().

diff --git a/core/kazoo_services/src/modules/kz_services_asr.erl b/core/kazoo_services/src/modules/kz_services_asr.erl
@@ -0,0 +1,62 @@
+%%%-----------------------------------------------------------------------------
+%%% @copyright (C) 2012-2019, 2600Hz
+%%% @doc
+%%% This Source Code Form is subject to the terms of the Mozilla Public
+%%% License, v. 2.0. If a copy of the MPL was not distributed with this
+%%% file, You can obtain one at https://mozilla.org/MPL/2.0/.
+%%%
+%%% @end
+%%%-----------------------------------------------------------------------------
+-module(kz_services_asr).
+
+-export([fetch/1
+        ,flat_rate/1, flat_rate/2
+        ]).
+
+-include("services.hrl").
+
+-define(DEFAULT_FLAT_RATE, 0).
+
+%%------------------------------------------------------------------------------
+%% @doc
+%% @end
+%%------------------------------------------------------------------------------
+-spec fetch(kz_services:services() | kz_term:ne_binary()) -> kz_json:object().
+fetch(?NE_BINARY=AccountId) ->
+    FetchOptions = ['hydrate_plans'],
+    fetch(kz_services:fetch(AccountId, FetchOptions));
+fetch(Services) ->
+    ASRDict = kz_services_plans:foldl(fun fetch_foldl/3
+                                     ,dict:new()
+                                     ,kz_services:plans(Services)
+                                     ),
+    kz_json:from_list(dict:to_list(ASRDict)).
+
+%%------------------------------------------------------------------------------
+%% @doc
+%% @end
+%%------------------------------------------------------------------------------
+-spec fetch_foldl(kz_term:ne_binary(), kz_services_plans:plans_list(), dict:dict()) -> dict:dict().
+fetch_foldl(_BookkeeperHash, [], Providers) ->
+    Providers;
+fetch_foldl(_BookkeeperHash, PlansList, Providers) ->
+    Plan = kz_services_plans:merge(PlansList),
+    kz_json:foldl(fun(K, V, A) ->
+                          dict:store(K, V, A)
+                  end
+                 ,Providers
+                 ,kz_services_plan:asr(Plan)
+                 ).
+
+%%------------------------------------------------------------------------------
+%% @doc
+%% @end
+%%------------------------------------------------------------------------------
+-spec flat_rate(kz_term:ne_binary()) -> kz_currency:dollars().
+flat_rate(AccountId) ->
+    flat_rate(AccountId, kazoo_asr:default_provider()).
+
+-spec flat_rate(kz_term:ne_binary(), kz_term:ne_binary()) -> kz_currency:dollars().
+flat_rate(AccountId, Provider) ->
+    Items = fetch(AccountId),
+    kz_json:get_number_value([Provider, <<"rate">>], Items, ?DEFAULT_FLAT_RATE).
diff --git a/core/kazoo_services/src/services.hrl b/core/kazoo_services/src/services.hrl
@@ -99,7 +99,9 @@
                                ]
                               }
                              ,{<<"voicemails">>
-                              ,[<<"mailbox">>]
+                              ,[<<"mailbox">>
+                               ,<<"transcription">>
+                               ]
                               }
                              ,{<<"faxes">>
                               ,[<<"mailbox">>]

diff --git a/core/kazoo_speech/doc/README.md b/core/kazoo_speech/doc/README.md
@@ -14,3 +14,34 @@ Kazoo currently supports as an ASR engine:
 
 1. [iSpeech](http://www.ispeech.org/api/#automated-speech-recognition)
 2. [Google Cloud Speech API](https://cloud.google.com/speech/)
+
+### Service Plan Definitions
+1. The ASR block defines usage rates for the providers
+2. The transcription block defines rates for having transcribe enabled on a vmbox
+```
+"asr": {
+       "google": {
+           "rate": 1,
+           "name": "Google ASR"
+       },
+       "ispeech": {
+           "rate": 1,
+           "name": "ispeech ASR"
+       }
+   },
+   "plan": {
+   ....
+       "voicemails": {
+           "mailbox": {
+               "name": "Voicemail Box",
+               "rate": 1.99,
+               "cascade": true
+           },
+           "transcription": {
+               "cascade": true,
+               "rate": 1,
+               "name": "VMBox Transcription MRC"
+           }
+       },
+....
+```