Skip to content

Commit

Permalink
[firebase_ml_custom] New firebase_ml_custom plugin to download custom…
Browse files Browse the repository at this point in the history
… models from Firebase (firebase#3122)

* [firebase_ml_custom] Add support for using custom ML models.
  • Loading branch information
PolinaGo authored Aug 7, 2020
1 parent eae1373 commit 974d634
Show file tree
Hide file tree
Showing 98 changed files with 4,294 additions and 0 deletions.
7 changes: 7 additions & 0 deletions packages/firebase_ml_custom/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.DS_Store
.dart_tool/

.packages
.pub/

build/
10 changes: 10 additions & 0 deletions packages/firebase_ml_custom/.metadata
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# This file tracks properties of this Flutter project.
# Used by Flutter tool to assess capabilities and perform upgrades etc.
#
# This file should be version controlled and should not be manually edited.

version:
revision: b041144f833e05cf463b8887fa12efdec9493488
channel: stable

project_type: plugin
3 changes: 3 additions & 0 deletions packages/firebase_ml_custom/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## 0.0.1

* TODO: Describe initial release.
27 changes: 27 additions & 0 deletions packages/firebase_ml_custom/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Copyright 2020 The Chromium Authors. All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
112 changes: 112 additions & 0 deletions packages/firebase_ml_custom/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Firebase Machine Learning Custom

[![pub package](https://img.shields.io/pub/v/firebase_ml_custom.svg)](https://pub.dartlang.org/packages/firebase_ml_custom)

A Flutter plugin to use the [Firebase ML Custom Models API](https://firebase.google.com/docs/ml/use-custom-models).

For Flutter plugins for other Firebase products, see [README.md](https://github.com/FirebaseExtended/flutterfire/blob/master/README.md).

## Usage

To use this plugin, add `firebase_ml_custom` as a [dependency in your pubspec.yaml file](https://flutter.io/platform-plugins/). You must also configure Firebase for each platform project: Android and iOS (see the example folder or https://codelabs.developers.google.com/codelabs/flutter-firebase/#4 for step by step details).

### Android

In order to use methods referencing those of [Firebase Model Manager](https://firebase.google.com/docs/reference/android/com/google/firebase/ml/common/modeldownload/FirebaseModelManager) minimum SDK version required is 24.
Otherwise minimum SDK version is 21.
This can be specified in your app-level `build.gradle` file.

### iOS

A minimum deployment target of 9.0 is required. You can add the line `platform :ios, '9.0'` in your iOS project `Podfile`.

You may also need to update your app's deployment target to 9.0 using Xcode. Otherwise, you may see
compilation errors.

## Using Firebase Model Manager

### 1. Create a `FirebaseCustomRemoteModel`.

Create a `FirebaseCustomRemoteModel` object.
You should already have a model in your Firebase console available for download. Use the name that you gave your model in the Firebase console.

```dart
FirebaseCustomRemoteModel remoteModel = FirebaseCustomRemoteModel('myModelName');
```

### 2. Create a `FirebaseModelDownloadConditions`.

Create a `FirebaseModelDownloadConditions` object.
Specify optional platform-specific conditions for the model download.

```dart
FirebaseModelDownloadConditions conditions =
FirebaseModelDownloadConditions(
androidRequireWifi: true,
androidRequireDeviceIdle: true,
androidRequiredCharging: true,
iosAllowCellularAccess: false,
iosAllowBackgroundDownloading: true);
```
All of these parameters except `iosAllowCellularAccess` default to `false` if not specified. `iosAllowCellularAccess` defaults to `true`.
Each platform looks only at its platform-specific parameters and ignores the rest.

### 3. Create an instance of `FirebaseModelManager`.

Create a `FirebaseModelManager` object corresponding to the default `FirebaseApp` instance.
```dart
FirebaseModelManager modelManager = FirebaseModelManager.instance;
```

### 4. Call `download()` with `FirebaseCustomRemoteModel` and `FirebaseModelDownloadConditions`.

Initiate the download of a remote model if the download hasn't begun.
If the model's download is already in progress, the current download task will continue executing.
If the model is already downloaded to the device, and there is no update, the call will immediately succeed.
If the model is already downloaded to the device, and there is update, a download for the updated version will be attempted.
```dart
await modelManager.download(remoteModel, conditions);
```

### 5. Call `isModelDownloaded()` with `FirebaseCustomRemoteModel`.

Return whether the given remote model is currently downloaded.
```dart
if (await modelManager.isModelDownloaded(model) == true) ) {
// do something with this model
} else {
// fall back on a locally-bundled model or do something else
}
```

You can also check if download was successfully completed by surrounding download method with `try` and `catch`.

### 5. Call `getLatestModelFile()` with `FirebaseCustomRemoteModel`.

Return the `File` containing the latest model for the remote model name. This will fail if the model is not yet downloaded on the device or valid custom remote model is not provided.

```dart
File modelFile = await modelManager.getLatestModelFile(model);
```

You can feed this file directly into an interpreter or preprocess it, depending on the interpreter of your choice.

Possible Flutter TF Lite interpreters:
- [tflite](https://pub.dev/packages/tflite)
- [tflite_flutter](https://pub.dev/packages/tflite_flutter)

Google does not recommend usage of any specific interpreter and leaves it up to the user to decide.

## Getting Started

See the `example` directory for a complete sample app using Firebase Machine Learning Custom.

## Issues and feedback

Please file Flutterfire specific issues, bugs, or feature requests in our [issue tracker](https://github.com/FirebaseExtended/flutterfire/issues/new).

Plugin issues that are not specific to Flutterfire can be filed in the [Flutter issue tracker](https://github.com/flutter/flutter/issues/new).

To contribute a change to this plugin,
please review our [contribution guide](https://github.com/FirebaseExtended/flutterfire/blob/master/CONTRIBUTING.md),
and send a [pull request](https://github.com/FirebaseExtended/flutterfire/pulls).
8 changes: 8 additions & 0 deletions packages/firebase_ml_custom/android/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.iml
.gradle
/local.properties
/.idea/workspace.xml
/.idea/libraries
.DS_Store
/build
/captures
67 changes: 67 additions & 0 deletions packages/firebase_ml_custom/android/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
group 'io.flutter.plugins.firebasemlcustom'
version '1.0'

buildscript {
repositories {
google()
jcenter()
}

dependencies {
classpath 'com.android.tools.build:gradle:3.5.0'
}
}

rootProject.allprojects {
repositories {
google()
jcenter()
}
}

apply plugin: 'com.android.library'

android {
compileSdkVersion 28

defaultConfig {
minSdkVersion 16
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
lintOptions {
disable 'InvalidPackage'
}
dependencies {
api 'com.google.firebase:firebase-ml-model-interpreter:22.0.3'
implementation 'androidx.annotation:annotation:1.0.0'
implementation 'androidx.exifinterface:exifinterface:1.0.0'
}
}

// TODO: Remove this hack once androidx.lifecycle is included on stable. https://github.com/flutter/flutter/issues/42348
afterEvaluate {
def containsEmbeddingDependencies = false
for (def configuration : configurations.all) {
for (def dependency : configuration.dependencies) {
if (dependency.group == 'io.flutter' &&
dependency.name.startsWith('flutter_embedding') &&
dependency.isTransitive())
{
containsEmbeddingDependencies = true
break
}
}
}
if (!containsEmbeddingDependencies) {
android {
dependencies {
def lifecycle_version = "1.1.1"
implementation "android.arch.lifecycle:runtime:$lifecycle_version"
implementation "android.arch.lifecycle:common:$lifecycle_version"
implementation "android.arch.lifecycle:common-java8:$lifecycle_version"
}
}
}
}

apply from: file("./user-agent.gradle")
4 changes: 4 additions & 0 deletions packages/firebase_ml_custom/android/gradle.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
org.gradle.jvmargs=-Xmx1536M
android.enableR8=true
android.useAndroidX=true
android.enableJetifier=true
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.2-all.zip
1 change: 1 addition & 0 deletions packages/firebase_ml_custom/android/settings.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
rootProject.name = 'firebase_ml_custom'
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="io.flutter.plugins.firebasemlcustom">
<application>
<service android:name="com.google.firebase.components.ComponentDiscoveryService">
<meta-data android:name="com.google.firebase.components:io.flutter.plugins.firebasemlcustom.FlutterFirebaseAppRegistrar"
android:value="com.google.firebase.components.ComponentRegistrar" />
</service>
</application>
</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.package io.flutter.plugins.firebaseml;

package io.flutter.plugins.firebasemlcustom;

import android.os.Build;
import androidx.annotation.NonNull;
import androidx.annotation.RequiresApi;
import io.flutter.embedding.engine.plugins.FlutterPlugin;
import io.flutter.plugin.common.MethodCall;
import io.flutter.plugin.common.MethodChannel;
import io.flutter.plugin.common.MethodChannel.MethodCallHandler;
import io.flutter.plugin.common.MethodChannel.Result;
import io.flutter.plugin.common.PluginRegistry.Registrar;

/** A flutter plugin for accessing the FirebaseML API for custom models. */
public class FirebaseMLCustomPlugin implements FlutterPlugin, MethodCallHandler {

private static final String CHANNEL_NAME = "plugins.flutter.io/firebase_ml_custom";

private MethodChannel channel;

/**
* Registers a plugin with the v1 embedding api {@code io.flutter.plugin.common}.
*
* <p>Calling this will register the plugin with the passed registrar. However, plugins
* initialized this way won't react to changes in activity or context.
*
* @param registrar connects this plugin's {@link
* io.flutter.plugin.common.MethodChannel.MethodCallHandler} to its {@link
* io.flutter.plugin.common.BinaryMessenger}.
*/
public static void registerWith(Registrar registrar) {
final MethodChannel channel = new MethodChannel(registrar.messenger(), CHANNEL_NAME);
channel.setMethodCallHandler(new FirebaseMLCustomPlugin());
}

@Override
public void onAttachedToEngine(@NonNull FlutterPluginBinding flutterPluginBinding) {
channel = new MethodChannel(flutterPluginBinding.getBinaryMessenger(), CHANNEL_NAME);
channel.setMethodCallHandler(this);
}

@RequiresApi(api = Build.VERSION_CODES.N)
@Override
public void onMethodCall(@NonNull MethodCall call, @NonNull final Result result) {
if (call.method.split("#")[0].equals("FirebaseModelManager")) {
ModelManager.handleModelManager(call, result);
} else {
result.notImplemented();
}
}

@Override
public void onDetachedFromEngine(@NonNull FlutterPluginBinding binding) {
channel.setMethodCallHandler(null);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

package io.flutter.plugins.firebasemlcustom;

import androidx.annotation.Keep;
import com.google.firebase.components.Component;
import com.google.firebase.components.ComponentRegistrar;
import com.google.firebase.platforminfo.LibraryVersionComponent;
import java.util.Collections;
import java.util.List;

@Keep
public class FlutterFirebaseAppRegistrar implements ComponentRegistrar {
@Override
public List<Component<?>> getComponents() {
return Collections.<Component<?>>singletonList(
LibraryVersionComponent.create(BuildConfig.LIBRARY_NAME, BuildConfig.LIBRARY_VERSION));
}
}
Loading

0 comments on commit 974d634

Please sign in to comment.