Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Howto use Mediapipe to create MLKit object detection model #5700

Open
cjohn001 opened this issue Oct 26, 2024 · 0 comments
Open

Howto use Mediapipe to create MLKit object detection model #5700

cjohn001 opened this issue Oct 26, 2024 · 0 comments
Assignees
Labels
os:linux-non-arm Issues on linux distributions which run on x86-64 architecture. DOES NOT include ARM devices. platform:python MediaPipe Python issues task:object detection Issues related to Object detection: Track and label objects in images and video. type:modelmaker Issues related to creation of custom on-device ML solutions type:support General questions

Comments

@cjohn001
Copy link

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Ubuntu 24.4

Python Version

3.11

MediaPipe Model Maker version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

Object detection

Describe the actual behavior

Cannot execute model with MLKIT

Describe the expected behaviour

can execute model with MLKIT

Standalone code/steps you may have used to try to get what you need

spec = object_detector.SupportedModels.MOBILENET_V2_I320
hparams = object_detector.HParams(export_dir='exported_model')
options = object_detector.ObjectDetectorOptions(
    supported_model=spec,
    hparams=hparams
)
model = object_detector.ObjectDetector.create(
    train_data=train_data,
    validation_data=validation_data,
    options=options)
model.export_model()

Other info / Complete Logs

Hello, I am currently trying to create a custom object detection model with Mediapipe model maker in order to use it with MLKit an Android and iOS devices. However, it seems the output shape of the generated model seems to not match what MLKit is expecting. I get bouding boxes
but when enabling classification  

customObjectDetectionOptions.shouldEnableClassification = true

I see the following error message:

failed to detect object with error Pipeline failed to fully start:
CalculatorGraph::Run() failed: 
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1)..

Is there a way to adopt the models output shapes in order to get them to work with mlkit?
@cjohn001 cjohn001 added the type:modelmaker Issues related to creation of custom on-device ML solutions label Oct 26, 2024
@google-ml-butler google-ml-butler bot added the type:support General questions label Oct 26, 2024
@kuaashish kuaashish added os:linux-non-arm Issues on linux distributions which run on x86-64 architecture. DOES NOT include ARM devices. python Pull requests that update Python code task:object detection Issues related to Object detection: Track and label objects in images and video. labels Oct 28, 2024
@kuaashish kuaashish assigned kuaashish and unassigned ayushgdev Oct 28, 2024
@kuaashish kuaashish added platform:python MediaPipe Python issues and removed python Pull requests that update Python code labels Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
os:linux-non-arm Issues on linux distributions which run on x86-64 architecture. DOES NOT include ARM devices. platform:python MediaPipe Python issues task:object detection Issues related to Object detection: Track and label objects in images and video. type:modelmaker Issues related to creation of custom on-device ML solutions type:support General questions
Projects
None yet
Development

No branches or pull requests

3 participants