MediaTek Backend on ExecuTorch

MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries.

Supported Chips

The examples provided in this repository are tested and supported on the following MediaTek chip:

• MediaTek Dimensity 9300 (D9300)

Build Instructions

Prerequisites

Before you begin, ensure you have the following prerequisites installed and configured:

1. Buck2 Build Tool

• Download Buck2: Obtain Buck2 from the official releases page.
• Add to PATH: Extract the downloaded file and add the directory to your system's $PATH environment variable.

  export PATH=<path_to_buck>:$PATH

2. Android NDK

• Download Android NDK: Acquire the Android NDK version 26.3.11579264 from the Android developer site.
• Set NDK Path: Ensure that the $ANDROID_NDK environment variable is set to the path where the NDK is located.

  export ANDROID_NDK=<path_to_android_ndk>

3. MediaTek ExecuTorch Libraries

Download NeuroPilot Express SDK from MediaTek's NeuroPilot portal:

• libneuronusdk_adapter.mtk.so: This universal SDK contains the implementation required for executing target-dependent code on the MediaTek chip.
• libneuron_buffer_allocator.so: This utility library is designed for allocating DMA buffers necessary for model inference.
• mtk_converter-8.8.0.dev20240723+public.d1467db9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl: This library preprocess the model into a MediaTek representation.
• mtk_neuron-8.2.2-py3-none-linux_x86_64.whl: This library converts the model to binaries.
• Copy NeuronAdapter.h under backends/mediatek/runtime/include/api/.

Setup

Follow the steps below to setup your build environment:

1. Setup ExecuTorch Environment: Refer to the Setting up ExecuTorch guide for detailed instructions on setting up the ExecuTorch environment.

2. Setup MediaTek Backend Environment

• Install the dependent libs. Ensure that you are inside backends/mediatek/ directory

  pip3 install -r requirements.txt

• Install the two .whl downloaded from NeuroPilot Portal

  pip3 install mtk_neuron-8.2.2-py3-none-linux_x86_64.whl
  pip3 install mtk_converter-8.8.0.dev20240723+public.d1467db9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

• Set evironment variables for building backend

  export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator>

Build

1. Navigate to scripts/ directory.

2. Build MediaTek Backend: Once the prerequisites are in place, run the mtk_build.sh script to start the build process, MediaTek backend will be built under cmake-android-out/backends/ as libneuron_backend.so

   ./mtk_build.sh

Run

1. Push MediaTek universal SDK and MediaTek backend to the device: push libneuronusdk_adapter.mtk.so and libneuron_backend.so to the phone and export it to the $LD_LIBRARY_PATH environment variable before executing ExecuTorch with MediaTek backend.

   export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH

Please refer to executorch/examples/mediatek/ for export and execution examples of various of models.