Skip to content

Commit

Permalink
Created using Colaboratory
Browse files Browse the repository at this point in the history
  • Loading branch information
@yzhou359
yzhou359 committed Nov 7, 2020
1 parent 26c332c commit 06c0762
Showing 1 changed file with 278 additions and 0 deletions.
278 changes: 278 additions & 0 deletions quick_demo.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "quick_demo.ipynb",
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyMBRCJTXl9FbLyzLznPmwHs",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/yzhou359/MakeItTalk/blob/main/quick_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GXaL7nU6TEsV"
},
"source": [
"# MakeItTalk Quick Demo"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2owgbZ22TQmz"
},
"source": [
"## Preparations\n",
"- Check GPU"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yB-ixde4R3nO",
"outputId": "1160444f-2065-4897-c5a5-b647e21e0a05",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi\n",
"import subprocess\n",
"print(subprocess.getoutput('nvidia-smi'))"
],
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"text": [
"Sat Nov 7 01:23:47 2020 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"|===============================+======================+======================|\n",
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 40C P8 9W / 70W | 0MiB / 15079MiB | 0% Default |\n",
"+-------------------------------+----------------------+----------------------+\n",
" \n",
"+-----------------------------------------------------------------------------+\n",
"| Processes: GPU Memory |\n",
"| GPU PID Type Process name Usage |\n",
"|=============================================================================|\n",
"| No running processes found |\n",
"+-----------------------------------------------------------------------------+\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o31a6SpeTXDM"
},
"source": [
"- Check ffmpeg"
]
},
{
"cell_type": "code",
"metadata": {
"id": "u4EcdzstSB71",
"outputId": "533c51d0-bef5-4d9b-8759-240c46dba3fd",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"print(subprocess.getoutput('ffmpeg'))"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers\n",
" built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)\n",
" configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared\n",
" libavutil 55. 78.100 / 55. 78.100\n",
" libavcodec 57.107.100 / 57.107.100\n",
" libavformat 57. 83.100 / 57. 83.100\n",
" libavdevice 57. 10.100 / 57. 10.100\n",
" libavfilter 6.107.100 / 6.107.100\n",
" libavresample 3. 7. 0 / 3. 7. 0\n",
" libswscale 4. 8.100 / 4. 8.100\n",
" libswresample 2. 9.100 / 2. 9.100\n",
" libpostproc 54. 7.100 / 54. 7.100\n",
"Hyper fast Audio and Video encoder\n",
"usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...\n",
"\n",
"Use -h to get full help or, even better, run 'man ffmpeg'\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "taPSDYiSTcM_"
},
"source": [
"- Install Github https://github.com/yzhou359/MakeItTalk"
]
},
{
"cell_type": "code",
"metadata": {
"id": "4G0XLqo4SofV",
"outputId": "196b2f4e-5650-4638-f129-fdf6563e440e",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"!git clone https://github.com/yzhou359/MakeItTalk"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": [
"fatal: destination path 'MakeItTalk' already exists and is not an empty directory.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AByGGO5fd14P"
},
"source": [
"- Download pretrained models"
]
},
{
"cell_type": "code",
"metadata": {
"id": "SU4abC3iTmXA",
"outputId": "ae550554-83e9-49a6-ee11-43fb654ce341",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 130
}
},
"source": [
"!mkdir MakeItTalk/examples/dump\n",
"!mkdir MakeItTalk/examples/ckpt\n",
"!pip install gdown\n",
"!gdown -O MakeItTalk/examples/ckpt/ckpt_autovc.pth https://drive.google.com/uc?id=1ZiwPp_h62LtjU0DwpelLUoodKPR85K7x\n",
"!gdown -O MakeItTalk/examples/ckpt/ckpt_content_branch.pth https://drive.google.com/uc?id=1r3bfEvTVl6pCNw5xwUhEglwDHjWtAqQp\n",
"!gdown -O MakeItTalk/examples/ckpt/ckpt_speaker_branch.pth https://drive.google.com/uc?id=1rV0jkyDqPW-aDJcj7xSO6Zt1zSXqn1mu\n",
"!gdown -O MakeItTalk/examples/ckpt/ckpt_116_i2i_comb.pth https://drive.google.com/uc?id=1i2LJXKp-yWKIEEgJ7C6cE3_2NirfY_0a"
],
"execution_count": 25,
"outputs": [
{
"output_type": "error",
"ename": "SyntaxError",
"evalue": "ignored",
"traceback": [
"\u001b[0;36m File \u001b[0;32m\"<ipython-input-25-02dcf59fc139>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m - Download pretrained models\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "37JeD3ZZdI-a"
},
"source": [
"- prepare your images/audios (or you can use the existing ones)\n",
" - An image to animate: upload to `MakeItTalk/examples` folder\n",
" - An audio (hopefully no noise) to talk: upload to `MakeItTalk/examples` folder as well\n",
"\n",
"## Step 0: import necessary packages"
]
},
{
"cell_type": "code",
"metadata": {
"id": "olj6VcfiTrd_",
"outputId": "1c849c70-0256-4ae6-e1aa-b068e42cc535",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 368
}
},
"source": [
"import sys\n",
"sys.path.append('thirdparty/AdaptiveWingLoss')\n",
"import os, glob\n",
"import numpy as np\n",
"import cv2\n",
"import argparse\n",
"from src.approaches.train_image_translation import Image_translation_block\n",
"import torch\n",
"import pickle\n",
"import face_alignment\n",
"from thirdparty.autovc.AutoVC_mel_Convertor_retrain_version import AutoVC_mel_Convertor\n",
"import shutil\n",
"import time\n",
"import util.utils as util\n",
"from scipy.signal import savgol_filter\n",
"from src.approaches.train_audio2landmark import Audio2landmark_model"
],
"execution_count": 24,
"outputs": [
{
"output_type": "error",
"ename": "ModuleNotFoundError",
"evalue": "ignored",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-24-e1c2c1a96271>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mcv2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0margparse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0msrc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapproaches\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_image_translation\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mImage_translation_block\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 8\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mpickle\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'src'",
"",
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0;32m\nNOTE: If your import is failing due to a missing package, you can\nmanually install dependencies using either !pip or !apt.\n\nTo view examples of installing some common dependencies, click the\n\"Open Examples\" button below.\n\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "58s-c9H8dWPW"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}

0 comments on commit 06c0762

Please sign in to comment.