Use Mobile-FFmpeg with Unity

4 minute read

Introduction

It became necessary to process video files and audio files within the application being developed in Unity, so we decided to use Mobile-FFmpeg. A memo of what I did before I introduced it and became able to use it.

What is Mobile-FFmpeg?

FFmpeg’s Android / iOS open source library. The content consists of FFmpeg and FFprobe implementations and external libraries. Updates are also being actively carried out.

The feature is that it is a simple API that is called by specifying an optional command character string in exactly the same way as the command line version of FFmpeg.
https://github.com/tanersener/mobile-ffmpeg

Caution

Since MobileFFmpeg has more than 40 externally dependent libraries, it is a good idea to first determine where to use MobileFFmpeg in your app and clarify the required functions and required libraries.

This article is written for Mobile-FFmpeg Version 4.4, Unity 2018 and above.

Pre-built binaries

You can build it yourself, but we also have pre-built binaries, so I used this one this time.
Eight kinds of variations are prepared for the binary depending on the external library included.

Packages

min: Minimal package
min-gpl: min with the GPL license library added
https: TLS related libraries added
https-gpl: https with the GPL license library added
ʻAudio`: Added audio-related libraries
video: Added video-related libraries
full: Full set (not including GPL license)
full-gpl: Full set (including GPL license)

Select the package from the above according to your purpose and download the file (https://github.com/tanersener/mobile-ffmpeg/releases)
Download aar file for library for Android
-(File name): mobile-ffmpeg-[Package]-[Version] .aar
Download the xcframework.zip file and unzip the library for iOS
-(File name): mobile-ffmpeg-[Package]-[Version]-ios-xcframework.zip

Check the following for details on each package.
2.1 Packages

Import to Unity

downloaded
.aar file
/Assets/Plugins/Android
Unzip the zip and open the framework files
/Assets/Plugins/iOS
set on.

Call from C # on the Unity side

Implement a wrapper for calling Plugin modules for Android and iOS respectively. Basically, it’s OK if you can call two APIs, Execute and Cancel.

I referred to the thread of Isshue # 258 on GitHub of MobileFFmpeg.
https://github.com/tanersener/mobile-ffmpeg/issues/258#issuecomment-663913978

code

There are three source files. Place the .mm file in ʻAssets / Plugins / iOS`.

`FFmpegWrapper.cs`

using UnityEngine;

public class FFmpegWrapper
{
    private static int Execute(string command)
    {
#if UNITY_ANDROID
        using (AndroidJavaClass configClass = new AndroidJavaClass("com.arthenica.mobileffmpeg.Config"))
        {
            AndroidJavaObject paramVal = new AndroidJavaClass("com.arthenica.mobileffmpeg.Signal").GetStatic<AndroidJavaObject>("SIGXCPU");
            configClass.CallStatic("ignoreSignal", new object[] { paramVal });

            using (AndroidJavaClass ffmpeg = new AndroidJavaClass("com.arthenica.mobileffmpeg.FFmpeg"))
            {
                int code = ffmpeg.CallStatic<int>("execute", new object[] { command });
                return code;
            }
        }
#elif UNITY_IOS
        return MobileFFmpegIOS.Execute(command);
#else
        return 0;
#endif
    }

    private static int Cancel()
    {
#if UNITY_ANDROID
        using (AndroidJavaClass configClass = new AndroidJavaClass("com.arthenica.mobileffmpeg.Config"))
        {
            using (AndroidJavaClass ffmpeg = new AndroidJavaClass("com.arthenica.mobileffmpeg.FFmpeg"))
            {
                int code = ffmpeg.CallStatic<int>("cancel");
                return code;
            }
        }
#elif UNITY_IOS
        return MobileFFmpegIOS.Cancel();
#else
        return 0;
#endif
    }

`MobileFFmpegIOS.cs`

using UnityEngine;
using System.Collections;
using System.Runtime.InteropServices;


public class MobileFFmpegIOS : MonoBehaviour
{
#if UNITY_IOS
    [DllImport("__Internal")]
    private static extern int _execute(string command);

    [DllImport("__Internal")]
    private static extern void _cancel();
#endif

    /**
     * Synchronously executes FFmpeg command provided. Space character is used to split command
     * into arguments.
     *
     * @param command FFmpeg command
     * @return zero on successful execution, 255 on user cancel and non-zero on error
     */
    public static int Execute(string command)
    {
        int result = -1;
#if UNITY_IOS
        if (Application.platform == RuntimePlatform.IPhonePlayer)
        {
            result = _execute(command);
        }
#endif

        return result;
    }

    /**
     * Cancels an ongoing operation.
     *
     * This function does not wait for termination to complete and returns immediately.
     */
    public static void Cancel()
    {
#if UNITY_IOS
        if (Application.platform == RuntimePlatform.IPhonePlayer)
        {
            _cancel();
        }
#endif
    }
}

The following is placed in ʻAssets / Plugins / iOS`

`MobileFFmpeg.mm`


//In unity, You'd place this file in your "Assets>plugins>ios" folder
//Objective-C Code
#import <mobileffmpeg/MobileFFmpeg.h>

extern "C"
{
    /**
    * Synchronously executes FFmpeg command provided. Space character is used to split command
    * into arguments.
    *
    * @param command FFmpeg command
    * @return zero on successful execution, 255 on user cancel and non-zero on error
    */
    int _execute(const char* command)
    {
        return [MobileFFmpeg execute: @(command)];
    }
    
    /**
    * Cancels an ongoing operation.
    *
    * This function does not wait for termination to complete and returns immediately.
    */
    void _cancel()
    {
        [MobileFFmpeg cancel];
    }
}

Call from Unity

Call the option command by passing it as a character string to the Execute parameter (note that “ffmpeg” is unnecessary). If successful, 0 is returned.

`Example`

private void Test()
{
    var input = Application.persistentDataPath + "/input.mov";
    var output = Application.persistentDataPath + "/output.mp4";
    int rc = FFmpegWrapper.Execute(string.Format("-i {0} {1}", input, output));
    Debug.Log("Return Code is " + rc);
}

There is also an asynchronously executed FFmpeg.executeAsync function, but I haven’t tried it. I’m sorry. .. ..

Other

○ If you get a linker error in Xcode

** If the following libraries do not exist in [Link Binary with Libraries] **, add them.
bzip2, iconv, libuuid, zlib, AudioToolbox, VideoToolbox, AVFoundation
(libz.tbd, libbz2.tbd, libiconv.tbd)

It seems that it does not support Armv7, so ** Architectures ** was built exclusively for Arm64.

○ Run-time error

-Fails if a file with the same name as the Output file that does not have file write permission already exists
-The required library is determined according to the command, so it will fail if there is no module.

Reference article

Analyze and convert audio files using FFmpeg on Android
https://qiita.com/tarumzu/items/a4d15957a144f520f842
You can do it with FFmpeg!
https://qiita.com/cha84rakanal/items/e84fe4eb6fbe2ae13fd8
Convert video to GIF with FFmpeg
https://qiita.com/wMETAw/items/fdb754022aec1da88e6e

Twitter Facebook LinkedIn