Build ffmpeg with 10-bit x265 support for encoding on Windows

27 October 2017

So I was trying to create some 10-bit test HEVC content with ffmpeg and ran into the following error:

ffmpeg -i "input.mkv" -c:v libx265 -preset slow -crf 18 -pix_fmt yuv420p10le -c:a copy -y output_10bit.mkv


Incompatible pixel format 'yuv420p10le' for codec 'libx265', auto-selecting format 'yuv420p'

I discovered that by default - most ffmpeg builds include only 8-bit support for HEVC encoding with libx265.

Certainly, you can get builds that already enable 10 or even 12-bit support - but I’m usually a glutton for punishment so I thought I would see how hard it was to just build it manually.

Here are the exact steps I did to build both ffmpeg and libx265 with Visual Studio 2017 on Windows. Some of the steps will also detail some of the errors I ran into along the way - in case someone else runs into trouble, hopefully then it will show how to overcome any future hurdles.

Note: This guide will only build the 64-bit versions, but you should be able to adjust for 32-bit.

Build Tools

Build libx265 for 10-bit support

  1. Download the x265 source code

    Create the directory c:\third_party and cd from a command prompt to it. Run the command:

    hg clone
  2. Navigate to third_party\x265\build. Find the directory that matches the VC compiler you want to use. It is likely there will not be one matching the newest version of Visual Studio, so use the latest one available (In my case it was vc12-x86_64)

    Using a text editor, open make-solutions.bat. Change the generator command to match the version of VS you want to use. I’m using 2017, so my command is:

    cmake -G "Visual Studio 15 Win64" ..\..\source && cmake-gui ..\..\source

    Update 12/9/2017 - libx265 should now have a corresponding make-solutions.bat file for Visual Studio 2017. This step is no longer required

  3. Run the make-solutions.bat - during the processing, the CMake GUI will open with properties highlighted red.
    • Check the HIGH_BIT_DEPTH for 10-bit
    • Set the CMAKE_INSTALL_PREFIX to a directory the x265 lib, headers, and importantly the pkg-config .pc file will be installed to
    • Verify the NASM_EXECUTABLE property is pointing the location where you installed NASM

      alt text

  4. Press Configure
    • Another property called MAIN12 will probably get highlighted in red. This is for 12-bit support (if you want that check it).
    • Press Configure again
    • Press Generate
  5. Open the x265.sln in Visual Studio - change the build type to Release and the Arch to x64

  6. Change the C++ runtime to use static CRT (/MT) rather than the default (/MD) to match ffmpeg

    This should be done for the cli, common, and encoder projects.

  7. Build the ALL_BUILD project.

  8. Build the INSTALL project - this is important for building ffmpeg next

  9. Navigate to the location used in the CMAKE_INSTALL_PREFIX property above. Verify you have bin, include and lib folders.

Build ffmpeg with custom x265 build from above

  1. Download the ffmpeg source code

    Create the directory c:\third_party\ffmpeg and cd from a command prompt to it. Run the command:

    git clone ffmpeg

    This will result in the ffmpeg source tree located at c:\third_party\ffmpeg\ffmpeg which will be handy for an out-of-tree build

  2. Create the directory c:\third_party\ffmpeg\ffbuild

  3. Open a VS command prompt - I used the x64 Native Tools Command Prompt for VS 2017

  4. Change the directory - cd to c:\tools\msys64

  5. Run the msys2_shell.cmd for msys2

  6. Verify our x265 build with pkg-config

    pkg-config --exists --print-errors x265

    This should fail with an error similar to the following, since we haven’t told pkg-config where our x265 is.

    Package x265 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `x265.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'x265' found

    Add the x265.pc install location to the PKG_CONFIG_PATH environment variable


    Run the verify command again - you should get no error messages

    pkg-config --exists --print-errors x265
  7. Configure ffmpeg for a build.

    cd /c/third_party/ffmpeg/ffbuild/

    There are many different ways to build ffmpeg - I’m doing the bare minimum here to get a ffmpeg build that can encode 10-bit x265:

    ../ffmpeg/configure --toolchain=msvc prefix="win-build" \
    --pkg-config-flags="--static" \
    --disable-shared \
    --enable-libx265 \
    --enable-gpl \
    --enable-asm \

    If you’re following this far, the above command should fail :(

    ERROR: x265 not found using pkg-config

    We already verified that pkg_config could locate our x265 above, but the ffmpeg configure script is complaining …

    Open the c:\third_party\ffmpeg\ffbuild\ffbuild\ffbuild\config.log and scroll to the bottom. You should see an error something like:

    LINK : fatal error LNK1181: cannot open input file 'x265.lib'
    ERROR: x265 not found using pkg-config

    Navigate to C:\x265\lib. The x265 build from above created a static lib named x265-static.lib … rename it to x265.lib

    I’m pointing this failure out because often when building ffmpeg you will need to refer to the config.log when there is a problem

    Run the ffmpeg configure command again - this time it should be successful.

  8. Build and install ffmpeg

    make install
  9. Your fresh ffmpeg build should be at c:\third_party\ffmpeg\ffbuild\win-build

Transcode some content to 10-bit HEVC

  1. Create a test clip with something like:

    ffmpeg -i "input.mkv" -c:v libx265 -preset slow -crf 18 -pix_fmt yuv420p10le -c:a copy -y output_10bit.mkv
  2. Verify the video stream in the output file is 10-bit (we’re looking for confirmation the yuv420p10le pixel format was selected)

    ffmpeg -i "output_10bit.mkv"
    Stream #0:0(eng): Video: hevc (Main 10), yuv420p10le(tv, progressive), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc