So I was trying to create some 10-bit test HEVC content with ffmpeg and ran into the following error:
ffmpeg -i "input.mkv" -c:v libx265 -preset slow -crf 18 -pix_fmt yuv420p10le -c:a copy -y output_10bit.mkv ... Incompatible pixel format 'yuv420p10le' for codec 'libx265', auto-selecting format 'yuv420p'
I discovered that by default - most ffmpeg builds include only 8-bit support for HEVC encoding with libx265.
Certainly, you can get builds that already enable 10 or even 12-bit support - but I’m usually a glutton for punishment so I thought I would see how hard it was to just build it manually.
Here are the exact steps I did to build both ffmpeg and libx265 with Visual Studio 2017 on Windows. Some of the steps will also detail some of the errors I ran into along the way - in case someone else runs into trouble, hopefully then it will show how to overcome any future hurdles.
Note: This guide will only build the 64-bit versions, but you should be able to adjust for 32-bit.
With a text editor, open the file C:\tools\msys64\msys2_shell.cmd and un-comment the line
rem set MSYS2_PATH_TYPE=inherit
This is done so msys2 can use your existing PATH environment variable. (thanks to Scott Davies for pointing this out)
Now run c:\Tools\msys64\msys2_shell.cmd. When the msys2 shell opens - use pacman to install the following:
pacman -S gcc pacman -S diffutils pacman -S make pacman -S pkg-config pacman -S autoconf
Download the x265 source code
Create the directory c:\third_party and cd from a command prompt to it. Run the command:
hg clone https://bitbucket.org/multicoreware/x265
Navigate to third_party\x265\build. Find the directory that matches the VC compiler you want to use. It is likely there will not be one matching the newest version of Visual Studio, so use the latest one available (In my case it was vc12-x86_64)
Using a text editor, open make-solutions.bat. Change the generator command to match the version of VS you want to use. I’m using 2017, so my command is:
cmake -G "Visual Studio 15 Win64" ..\..\source && cmake-gui ..\..\source
Update 12/9/2017 - libx265 should now have a corresponding make-solutions.bat file for Visual Studio 2017. This step is no longer required
CMAKE_INSTALL_PREFIXto a directory the x265 lib, headers, and importantly the pkg-config .pc file will be installed to
NASM_EXECUTABLE property is pointing the location where you installed NASM
MAIN12will probably get highlighted in red. This is for 12-bit support (if you want that check it).
Open the x265.sln in Visual Studio - change the build type to Release and the Arch to x64
Change the C++ runtime to use static CRT (/MT) rather than the default (/MD) to match ffmpeg
This should be done for the cli, common, and encoder projects.
Build the ALL_BUILD project.
Build the INSTALL project - this is important for building ffmpeg next
CMAKE_INSTALL_PREFIXproperty above. Verify you have bin, include and lib folders.
Download the ffmpeg source code
Create the directory c:\third_party\ffmpeg and cd from a command prompt to it. Run the command:
git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
This will result in the ffmpeg source tree located at c:\third_party\ffmpeg\ffmpeg which will be handy for an out-of-tree build
Create the directory c:\third_party\ffmpeg\ffbuild
Open a VS command prompt - I used the x64 Native Tools Command Prompt for VS 2017
Change the directory -
cd to c:\tools\msys64
Run the msys2_shell.cmd for msys2
Verify our x265 build with pkg-config
pkg-config --exists --print-errors x265
This should fail with an error similar to the following, since we haven’t told pkg-config where our x265 is.
Package x265 was not found in the pkg-config search path. Perhaps you should add the directory containing `x265.pc' to the PKG_CONFIG_PATH environment variable No package 'x265' found
Add the x265.pc install location to the
PKG_CONFIG_PATH environment variable
Run the verify command again - you should get no error messages
pkg-config --exists --print-errors x265
Configure ffmpeg for a build.
There are many different ways to build ffmpeg - I’m doing the bare minimum here to get a ffmpeg build that can encode 10-bit x265:
../ffmpeg/configure --toolchain=msvc prefix="win-build" \ --pkg-config-flags="--static" \ --disable-shared \ --enable-libx265 \ --enable-gpl \ --enable-asm \ --enable-x86asm
If you’re following this far, the above command should fail :(
ERROR: x265 not found using pkg-config
We already verified that pkg_config could locate our x265 above, but the ffmpeg configure script is complaining …
Open the c:\third_party\ffmpeg\ffbuild\ffbuild\ffbuild\config.log and scroll to the bottom. You should see an error something like:
LINK : fatal error LNK1181: cannot open input file 'x265.lib' ERROR: x265 not found using pkg-config
Navigate to C:\x265\lib. The x265 build from above created a static lib named x265-static.lib … rename it to x265.lib
I’m pointing this failure out because often when building ffmpeg you will need to refer to the config.log when there is a problem
Run the ffmpeg configure command again - this time it should be successful.
Build and install ffmpeg
make make install
Your fresh ffmpeg build should be at c:\third_party\ffmpeg\ffbuild\win-build
Create a test clip with something like:
ffmpeg -i "input.mkv" -c:v libx265 -preset slow -crf 18 -pix_fmt yuv420p10le -c:a copy -y output_10bit.mkv
Verify the video stream in the output file is 10-bit (we’re looking for confirmation the yuv420p10le pixel format was selected)
ffmpeg -i "output_10bit.mkv" ... Stream #0:0(eng): Video: hevc (Main 10), yuv420p10le(tv, progressive), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc