There is no magical way to ensure perfect real-time playback of Sony XAVC, XAVC-S (or any AVC, H.264, H.265 / HEVC based format) natively in DaVinci Resolve, but I do want to explain the reasons why you might be experiencing poor playback performance so it’s easier to accept and embrace the solution.
Resolve has to decode H.264 and H.265 / HEVC encoded files using your system CPU before full raster video frames can be processed, and while this has been improved in Resolve, it’s still a problem for many on laptops and lower power systems to do this in real time. Working with these codecs natively can also result in audio drift and sync issues over the duration of long clips.
From the DaVinci Resolve 15 Hardware Selection and Configuration Guide on page 5:
However, for editing, VFX, and grading, the compressed data needs to be decompressed to the full RGB per pixel bit depth that can use four times or more processing power of a HD image for the same real time grading performance. The decompression process, like compression, uses the CPU, so the heavily compressed codecs need more powerful and a greater number of CPU cores. H.264 and H.265 are heavily compressed formats and while not idea for editing are often used by lower cost cameras. If you use these types of compressed codecs you will need a more powerful CPU or be prepared to use proxies or Resolves optimized media feature.
The fact is, these codecs have never been good for post production. The solution is to transcode your camera media to an easily decoded post-friendly codec. If you don’t want to do this as a separate step before you start editing, you can create Optimized Media within Resolve, and / or use Render Cache to transcode your clips on the timeline. Each of these has pros and cons.
What is XAVC?
Sony’s XAVC (and its little brother XAVC-S) are very efficient compressed video formats, enabling high quality video encoding in low to intermediate bit-rates and small file sizes. XAVC and XAVC-S are based on the H.264 codec with uncompressed LPCM audio.
A wrapper is the container for audio and video data. It is where your video file extension comes from… .mov, .avi, .mxf, .mp4 etc. XAVC uses the MXF wrapper, and XAVC-S uses MP4 wrapper, but the video inside is still H.264.
The H.264 codec is a very flexible codec for efficient video encoding. It comes in a number of profiles which can accommodate different capabilities when it comes to color bit-depth, chroma sub-sampling, resolution, and intra or inter-frame compression. Sony make use of the “high” profile levels 5.1 and 5.2 which have had some compatibility issues with common decoders.
XAVC / XAVC-S / H.264 / HEVC and DaVinci Resolve
DaVinci Resolve has a long history, with its origins dating back to the early 1980’s as da Vinci Systems, long before the technology was bought by Blackmagic Design in 2009.
DaVinci has always been a high-end color correction and finishing system. It was designed primarily for performance with uncompressed video sources and media. This has changed and the system has evolved to be far more flexible and inclusive of all kinds of compressed and uncompressed media, but it is important to keep in mind Resolve’s heritage. Resolve is not exactly like your other NLE’s and it doesn’t process image data in quite the same way.
CPU and GPU
Any discussion of performance in DaVinci Resolve is always spoken about in close connection with the GPU (Graphics Processing Unit). However the decoding of source media is tasked to the CPU, not the GPU. It is important to understand the internal processing pipeline involved. Resolve first must decompress and decode the source media into its uncompressed 32-bit floating point YRGB space regardless of source media color space, color bit-depth or chroma sub sampling scheme.
In the case of H.264 and H.265 this is a computationally intensive task. In order for Resolve to play back your XAVC or XAVC-S (or AVCHD for that matter) at full resolution in real-time, your system CPU must be capable of decoding and expanding the video data into 32-bit float YRGB space in memory in real-time. This is a vast amount of data to generate, from scratch, on the fly. It’s all down to your CPU to decompress, decode and then generate all this data.
We also talk about storage read bandwidth, and this is very important with less compressed media, but it is not the bottleneck preventing your system from playing back your H.264 based media in real-time. The file sizes are small, and do not require the same read-speed from storage as less compressed, larger formats.
Why your system handles uncompressed media and struggles with H.264
So you’ve built or bought a monster workstation with a SSD RAID and maybe it’s equipped with one (or a few) Titan-X GPUs. It eats through high resolution uncompressed files like a hot knife through butter and even churns through 8K R3D without breaking a sweat. Why not H.264?
The simple answer, is H.264 and H.265 / HEVC codecs take a lot more processing to decode. Uncompressed media, RAW, and compressed formats are much easier for your CPU to decode. The CPU can take data from disk, decompress, and with or without the help of GPU, generate the required uncompressed 32-bit float YRGB image data fast enough for further real-time image processing and playback. In the case of RAW formats, the GPU handles the debayering once the CPU has dealt with any decompression.
So, it’s a CPU + GPU dance, and depending on the type of media, one may do more work than the other.
In the case of H.264 and H.265 / HEVC, it’s pretty much all on the CPU. Your GPU is being held up waiting for your CPU to decode and generate the required frames. It just can’t compute it all quickly enough.
H.264 and H.265 / HEVC are not post-friendly codecs
The fact of the matter is XAVC and XAVC-S, or AVCHD, or any other similar format is very efficient for encoding video data into small file sizes. It is not however easy or quick to decode for high-quality finishing and post production processing.
What you can do, is transcode your source media to a intermediate format which Resolve is able to process far more efficiently. These files then become your master media for post. Apple ProRes is a great general purpose format for this, as is Avid DNxHD or DNxHR and as long as you are transcoding at the same resolution, to a less compressed format, in the same or greater color bit-depth, you should not be losing any image quality as a result. Here’s a guide of possible transcode formats to consider.
- Apple ProRes 422 HQ – for source media of 8 or 10-bit color depth, 4:2:0 or 4:2:2
- Apple ProRes 4444 / 4444 XQ – for source media of 10-bit color depth or more, 4:4:4, suitable for HDR
- Avid DNxHD 220 – for source media of 8-bit color depth, 4:2:0 or 4:2:2
- Avid DNxHD 220x – for source media of 10-bit color depth, 4:2:2
- Avid DNxHD 444 – for source media of 10-bit color depth, 4:4:4
- Avid DNxHR HQX – for source media of 8 or 10-bit color depth, 4:2:2 or 4:2:0
- Avid DNxHR 444 – for source media of 10-bit color depth or more, 4:4:4, suitable for HDR
The resulting transcoded media should have the same timecode as the source media as well as the same filename. Only the file extension may have changed. Transcode into a separate folder in order to avoid overwriting your camera source files if the extension is the same. The resulting files will be larger, but since all the decoding of the source files has been done, Resolve no longer has to rely on the CPU to do it in real-time.
DaVinci Resolve Optimized Media
If you don’t want to transcode all of your source media before bringing it into Resolve, you can use Resolve’s built in Optimized Media tool. DaVinci Resolve gives you the option of generating optimized media directly from clips in the media pool and it handles all the internal linking of files between your original camera files and the optimized media, allowing you to switch between camera media and optimized media in your timeline. You can set the resolution and format for the optimized media in the General panel of the Project Settings.
There are a number of formats to choose from, but keep in mind your storage configuration when you select one. Resolve stores optimized media on the same scratch disk location it stores render cache files. You’ll need to make sure you’ve specified a location on fast enough storage to read your optimized media real-time.
Resolve lets you make optimized media in any flavor of Apple ProRes from “Proxy” up to 4444 XQ as well as Avid DNxHR LB through 444. You also have the option of making Uncompressed 10-bit or Uncompressed 16-bit float media which is stored in Resolve’s own .dvcc image format.
Transcoding is the solution
At the moment, transcoding your media before bringing it into Resolve, or generating optimized media within Resolve is your best solution to improve performance when editing media from cameras that record to XAVC, XAVC-S, AVCHD or any H.264 or H.265 / HEVC based codec.