FiLMiC Pro’s new computational image processing and LogV2 profile can stretch the video dynamic range of the iPhone XS, iPhone XS Max and iPhone XR up to 12 stops in ideal conditions.

The engineers at FiLMiC Pro have combined Apple’s new imaging dynamics and smart HDR with their own computational imaging module called “Cubiform” to greatly improve the dynamic range of video captured using the new FiLMiC LogV2 gamma encoding profiles in the FiLMiC Pro app. FiLMiC LogV2 works on all iOS devices and most Android camera 2.0 API capable devices.

The age of computational imaging has only just begun, and smartphone manufacturers seem to be the ones leading the race. It’s the battle for the hearts and minds of everyday consumers that will now drive innovation in digital imaging, not the stagnating business and technology of professional video. You can read more of my thoughts on this here: Computational Imaging, Mobile Computing and the Future of Photography.

This became especially clear when I first saw the results of a new generation of active image processing in Apple’s iPhone XS Max. This technology intelligently automates adjustment and manipulation of exposure, tone, color and improves the quality of photos and videos for average users. It was a bit of a shock to realise this new image pipeline did not play well with many existing apps intending to allow a high level of control. The new devices did not behave like previous generations. Developers would have to adapt.

Back in September 2018 I had the opportunity to test the iPhone XS Max with the FiLMiC Pro team in Barcelona, using the existing latest build of FiLMiC Pro at the time, which was in no way optimised for Apple’s new hardware. It was an interesting experience to say the least, as it became very clear that a lot had changed in the new devices and the FiLMiC team had a lot of work ahead of them.

I shot and color graded an impromptu and unplanned video a day or two after the iPhone XS Max dropped into Apple stores, which, while not my best work, showed a hint of the potential in Apple’s new hardware and image processing. One of the reasons the grade is such a disaster is that Apple’s brand new smart HDR / dynamic tone mapping was constantly fighting with the app’s own image processing. Many aspects of the image were almost uncontrollable. Still, it was the first day the FiLMiC Pro app and FiLMiC team had ever met the new Apple hardware, so there were bound to be some issues.

I was subsequently roasted in the video comments for the grade… thanks everyone, you guys are ruthless. The internet has no mercy.

Apple’s new image processing seemed like Voodoo at the time, but things have become much clearer in the months since.

Fast forward to November 2018, and I started testing a secretive build of FiLMiC Pro that incorporated a brand new custom computational imaging module called “Cubiform” that the FiLMiC team had been building for quite some time.

Now this is known as FiLMiC LogV2.

What has become very clear in my months of testing FiLMiC LogV2 is that using a precise reference chart, such as the X-Rite ColorChecker Passport Video is the best way to ensure consistency in color correction.

What is FiLMiC LogV2?

The performance of Apple’s A12 SOC has allowed the FiLMiC Pro engineers to perform more complex math on the fly than was possible before. The new algorithms from FiLMiC Pro work to complement Apple’s new dynamics and smartHDR, not fight against it.

The result is up to 12 stops of dynamic range from the iPhone XS, iPhone XS Max and iPhone XR (under suitable conditions) and an optimised new intelligent Log gamma encoding profile called FiLMiC LogV2 which performs better than the original FiLMiC Flat profile with none of the characteristics that kept me from using FiLMiC Pro’s first generation Log profile.

Even the iPhone SE and iPhone 6S Plus get a 1 to 1.5 stop boost by using the FiLMiC LogV2 profile, approaching 10 stops of dynamic range.

FiLMiC LogV2 Dynamic Range Test Charts

First let’s look at the charts, then I’ll explain a bit about how this is achieved. No voodoo is involved, just clever math and processing power.

As you can see, the latest generation iPhones show a notable increase in dynamic range. This is the larger image sensor, improved optics and of course the Apple A12 chip and neural engine at work powering FiLMiC’s Cubiform computational processing.

Even the iPhone SE without a neural engine at all manages an extra stop of dynamic range when shooting with FiLMiC LogV2.

These tests were performed with a Xyla 21 stop dynamic range testing chart in a light sealed room. All test footage was captured at 4k extreme (H.264) with a framerate of 24fps and a manual white balance of 3800k. Electronic stabilization was disabled.

Below is FiLMiC Pro’s testing methodology as officially documented.

  • The smartphone was held approximately 5-7cm from the Xyla chart with the lens positioned over the third f.stop, which was used to set exposure (we purposely chose to over expose stops 1 and 2).
  • Digital zoom was then employed from the rocker so that f.stop 3 completely filled the frame to aid in setting exposure.
  • Exposure was then locked, and FiLMiC Pro was zoomed all the way back out.
  • We used our live analytics to confirm f.stop 3 was correctly exposed, with f.stop 2 and 1 clipping.
  • Smartphone was then placed in a tripod approximately 2.5 feet from the Xyla test chart.
  • A clip using the ‘Natural’ curve was recorded.
  • A clip using the ‘LOG’ curve was recorded.
  • Clips were imported to a computer using iTunes File Sharing to ensure a lossless transfer.
  • Using Premiere Pro CC 2019, natural and log clips were composited on top of one another in a new sequence and exported as a still image frame.

FiLMiC LogV2 is silky smooth to color grade and resists the banding and other artefacts you’d expect from an 8-bit implementation of a log gamma encoding. Only when grading some extreme styles did I need to employ some debanding on skies, for example, but with a lot of normal looks, this would not be necessary.

I believe FiLMiC LogV2 has pretty much hit the limit of what can be done within the 8-bit 4:2:0 encoding imposed by Apple.

Beyond this Apple would have to allow or enable higher video color bit depths. So far this is not possible, so it has forced the FiLMiC engineers to get creative.

How is FiLMiC LogV2 implemented?

For a start, FiLMiC Pro is not simply globally reducing contrast on an already baked in video stream to make it look like flat or log video. If this was the case, as with a LUT, the result would be a sacrifice in recorded image information and quality. All of the problems associated with pulling and pushing 8-bit encoded video around in post would be even worse.

FiLMiC LogV2 is not fake news, it is not a LUT, it is not a global contrast curve, it is constructive, not destructive. Welcome to the age of computation imaging, where you can get out more than you put in.

AI powered computational imaging processes make all kinds of things possible. From sophisticated noise reduction to removing entire objects from images, and rebuilding the background, these processes and algorithms are often context aware. It is possible to intelligently, and accurately infer more useable information at the output of the process than was ever received at the input.

Now this goes against everything digital imaging experts in the video and motion picture fields have staked their expertise and understanding on. While the rule that you can’t get out more than you put in, has been absolutely and steadfastly true of analog and digital video processing up to this point, it is simply no longer true.

Are these computational processes 100% accurate? No, but in many applications the results are near indistinguishable, and certainly good enough to be extremely useful.

FiLMiC Pro is taking the source 8-bit luma and chroma image data into an expanded 64-bit space and using these known values to model a high quality luma and chroma map in a mathematical process they call gamma vectorization. This computed luma map is derived from a flattened 24-bit RGB vector, and results in a computed fidelity greater than the 8-bit precision of the individual source RGB elements.

For encoding, this 64-bit scalar representation of the image is mapped back into an 8-bit container, but the output bits are in fact brand new, sampled from the model rather than directly from the source input stream. The result is a cleaner, high dynamic range, 8-bit encoding containing optimized pixel values that preserve more overall dynamic range and better tonal coherence.

How does FiLMiC LogV2 hold up in post?

Over the past months I’ve shot many tests, mostly with an X-Rite Video ColorChecker Passport under a variety of conditions. I’ve over-exposed, under-exposed, shot with ND’s, no ND’s and can say that as long as white balance and exposure are correct in-camera when recording, and are locked, FiLMiC LogV2 is capable of producing truly excellent images.

With the first FiLMiC Pro Log and Flat profiles, I quickly realised the Log implementation was just too aggressive. As a result, all of my videos shot before this have been recorded using FiLMiC Pro Flat profile. However from now on, FiLMiC LogV2 has won me over completely.

FiLMiC Pro LogV2 when underexposed a little, does even hold up to being corrected although there is no reason to underexpose on purpose. When over exposed however, there is no possible recovery of clipped highlights. For the best results in post, carefully watch any highlights that you want to protect. There is little room for error, and limited room for correction, but there is enough room, as long as you take the time to white balance and expose correctly.

I’ve created two video tutorials and written up an article to teach you exactly how shoot with FiLMiC LogV2 and color correct easily by using the X-Rite ColorChecker Passport Video: Shoot and Color Grade FiLMiC LogV2 with the X-Rite Colorchecker Video

FiLMiC have a deLogV2 LUT and deFlatV2 LUT available for download that help give you a good starting point to getting your video close to Rec.709. I found that I still needed to tweak things after the LUT in order to line everything up on the scopes.

FiLMiC LogV2 Tips

  1. Invest in an X-Rite Video ColorChecker Passport Video.
  2. Use the Video ColorChecker grey card reference to set and lock white balance often. The light can change quickly outside and can throw off your color. Always place the grey card reference so it is directly illuminated evenly by your key light source or full ambient light and not in shadow. You need to place your Video ColorChecker grey card reference close enough to the camera lens so it fills the frame before locking white balance, or use the zoom function temporarily to fill the frame with the grey card.
  3. Shoot your Video ColorChecker color chart at the beginning of each camera setup or position. Ensure the chart is illuminated evenly by your key light source or full ambient light and not in shadow and is clearly visible in the frame. Try to avoid reflections in the black reference chip to ensure you get a usable black point reference.
  4. FiLMiC Pro’s exposure reticle works very well, you can place it over the brightest area of the image, and lock it once set. Outside in daylight this is often the brightest part of the sky as an example. Set and lock your exposure after locking white balance (see point 2 above).
  5. Don’t worry if the ISO is above minimum after locking the exposure. The phone will find the best ISO for the scene lighting ratio at the minimum possible shutter speed (for a correct exposure) taking into account the image processing dynamics.
  6. Don’t try to fight the phone.If you try to manually adjust shutter speed or ISO after locking the exposure reticle you can get some image artefacts and inconsistencies in tones and levels. The “Pro” in you wants to go all-manual, but resist the urge.
  7. Always record the highest possible bit-rate “FiLMiC Extreme”.
  8. If maximum image quality is your goal then get over your obsession with high frame rates. Slow motion is not worth the sacrifice in resolution and image quality. Stick to 24, 25 or30p.
  9. Always download your FiLMiC Pro video files using iTunes, don’t ever save or export to the iOS camera roll.
  10. Use the FiLMiC Pro deLogV2 or deFlatV2 LUT as a starting point for color correction in post.
  11. Use your recorded XRite Video ColorChecker Passport chart as a reference on a waveform and vectorscope in your color grading software to make corrections to levels, hue and saturation for each shot that you have recorded a chart. This will guarantee a technically correct Rec.709 image, and that your shots will match.

Disclaimers

FiLMiC LogV2 and FiLMiC Flat do not correct identically.

I have found that if you are relying on an XRite Video Colorchecker for color correction and shot matching in post, it may be best not to mix FiLMiC Flat and FiLMiC LogV2 video as there are differences in tone even after manual correction using the chart. Of course they can be matched with more granular and time consuming tweaks, but this can be tricky.

Extreme high contrast and backlit scenes.

Very high contrast backlit scenes can cause severe problems and completely unusable video. Apple’s dynamic tone mapping and smartHDR will always try to reduce extremes of contrast, both attempting to protect highlight information, at the same time as raising shadow and midtone levels. In cases of extreme contrast, the shadow and midtone levels being raised may meet the highlight levels being protected, and the image will completely fail.

Other notable observations using FiLMiC Pro with the iPhone XS Max.

  1. The iPhone XS Max doesn’t heat up when running FiLMiC Pro nearly as much as the SE or iPhone 7 Plus that I use. I have not tested other iPhones to compare. I have experienced no dropped frames or slow-down of FiLMiC Pro on the iPhone XS Max.
  2. I can shoot more or less continuously recording 4K in FiLMiC Extreme with no issues on the iPhone XSMax.

Conclusions

FiLMiC Pro’s new “Cubiform” image processing combined with Apple’s smartHDR and dynamic tone mapping result in a very usable high dynamic range 8-bit encoding in FiLMiC LOGv2 and FiLMiC FLATv2 profiles. The resulting video, if correctly exposed and recorded, preserves a wide dynamic range and is responsive to color correction and grading.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.