It was with a fair amount of excitement that we filed into the McEnery Convention Center in San Jose, for Apple’s WWDC 2019. With the changes in product roll outs, WWDC has become of the most significant events for professional and makers of creative tools from Apple. This year was particular relevant to our industry.
New Apple Mac Pro
Apple relaunched the Mac Pro this week with one of the most powerful workstations on the market. We were on hand to see the launch and talk with the Apple Engineers about this new machine, which is squarely aimed at Media and Entertainment.
Apple badly needed to solve their Pro line of Mac computers. Apple had constrained the old Mac Pro and created upgrade issues. The “trash can”, as it was nicknamed, did not support NVIDIA cards, and expansion was compounded by thermal issues. The old Mac Pro looked great, but it went nowhere, it was a supermodel when people wanted a workhorse.
The new Mac Pro 2019 is inherently a modular system that seeks to address most of the issues with the old design. Prior to yesterday, the maximum RAM in a Mac was 256GB, available on the iMac Pro. The new Mac Pro can be fitted with up to 1.5TB of RAM. Compare that to the old Mac Pro “Dust Bin” model which had a maximum of 64 Gig of RAM. The current iMac Pro is an 18-core processor with a 5K display. The new Mac Pro can be configured as a 28 cores, a high-performance system with multiple 6K HDR monitors.
AMD Graphics and not NVIDIA (or is it ?)
The official graphics position from Apple is that the Mac Pro graphics are all AMD. The options start with the Radeon Pro 580X. Mac Pro with the Radeon Pro Vega II, provides up to 14 teraflops of compute performance and 32GB of memory with 1TB/s of memory bandwidth. Mac Pro also has the Radeon Pro Vega II Duo, which features two Vega II GPUs for 28 teraflops of graphics performance and 64GB of memory, making it the world’s most powerful graphics card.
But Apple go further, the Mac Pro can accommodate two MPX Modules so users can use two Vega II Duos for a staggering 56 teraflops of graphics performance and 128GB of video memory.
Apple also launched Afterburner, an accelerator video processing card. The new Afterburner hardware features a programmable ASIC capable of decoding up to 6.3 billion pixels per second. With Afterburner, any editors needing to transcode native file formats into proxies or ProRes, can now decode up to three streams of 8K ProRes RAW video and 12 streams of 4K ProRes RAW video in real time, eliminating the need for proxy workflows.
For RED cameras, FCPX still needs a plugin for R3D files. RED is very excited for these new machines, and is working on getting their code running on Metal (Apple’s API for hardware-accelerated graphics), and hopefully releasing their plugin around September. As such, while fxguide was able to edit, grade and play with stunning RED shot 8K footage after the Keynote, it was all transcoded to ProRes. “Apple’s new hardware will bring a mind-blowing level of performance to Metal-accelerated, proxy-free R3D workflows in Final Cut Pro X that editors truly have never seen before. We are very excited to bring a Metal-optimized version of R3D in September.” commented Jarred Land, president, Red Digital Cinema.
The headline of the event was that there is no NVIDIA cards in the Mac Pro, however we spoke to the hardware team after the keynote. The Mac Pro has eight PCIe expansion slots. For the Mac Pro to use the graphics cards they would only need be fitted and the Apple provide the drivers. The machine can easily take the NVIDIA cards in terms of power, cabling and slots. Apple confirmed that, short of a legal or other non-technical reason, the only issue stopping the use of NVIDIA cards is the drivers. Apple is not confirming that they will release drivers, but there is nothing physically stopping them. The NVIDIA would have to dovetail with Apple’s Metal as Apple no longer supports OpenGL.
Why do we even care about NVIDIA? At the moment NVIDIA is critical to many people as NVIDIA has been leading the field in real time raytracing combined with Machine Learning support. Until yesterday, Nvidia’s CUDA-compatible cards are required to run apps like Redshift, Octane (v7.5 is NVIDIA RTX only) and Thea Render.
At the keynote, Apple announced both Redshift and Octane are moving to support Apple and Metal. These are implementations that are already advanced. We spoke to Jules Urbach, Founder & CEO of Otoy after the event and he showed fxguide 8K test images rendered from their new Metal prototype of Octane.
Metal is the key to GPU accelerated graphics in the Mac Pro. It is the computational framework that allows developers build everything from Pro apps to games. Metal includes deferred and tiled forward-rendering, it is an advanced and modern piece of software. The Metal is key as it schedules GPU work, manages workflows and does so with little CPU interaction. Apple fully controls drivers for Mac OS. Unfortunately, NVIDIA currently cannot release a driver unless it is approved by Apple. There’s certainly no reason Apple’s Metal 2 UI can’t run on Nvidia hardware.
Apple also announced extensive support for the new Mac Pro from Adobe, Autodesk, BlackMagic and Unreal. Additionally, RED, SideFX, AVID, Pixar, Unity, The Foundry, Maxon and others all signing on to support and work with the new Mac Pro system.
A long list of companies have come out in saying they will support the new Mac Pro and fxguide saw various demos after the keynote. For example, we got to see the extremely impressive render speed from Arnold rendering in Maya on the Mac Pro (see above). Autodesk is fully embracing the all-new Mac Pro and we are already working on optimized updates to AutoCAD, Maya, Fusion and Flame. “This level of innovation, combined with next-generation graphics APIs, such as Metal, bring extremely high graphics performance and visual fidelity to our Design, Manufacturing and Creation products and enable us to bring greater value to our customers,” stated Amy Bunszel, senior vice president, Autodesk Design and Creation Products.
Pixar tech was on display in a USD viewer at WWDC. The Apple demo was remarkable, showing interactive real time navigation of an entire production asset of Pixar’s new Toy Story 4 Fairground environment. One could move from individual assets inside the Antique shop to a wide shot of the whole fairground and surrounding town. It was a very impressive combination of the new Mac Pro and the open source USD.
Pixar are very keen on full Metal support in Hydra in an upcoming release of USD, toward the end of the year. “Together with this new release, the new Mac Pro will dramatically accelerate the most demanding 3D graphics workflows thanks to an excellent combination of memory, bandwidth and computational performance. This new machine clearly shows Apple is delivering on the needs of professionals at high-end production facilities like Pixar,” commented Guido Quaroni, vice president of Software Research and Development, Pixar. (Fxguide will have a much bigger USD story coming soon).
GRADING: New Apple 6K monitors
Pro Display XDR features a massive 32-inch Retina 6K display with gorgeous colour, an incredible 1,000,000:1 contrast ratio and a super-wide viewing angle: all at a very attractive price point for this type of monitor. (And with the sexiest and most expensive monitor stand in history, which alone is US$1000). The 6K monitor is 40% larger than the screen on the iMac 5K.
The remarkable aspect of the new Apple monitors is the Brightness.
The new Pro monitor supports incredible brightness with a peak of 1600 nits, (see below). This allows the monitors to support high-dynamic-range (HDR) images which have a far greater range of luminance than do standard-dynamic-range (SDR) images. HDR is designed to represent luminance values as high as 10,000 nits, creating more realistic color transitions and revealing more detail in both shadows and highlights.
For the last few years there has been a steady race to have both larger television and higher resolution monitors. The once humble monitor has migrated from 1920×1080 HD (SDR) to 4K , 5K even 8K screens. Apple has focused onscreen quality for some time and marketed their newer screens as Retina displays. The ‘Retina’ brand name applies to Apple’s IPS LCD, and OLED displays that have a higher pixel density than traditional Apple displays. In other words, the pixels are smaller, and more densely packed, leading to sharper images. The Pro Display XDR is also a Retina display with Apple’s dense pixel integration.
The MacBook Pro has a resolution of 2880 x 1800, while the iMac has a screen that is 5K resolution. But for some time Apple has been aware of the colour depth and brightness of the screen are every bit as important as resolution. The new Apple monitor is not only 6K resolution but it is able to display a wider gamut of colours and has tremendous brightness. It supports high dynamic range (HDR) workflows which require tremendous brightness and colour space range.
Depth not Width (or Colour Space vs. Screen Size)
HDR is not just more brightness. HDR is a package or complete system. Film Makers and postproduction pipelines can encode or master their content in a special HDR format to take advantage of the new technology. Special sets can then display this content, or a version can be displayed on standard HD (SDR) TV. There are two HDR formats in use for televisions today: Dolby Vision, engineered by Dolby, and a more generic HDR, also known as HDR10.
It is important to understand that you cant technically just run a HDR movie on a standard TV. The Dolby Vision HDR process, for example, encodes how to display on either an HDR or SDR screen using meta data. This means that you can grade in HDR and then get a more standard HD grade as a calculated version of it. If not handled properly, an HDR film would clip or just look wrong in terms of contrast and chroma.
Colour space is also an issue, A standard (SDR) TV might display 100nits of brightness but the range of colours is defined by the colour space or colour gamut. SDR images are Rec.709. HDR colour space is Rec. 2020, which means a wide set of colours.
The SDR and HDR colour space differences can be seen above in the classic ‘horseshoe’ diagram. This technical diagram is actually very simple to read, it just looks complex. The horseshoe is all colours as you can make from Blue, Green and Red (for complex reasons it is not a triangle in shape). But this diagram only refers to colours, it is independent of illumination (it assume full light effectively). The older HD monitors set of possible colors is the smallest inside dotted Rec 709 triangle. The iMAC supported colour space is the next largest triangle outside that: P3 (the solid line triangle). The new Pro Display XDR is slightly outside that P3, but not shown explicitly on this diagram. The largest triangle is Rec.2020
The Pro Display XDR does not support full Rec. 2020, no monitor does yet, but it supports 93% of Adobe RGB which is a bit more than P3. The new monitor’s colour gamut is incredible rich but no wider officially than the current iMac. The monitor is revolutionary for its 1,000,000 to 1 contrast ratio and brightness but in terms of workflow, the new monitor can only officially display P3. As with all HDR workflows, you can still master in Rec.2020 even if you can’t display it. The same is true for brightness. For example, the HDR standard is zero nits to 10,000 nits. No one commercially makes display that cover that range, but an HDR grade can still be done and thus your final HDR file would simply be a window of perhaps 0-2000 inside the HDR package which in the standard supports up to 10,000 nits.
What a nit anyway?
If the horseshoe diagram above is just about colour then luminance is just about brightness.
Brightness in this world of monitors is measured in ‘nits’. Actually the term nits is just shorthand for ‘candela per square metre (cd/m2)’ as the standard unit of luminance. (1 nt = 1 cd/m2). A nit is a measure of light emitted per unit area and the agree way to specify the brightness of a display device. The scale is logarithmic in the sense that it jump up by ‘stops’ and so 2000 nits to 3000 nits is half a stop difference. Significantly that is how your eyes also work, – and also camera lens. F-stops are equally non linear and so is your perceptive system.
• The spec for sRGB Rec 709 standard definition TVs and monitors targets around 80 nits.
• Calibrated monitors are over 120 nits.
• Most consumer desktop LCD screen companies have 200 to 300 nits, (and this is also common in retail ‘Point of Display’ LCD screens in shopping malls).
• High-definition screens range from 450 to 1500 nits. A high end reference PRM 4200 monitor is 600 nits.
• HDR grading is agreed to required at least 1000 nits, some would say more.
• Samsung’s flag ship, top of the line QLED TV monitors are 1500 nits to 2000 nits ( $$$+ monitor only)
• In Dolby HDR professional grading suites they have custom made professional 4000 nit Dolby Vision monitors.
By comparison, in the real world if someone with fair skin was to actually stand in the sun outside the keynote theatre in San Jose, their skin in sunny California would be perhaps 9000 nits and in the shade of the convention centre, that would be perhaps 600 nits.
Computer monitors are only now starting to support HDR, allowing for viewing with more detail in the brightest and darkest parts of an image. But most domestic ‘HDR’ TVs sold only display 300-400 nits.
What is confusing is that some monitors are designed and manufactured to reach peak of 2000 nits. They can’t sustain this brightness level over the whole screen for long, only a few moments, but if they can peak just in even part of the screen at 2000 nits, they claim they are a 2000 nit monitor. While professionals prefer to refer to a 600 nit monitor being 600 nits – full screen. Dolby won a technical Emmy when they released a few years ago the PVM 4200 monitor, it is rated at 600 nits and it was 600 nits, full screen, indefinitely.
Looking at domestic TVs and monitors, the BenQ EX3501R is a 35” curved panel with 3400 x 1440 resolution but while it has an HDR mode it is only able to deliver 300 nits full screen. The Samsung CHG90 is a 32:9 aspect ratio (3,840 x 1080) but delivers just 350 nits. 350 nits is good, much better than the standard 100 nits, but at the fxguide test studios, our top of the line iMac Retina screen is 5K resolution with a wide colour gamut of P3, but delivers 500 nits and we really want more. The billion colours that P3 supports is great, but not Rec. 2020 and yet 500 nits is not far off the Emmy award winning Dolby PRM 4200 (600 nits). As great as the iMac is, it is not marketed as a calibrated HDR grading monitor into professional studios. You can use it, but the professional market really requires more. The Pro Display XDR is more !
The Pro Display XDR was stated in the Apple keynote to be 1000 nits and yet in the press release it discusses 1600 nits. This is because the Pro Display XDR can display 1000 nits full white, full screen, indefinitely. However we played with footage shot by Director Mark Toia in Africa after the keynote, where the highlights were all 1600 nits and this image could be displayed infinitely. What the monitor could not do is guarantee to display a full screen of white at higher than 1000 nits infinitely. The Pro Display XDR can show normal HDR images and work at 1600 nits, but for a technical specification, it is rated as a full screen 1000 nits monitor.
Applications for HDR
Some people away from WWDC, might discuss nits as something you would only want for very dark Game of Thrones episodes with super bright Dragon fire sequences but Shane Mario Ruggieri CSI, at Dolby Vision research centre has a different point of view. Ruggieri showed us around Dolby’s HDR grading suite the day before the Keynote at WWDC (unrelated to the Apple event). Dolby are world leaders in HDR and building HDR grading suites and theatres. Ruggieri believes when you “get past 1500 nits, all of a sudden the image starts to take on depth. You start to see intonation and skin tones that just simply were not there before. Your eye is so good at understanding blushing and the different emotions in a face that it’s amazing to see on a screen” Add that believes, “once we get into real light luminance levels in mastering monitors and eventually consumer devices, content creators will be able to tap into everyone’s intimate understanding and interactions with light and as a result, craft audience reactions like never before”. Having viewed a professional suite at Dolby with a range of monitors up from 600 to 4000, we agree he’s right. The 4000 nits monitors at Dolby have nothing to do with Apple, but they some of the world’s best custom build precision monitors, vastly expensive to produce but Dolby services the most exacting end of the high end professional market. They are also not for general sale. (To hear more from Shane Mario Ruggieri CSI, check out Ben Allan’s DOP T-stop podcast with Shane).
There are many aspects about light, for example you can’t perceive saturation at low luminance levels. Dark things are much more monochrome. Thus incredible bright screens provide real world levels of saturation and incredible imagery. But in the world of post-production it gets more complex since a lot of people have found grading in HDR, even when then providing a normal HD (SDR) version, is a much better approach.
You might think that HDR is irrelevant unless everyone has an HDR monitor at home. HDR monitors are still rare, but they are taking off and steaming services such as Apple’s iTunes, Netflix, Amazon Prime now have downloadable HDR 4K content. But Apple understands that even if you are grading for normal SDR, it pays to grade in HDR and then output the SDR version.
In the old model of post-production that grading would happen at the end of the production, and it still happens that way today often. The traditional process is to offline in low res proxy, the visual effects done in OpenEXR, but reviewed in a standard Rec. 709 quicktime and then the final grade re-access the source footage and grades your final output. This method delays working in HDR until near the end of the pipeline.
What Apple is embracing is a whole different group of film makers who shoot digitally, capturing all the complexity of the world, but focusing on the actors performance. While they aim to get the right contrast ratios and not clip the highlights, they work knowing that they essentially have flexibility in post. This is not lazy film making. These are high end professionals who know their craft but want to focus on performance on set, not tech. Far from being lazy, they build complex pipelines that allow themselves to work with the actors to capture a great performance and avoid having to be distracted on set by technology. The master the technology so they can not focus on it.
For these filmmakers, they want to edit in HDR. This desire to avoid low res proxies is not because they are being self indulgent, but rather that they cannot judge a shot unless they can see the full range of what they have captured. Such teams partially grade as they edit. They know that they can dive into the blacks and play with the full dynamic range of the shot to choose the best takes of the actor’s performances. With this second approach in mind, Apple have not only produced hardware that can deliver amazing HDR performance, but they have also expanded Final Cut Pro.
Editing in HDR in Final Cut Pro
At WWDC Apple announced the latest version of Final Cut Pro (FCPX). This new version is not released yet, but we got hands on it after the show. The next version of FCP has a range of improvements, but we were just focused on the HDR pipeline. For sometime you have been able to edit in FCPX in HDR, but the process was limited by the complexity of monitoring and by the sheer processing power required to edit multiple streams of HDR.
Director and DOP Mark Toia flew to Africa to shoot on the RED cameras a stunning set of visuals that Apple showed during the event. The raw footage is for a documentary Apple wanted to use as a test case for the new FCP pipeline. Toia is a highly successful filmmaker who does vast big budget projects and TV commercials all of the world. On his elaborate shoots, it is not uncommon for Toia to shoot with 5 RED cameras simultaneously and combine this with multiple audio recordings. Toia is leading film maker who prefers to edit his work in HDR, even if the final deliverable is a SDR master.
FCPX has gained a lot of ground in the past few years for it robust interface which often requires less keystrokes to produce the same editorial changes as many other packages. It is elegant, fast and robust. It also has a very powerful set of built-in colour grading tools. While many FCPX productions still work in harmony with BlackMagic’s daVinci, many people finish expensive high end projects and grade them entirely in FCPX.
Mark Toia is a hardcore RED camera filmmaker, although he has a range of high end cameras. He is known as an uncompromising adventurous and large scale director who uses all the latest tech from drones, to Russian arms (complex car rigs), cranes and Steadicam. He is beholden to no one, he just seeks to work fast and at the highest quality. For Mark Toia and film makers like him, they can choose to now edit in FCPX, in HDR, with 3 layers of real time 8K footage or 12 streams of 4K. The monitoring is simple and the new displays just plug in to the Mac Pro without any additional boxes or convertors.
Looking to the future, as Apple’s HDR pipeline is more widely adopted as most think it will be, the illumination levels that someone sees in the edit suite are becoming much closer to what the actor actually experiences themselves on set. This means there is a much more direct relationship between the reality in which the actor is making choices and the world where we see that performance viewed. This means Actors can make acting choices more confidently about how the camera will view their performance.
This is part of what is sometimes referred to as Psychophysiological aspects, which is just a fancy way to saying ‘how the light actually affects you personally and physically’. Directors can use HDR as a new tool and play with how dramatic light changes physically work on an audience. When we walk from a dark room to a bright exterior, our eyes naturally take a moment to adjust. In a standard 100 nit SDR world, the film maker can not carry the shot from inside to out without clever adjustment of the lens aperture. But even with a blown out exterior, the audiences eyes are never reacting to that experience as they would in the real world. Your eyes are only being presented with a change from say 10 nits to 100 nits. We don’t blink or even notice when watching this on a screen. But if you watch an HDR film, your eyes have to adjust a lot more, just as they would in the outside world, since they could be transitioning from 10 nits to 2000 nits of brightness. This sudden light change, such as stepping into a spot light or flood light, requires your eyes to take a moment to adjust even when you are just sitting in the edit suite. HDR is an interesting tool that we are yet to see fully creatively embraced. Just as the move from 525/640 Standard definition video to 1920 HD resolution meant changes to wardrobe and makeup, it will be interesting to see how film makers embrace the creative side of moving from 100nits to 2000 nits brightness with 1,000,000:1 contrast with tools such as the new Apple Mac Pro. It is true the new Mac Pro configured fully will be expensive, but it is very much a professional tool aimed at a professional market.
fxguide will have more from WWDC and a special podcast from San Jose later in the week.