Cloud Rendering in an Age of COVID

 

With the advent of COVID and the rapid deployment of remote working many companies in VFX had to quickly solve issues surrounding rendering and computer allocations.  Many studios had to leverage the cloud to allow artists to work from home where typical studio infrastructure simply did not exist. While the shift to the cloud had already gained momentum, the global COVID conditions significantly accelerated demand for cloud-based solutions such as remote rendering. What is surprising is how this actually played out and how some cloud solution areas have grown dramatically while others have not.

While some companies moved to use fully virtual workstations, with facilities deploying collaborative tools from Amazon Web Services (AWS), Teradici and others, given the very uneven nature of the effects of the global pandemic, this was not a universal response. GridMarkets for example is a company that provides cloud rendering. The company did see a huge spike in engagement with COVID, and they even offered a fully virtual workstation solution, and yet as much as their cloud rendering expanded, there was such limited interest in virtual workstations that the company has since withdrawn the service.

This surprising imbalance in remote solutions points to the VFX community have not responded in one consistent way to the pandemic and the changing nature of the business. This is not to say GridMarkets has not grown. The company has expanded its cloud renderfarm offerings and especially new workflows to pipe renders from Houdini into Nuke scripts directly in the cloud, and it plans to do more in the future, especially with Machine Learning. This all comes at a time when the Zync Renderfarm is shutting down on June 1, 2021. Zync had become part of the Google Cloud Platform but the company has decided to stop the service. Given the GridMarkets and other companies in Cloud Rendering, COVID rapidly accelerates the move to cloud solutions and work in a distributed environment. GridMarkets services both the pharma/biotech and Media & Entertainment. For M&E the company supports all the major renderers such as Renderman, Arnold, V-Ray, Mantra, and Redshift.

The company has an advanced pipeline that allows simulations and outputs from, for example, Houdini to be pipelined directly into Nuke. As part of an ongoing GridMarkets series of artist presentations Fabian Nowak, (Jungle Book, The Passenger) an award-winning VFX Houdini artist recently outlined his workflow (linked here). “I always worked in giant studios like MPC or ILM that have dozens of thousands of machines, the best machines in the world in their farm and I started myself looking for cloud solutions and I knew the name GridMarkets popped already a bunch of times on the Houdini Group that I’m administrating,” he commented. “I made my choice quite quickly and in a couple of days I was feeling like at home with GridMarkets. For the user you know, it’s super user-friendly. It’s super reliable. It’s… I don’t know, I really felt like I had something quite similar to the farm at ILM.’’ GridMarkets general series of artists talks can be found here.

Fabian Nowak

Mark Ross, co-founder & Director explains that the company is “actually a software company that sits between the customer and various Cloud providers”. For example, GridMarkets works with cloud newcomer Oracle, although not exclusively. They also work with other software partners, leaving GridMarkets to solve version and packaging issues or ‘software containers’. For each studio and often each project the pairing of the right versions of the various software with each other is incredibly important. During production, a team cannot just have an arbitrary new version of the renderer or creation packages suddenly used, each version rollout needs to be managed and timed to each client’s specific pipeline and feature locks. “GridMarkets has partnerships with Pixar, SideFX, Maxon, Chaos Group and Autodesk, through which we source the rendering licenses,” explains Ross.  “Because we do this, . . . when the customer submits, we leverage our cache of licenses from our 3d software partners to scale up and down the machine count and license counts as needed.  This happens ‘automagically’ for the user – they simply tell us how many machines they need.”

GridMarkets provides SaaS, connecting users, cloud and software partners

GridMarkets provides CPU and GPU solutions. The GPU renderer Redshift is very popular. Currently, Nuke is supported but the Machine Learning options around nodes such as CopyCat are not supported, but Ross commented that “ML solutions are coming soon, as we can see how important they are becoming.” GPU ML especially ML training is a natural fit with GPU technology, but depending on the application it can require vast data uploads. Unlike content creation packages, compositing and especially large machine learning training data can be an issue due to upload times. the Foundry itself had to revisit its’ cloud offerings for related reasons.

To help with output review and mobile monitoring of projects, GridMarkets has recently started working with Frame.io and Ross sees this as a key technology for both the company and the industry generally moving forward. The other critical issue that Ross and the team focus on is Cloud security, “when we speak to the Studios, it is the first thing they ask about,” he explains. GridMarkets is secure and offers multiple levels of data and process security.

With 4000 customers and growing steadily GridMarkets seems to have found the right path in a complex and very uneven evolving landscape of cloud applications and remote rendering. Given that remote working is the new normal, and unlikely to be fully reversed as countries open up, it is likely that cloud ecosystems will continue to evolve and innovate.