One of the more interesting products floating around the show floor at NAB was the Furnace plug-ins, a set of extremely useful tools developed by The Foundry. Longtime discreet users are probably well familiar with Tinder, an outstanding set of tools that are currently available as plugins for Shake. The Furnace tools provide some “automagic” cleanup of some common tasks effects artists have to deal with in the course of our jobs, as well as some outstandi
One of the more interesting products floating around the show floor back in the spring at NAB was the Furnace plug-ins, a set of extremely useful tools developed by The Foundry. Longtime discreet users are probably well familiar with Tinder, an outstanding set of tools that are currently available as plugins for Shake. The Furnace tools provide some “automagic” cleanup of some common tasks effects artists have to deal with in the course of our jobs, as well as some outstanding grain tools.
The Furnace set of plugins contains:
• WireRemoval
• Rig Removal
• DeGrain, ReGrain
• Kronos (motion estimation speed change)
• DeFlicker
• Steadiness
• Tile
• PixelTexture and BlockTexture
imgmiddle(furnaceimgs/nodes.jpg)The Furnace set has quickly turned many artists into fans of the software package. “The Furnace grain tool is hands down the best de-grain tool we have seen on the market,” relates Payam Shohadai, President of Luma Pictures in Los Angeles. “Even in times when we are compositing in packages other than Shake (which I think was the only package that had Furnace at the time), we run many film plates through Shake just for Furnace. We’ve used it on five of the films we’ve worked on this year, because it removes the grain usually without any ‘de-grain’ artifacts, which is uncommon in this type of tool.”
The foundation of the toolset finds its roots in motion estimation technology. Bill Collis joined The Foundry in 2000 to develop the algorithms for Furnace, after spending the previous five years working on motion estimation for Snell & Wilcox. During this time he developed the motion estimation algorithms used in the Bullet Time sequences of the first Matrix Film. More robust techniques were needed for a commercial product like Furnace and so Bill brought in Anil Kokaram, a Fellow of Trinity College Dublin and expert on Bayseian signal processing. The Foundry were then able to use this expertise to develop the gradient based, motion estimation algorithm that is at the heart of Kronos and many of the other plug-ins within Furnace.
Furnace for Discreet (in the form of a Sparks package) is currently entering the beta testing stage and should be available for purchase in September. Furnace for Discreet will be priced at $7,500US (£5,000), while the plugins for Shake are priced at $5,000US (£3,000). The price difference between the two packages goes a long way towards paying for a license of Shake, and since many of the operations are processor-intensive there are benefits to running it in the background on a lower-cost workstation.
The plugins are very processor-intensive because of their reliance on motion estimation technology, so it will be interesting to see how performance on the sgi workstation compares with Shake performance. Because of the difference in price between the discreet and Shake versions, it might make sense for many facilities to use Shake as the platform for Furnace, offloading time-intensive processing to a lower cost workstation with multiple Intel or G5 processors. Using floating licensing, it can provide flexibility of where work is completed — sometimes the lead artist might want to do the work and sometimes it could be farmed off to junior artists or assistants. The difference in price between the two goes a long way towards the purchase of Shake for an OS X workstation.
In addition to being processor-intensive, the Furnace plugins are extremely memory hungry. The downside is that they can become somewhat unstable as scripts grow large and complex, using nodes that consume lots of RAM. According to Matt Plec, currently at ESC Entertainment, “The Foundry has done a pretty good job of managing it as well as possible, but the fact is that what’s going on in a lot of these tools doesn’t really lend itself to efficient tiled processing. You have to use them a bit judiciously and precomp to avoid having too many of the memory-hungry tools in the tree at once.”
Network rendering is supported with the Furnace plugins at an additional cost for each render node. On the Shake platform much of this usefulness is lost because sampling of previous/next frames cannot be done with renders which are distributed over multiple machines. Depending upon the settings used for plugins, this can negate the use of distributed rendering. On the Discreet platform, burn rendering will be supported and since batch has tools for accessing previous/next frames network renders will be more robust using burn. And based upon fxguide’s tests, processor-intensive sparks get a dramatic increase in speed when using burn versus rending on the sgi workstation.
Because of the time-based nature of the optical flow technology and the way time is handled (or not handled) in Shake, tools such as Kronos must work directly on inputs. Careful consideration must be given to this, becuase it is totally possible that two branches out of the same node are rendering different points in time and the results can be unpredictable. Its something Shake users are fairly accustomed to dealing with, but still important to be aware of when working with Furnace.
Support of Furnace (and Tinder) from The Foundry is excellent. “We got great support from The Foundry through the beta period and even after the first release,” according to Plec. “Furnace tools were generally stable and the few significant problems we found were fixed very quickly. I think that made a good impression on users (and people paying the bills) and gave them confidence in relying on it. And as you know, with the way things go when you’ve got a lot of work and a hard deadline that’s really important.”
Here’s a look at the plugins on a module-by-module basis. FXguide spent a considerable amount of time with Furnace and the supplied tutorial images certainly helped in getting up to speed. But they are so deep in nature with numerous controls that a cursory look only goes a small part of the way in giving the plugins their due. We’d certainly suggest to anyone to download the software and get a demo license from The Foundry — its easy to do and the supplied docs and images provide a fantastic overview of the package.
Wire Removal / Rig Removal
Wire Removal will remove an object in straight line from a scene, which can be useful for removing film scratches as well. This worked exceedingly well for some wire scenes that I had from a recent job — the handles to deal with grain and motion going through the wire area are outstanding. There are four selections in the way the wire is removed. The first, simple, interpolates directly from the pixels on one side of the wire to the other. The gradient method will analyze the image and interpolate the pixels across the wire at the angle that the algorithms determine it to be correct. Inpainting, will actually replace the wire with pixels from near the wire, creating a texture to fill the wire gap. Finally, temporalInpainting does the same thing, but also analyzes the previous and next frames. I really found the ability to remove wires outstanding.
As an additional test, I used a film sequence of a close-up of an actor’s face that had three film scratches running over the talent’s face. One of those nightmare fixes. The talent turned their head several times and also changed expressions, making the use of some of the more normal scratch removal techniques less than ideal. As with the other Furnace plugins, there are numerous handles and ways to attack the job. Each node in the Shake node tree can remove a single wire, so for my purposes I used three nodes to remove the three scratches from the talent’s face. Using two renders, simple and temporalInpainting, I was able to complete a large portion of the fix with very little effort and manual labor. As with most of the plugins which use pixel analysis for the results, *some* manual work is to be expected….there were obviously areas of the image that didn’t work at all. But the idea is to be able to get a great deal of the way there without problems. Grain is nicely duplicated within the results, since it is part of the pixels that are being used to clean up the wire — and there are controls to manage how much grain is left in the image.
RigRemoval allows you to remove images passing through the frame. The software analyses previous and later frames to find pixels to replace within the image. You start by defining the region that contains the foreground object you want to remove — you can use either an animated rectangular region for feed an alpha image to define the region. Next, you need to define the number of frames (lookAheadFrames) it takes to sample out the object from the scene. Furnace shows you the area in red — when the red is not displayed, you’ve selected enough frames to sample. When multiple objects pass over the original, you can define matte areas (using gray instead of white) that flag regions that should not be used for repairing the background. What is quite interesting about the plug-in is that it can deal quite effectively with panning shots. I tried some shots of my own (instead of simply the supplied tutorial images) and had fairly good success. The thought initially that went through my mind at times was that it might have been faster just to fix the manual, old-fashioned way of tracking various parts of the image to fix the scene. It took some time to get things right, and each attempt involved some manual cleanup afterwards to deal with problem areas. However, I found that in the short time I was experimenting with the plug-in, that I got a better handle on using the tool.
I was curious as to how more-experienced Furnace users found the wire and rig removal tools….had I just gotten lucky? Not at all, according to Shohadai, who is a huge fan of the tools based upon work Luma Pictures has completed. “The most appropriate word for the rig & wire removal tools is ‘jaw-dropping’,” he states. ” I think for many people it gets them to 70-80% of completing the shot. We haven’t necessarily used it to get the shot done faster, but rather used it to get the shot done better in the same amount of time. In many situations where you may have had to create a clean plate (which has it pros and cons), we’ve been able to instead use the tool to clean the original moving plate and with the help of the de-grain tool clean up remaining artifacts.” They have also found other uses for the tools as well, which is a common thread amongst Furnace users. “For a shot for which we had to do a creature transformation, we needed to remove the actors arm and replace it with a CG human arm which transforms to a creature’s arm,” comments Shohadai. “The actor’s arm was crossing over his moving body, water and fabric. Without these tools it would have taken us no less than a week to paint out the arm, where Furnace did it in 5 hours. That was incredible.”
DeGrain and ReGrain
Artists working at film resolution are well familiar with the need to have effective grain tools for compositing. For artists such of myself who began in working with standard definition video images it wasn’t as obvious until we began working with HD telecine transfers. As the built in flame/inferno grain tools have begun to show their age users have been on the lookout for improved grain tools. Furnace certainly provides an outstanding toolset for this.
At the simplest level the idea of using the tools is to sample an area of an image with little detail by dragging a box over the region. For the DeGrain tool, the software analyzes this area and uses both spatial and temporal filtering to remove grain in the image. Since pixel analysis is being done, an “intelligent” job is done at determining what is grain and what is image detail. The results are quite good, and there are handles to deal with the various color channels, motion thresholds, and smoothing. ReGrain does an amazing job of replicating the grain in an image. Sampling of the image is done the same way — select a region with little background variation. There are also controls for film stock selection (a variety of Fuji and Kodak stocks are available, each with 2K, 4K, aperture and non aperture corrected selections), grain scale, grain luminance, and filmic curves (for adjusting grain applied based on luminance within the image).
Kronos
Kronos uses motion estimation pixel analysis to slow down a clip. Instead of simply changing the frame rate and using blending techniques, motion within the frame is analyzed and new frames are created in order to slow down the motion. This tool is similar in nature to REALVIZ ReTimer (standalone and Adobe/Discreet Plug-in) and Re:VisionEffects Twixtor (Adobe/Discreet and Shake Plug-in). The algorithims used for Kronos are quite good in comparison to the other two products. As a new user to Kronos, I found that the Twixtor plug-in have some additional controls (such as masking and foreground isolation tools) for dealing with problems. However, the quality of the Kronos render is quite impressive.
Plec states that he has seen “a lot of good results coming out of Kronos both for retiming and also for adding motion blur. Results of these kinds of tools do depend on the source material but it seems to be pretty robust and in most cases provided good results with very little tweaking — usually just bumping up the sampling/quality related controls. ”
At THE MILL in London, the 3D group uses Kronos to help generate motion blur due to long render times of raytraced results. “We have seen various solutions (some much cheaper) but this one is the best in terms of not getting confused on ambiguous motion points,” according to Dave Levy of THE MILL. “It copes well with the hidden-now-exposed object issue and performs extremely nicely in a high noise/grain situation.”
DeFlicker
DeFlicker is quite interesting, mainly because it doesn’t simply adjust the overall luminance in the scene as other deflicker tools such as Sapphire FlickerRemove complete the task. Instead, Furnace DeFlicker attempts to analyze the image and correct the flicker only in certain regions of the image. This involves analyzing the motion within the image to determine if it is flicker or simply moving content within the scene. I used some old film test footage that had a bit of light leak in the transfer to videotape. The flicker manifested itself as a region of varying blue that could appear and disappear, as well as move around the scene. By separately analyzing the RGB values, DeFlicker was able to greatly reduce the flicker from the scene in a manner which was much more effective than the manual techniques which I had previously tried. The plug-in did not remove the entire flicker, but got it to the point where it very acceptable in less time and with much less work.
Steadiness
The Steadiness node is quite powerful — it will smooth out a shot by analyzing the motion within a frame and then adjusting the output accordingly. In my time with plugins, I only used the supplied test footage, but the results were quite outstanding. The basic process in using the node is to first analyze the motion within the frame — you can select whether you want to stabilize or just remove the shake and keep the main camera motion. You can also provide clues to the plug in as to whether there are rotation, scaling, and perspective changes in the original footage. After the analysis is completed (which is the time-intensive part), you select how much of the camera shake you want to remove and how much of the camera motion you want to keep. You can also feed a second image into the node, which is used to align the first input image with the second input image.
Zave Jackson of Skaramoosh in London is fan of the module. “The setup time is dramitcally reduced due to the fact that Furnace Steadiness tracks every pixel in the shot and does not require the user to configure search areas and refernce points for individula trackers,” according to Zave. “If the user is happy to dabble with Shake expressions the steadiness node can really come into its own by using the generated data to link into Shake’s own tracking and transformation tools and used as matchmove data.”
Tile
Tile is a plug-in that will take a source image and tile it into a larger image (as large as you want). As an automated process, this could be considered similar to tiling an image multiple times and then smoothing out the hard lines between the tiled images. Again, the results aren’t perfect, but I was quite amazed with how quickly the plug-in got 90% of the way to the end product. I found that smaller objects (such as pebbles, grass, etc) seemed to hide the imperfections of the algorithms better than larger objects. There is also a feature that allows the algorithms to attempt to remove any shading in the image that make the tiling obvious (such as having the top of source image brighter than the bottom of the source image).
PixelTexture/BlockTexture
This isn’t a tool I played with much during the time I had to check out the software, but what I saw was quite interesting and powerful. If the tiling of a texture doesn’t quite do the trick, there are other tools in the Furnace arsenal to help you out. PixelTexture and BlockTexture serve the purposes. The basic idea of these tools is to provide the plug-in with the image which you need to have repaired, provide a matte determining where you want to have the texture applied, and then define the area which you want to sample for the texture. The Furnace docs included a good example of crowd replication in a stadium as a possible use of this technique.
