Founders Series: industry legend Jim Blinn

When this author was much younger – long before fxguide – I had the pleasure to hear John Lasseter talk at a BDA/Promax event in Los Angeles. This was before Pixar became the power house it is today, and before Lasseter was a household name. As part of his talk, Lasseter told a story about the original Luxo Jr. animation that today is the mascot of the company, but was then a new short film. Below I paraphrase this story as best I can, the way I recall Lasseter telling it:

“At the end of showing the film at SIGGRAPH there was a group of people standing around, all discussing the film, and I saw Jim Blinn walking over to talk to me…Jim Blinn! So I looked around for Ed or someone technical, I mean, if you don’t know Jim Blinn, he is a huge guy with this big beard – but more than that – I mean he invented bump mapping(!). He is a giant in every sense. Jim walks up to me, towering over me and says, “John,about the big lamp…”, and I am like, “Yes Jim?'”, thinking there must be someone here who can answer this question ’cause I am not going to know anything Jim Blinn would want to know. I am scanning the room for someone technical, I am thinking is this about the lighting or the modeling… 
“John, about the big lamp,” he repeats. “Ah, yes Jim?” Blinn leans over and says, “Was it a mother lamp or a father lamp?”
 –  And then I knew we had won – the film was a success.”

Dr James (Jim) Blinn is one of the true founders of the CG industry. He has spent his life expanding our knowledge in the industry and using that same technology to educate others. Rather than someone who sought fame or even wealth, his career is marked with the love of perhaps two things: space exploration and education. His career spans the invention of environment mapping, bump mapping, the critical JPL NASA flyby animations, multiple SIGGRAPH papers, awards, books and several TV shows.

In short, Blinn is one of the most beloved and respected founders of the computer graphics industry, and this week we got to discuss his amazing career. You can hear the whole interview as a podcast here, and below are notes from that interview and also extremely rare reproductions of key images that changed the industry, in fact many are famous images in the history of computer graphics that Jim Blinn himself has provided for this story. (Our thanks also to Paul Debevec for some of these images).

Listen to Mike Seymour’s in depth fxpodcast with Jim Blinn here.

University of Michigan

Blinn received his bachelor’s degree in physics and communications science from the University of Michigan in 1970, before computer science was offered as a college subject. He went on to earn a master’s degree in engineering at Michigan and a Ph.D. in computer science at the University of Utah in 1978. Blinn himself has pointed out how important this education would be to his later work, providing him with a brilliant key part of the jigsaw puzzle he would need to produce his highly awarded educational animations years later. Blinn did seven years of computer graphics work in Michigan. The University of Michigan may not have had a computer graphics department when Blinn was there, but he underscored to us after our formal interview just how important a role the University played in the broader advancement of computer graphics, something Blinn himself is working on writing about for an upcoming piece.

University of Utah

Most discussion of Blinn’s career, however, normally starts after Michigan with the famous University of Utah. Blinn had worked with various imaging techniques while at Utah, which to this day remains one of the key academic institutions for computer graphics research. So many of the founders of computer graphics can draw a direct line back to that University. The late 60’s and early 70’s period at the University of Utah was the golden era of computer graphics research. At that time they had the perfect mix of funding, people, equipment and opportunity to lead the world in computer graphics. To quote the ACM, “students came from diverse backgrounds, and for them the discovery was a labor of love. There were no Computer Science or Computer Graphics fields, nor jobs waiting for them upon graduation.” Various students from the University would go on to form Lucasfilm’s computer division, Pixar, SGI and Adobe, among other companies.

The world’s most famous 3D model – the Utah Teapot – was created at this time. Not many people know that the actual teapot created by British-born computer scientist Martin Newell was not always this exact shape. This difference in aspect ratio between the real teapot and the digital model is mis-quoted as having been due the aspect ratio of early displays at the University of Utah. Not so. Jim Blinn recalls having the original teapot up on the screen and during a demo in the lab. To show that they could manipulate the computer model, they scaled the model in the vertical axis. They simply then liked the new model’s look – “we thought it looked good” – and so they saved another version of the file, with it squashed.

Ivan Sutherland had just left Utah at this time, but someone recommended Blinn to him as a promising student. Years later, Ivan Sutherland would remark: “There are about a dozen great computer graphics people and Jim Blinn is six of them.” From 1968 to 1974, Sutherland was a professor at the University of Utah. Among the students at that time were Alan Kay, inventor of the Smalltalk language (Apple Fellow), Henri Gouraud, who devised the Gouraud shading technique, Frank Crow, who went on to develop antialiasing methods, and Edwin Catmull, computer graphics scientist, co-founder of Pixar and now President of Walt Disney and Pixar Animation Studios.

In 1968, Sutherland co-founded Evans and Sutherland with his friend and colleague David C. Evans. The company has done pioneering work in the field of real-time hardware and accelerated 3D computer graphics. Their story and hardware was central to the history of the whole industry. When Sutherland left Utah, “he called back to Martin Newell asking for someone to help him make a test image for a presentation,” recalls Blinn. “So I did that for him while I was a graduate student, sort of remotely as he was not at Caltech yet. So when I graduated from Utah, he had just started his tenure at Caltech, so I called him up and said hi and I am interested in maybe joining your department at Caltech, but to be honest with you what I am really interested in is JPL, because JPL is run by Caltech and I am interested in the space exploration.” Sutherland responded that Blinn’s timing was excellent as “someone at JPL had just got a duplicate of what was at Utah and they were looking for someone to run it.” So Blinn went to Caltech as a Postdoc, and spent half is time teaching at Caltech and the rest at JPL.

JPL and NASA

The Jet Propulsion Laboratory (JPL) was and is a federally funded R&D center and NASA field center located in the San Gabriel Valley area of Los Angeles County, California. Blinn would become synonymous with JPL and with graphics in general. This link between JPL and Blinn turned on the now famous NASA Voyager / JPL flyby simulations of Jupiter and Saturn. So young was computer graphics and its use so rare in the public arena that when Blinn’s shots were shown widely on the national news, people thought they were real. “There were two Voyager spacecraft sent up to Jupiter and Saturn,” recalls Blinn, “and they got there about six or eight months apart. Some people thought that the movie was made by one of the spacecraft watching the other one.”

Blinn worked very hard to make the films accurate and scientifically valid. “The thing you have to understand about the JPL flybys is that JPL never actually requested me to do this!,” Blinn told fxguide. “The gear I used was the duplicate of what I had used at Utah, and it was purchased with funds left over at the end of the fiscal year by a guy who thought it would be cool to do computer graphics, but wasn’t exactly sure what could be done with it. So there was no official project to do this animation – it was a back room sort of thing. I spent a lot of my time trying to make it accurate, but the early planetary surfaces we used a paint program to do an artist’s impression of what we thought they would look like, and then later on we were able to use actual photographs to use more accurate textures.”

Amazingly, the films were almost never seen by anyone.

“When I made the movies, JPL in a certain sense did not know what to do with it. The head of the public information office didn’t even know we were doing it and wasn’t sure what to do with it, and it just so happened that someone in flight planning showed the film to the head of NASA who said, ‘Great, let’s make a whole bunch of copies of this film and put it as part of our press kit’. The problem was the office of public information felt like they had been cut out of the loop.”

“Anyway,” continues Blinn, “they said, ‘Send us the negative and we’ll make a bunch of copies and we’ll send them out’. So we did and for ‘some reason’ they lost the negative! So I had to make another run of the program to make a new negative.” This time, however, Blinn made his own copies and prints. So successful was the reaction to the stunning imagery Blinn produced that from then on, “It was just expected that a new movie be made for each new mission.”

Environment and bump mapping

Blinn credits environment mapping as being mainly Martin Newell’s idea. At Utah, Newell was Blinn’s thesis advisor. “We were teaching the computer graphics course at Utah at the time,” says Blinn. “He was a professor there and I was a teaching fellow, we spent a lot of time taking about rendering and geometry in preparing for the lectures and I think we taught each other more than we ever taught the students. He came in one day, he had been thinking in the shower and said, ‘What if were to take the calculation for the lighting and instead of doing the regular dot product with the light source that would give you the color of the surface, what if you took the light and reflected it backwards? Like a mirror reflection and project that off to infinity and then look that up in amps and look up what would be reflected at that spot?’ And I thought that was pretty neat.”

Blinn had already written a test bed renderer and there was a frame buffer at Utah (plus Blinn had recently written a paint program), so he painted a rough room, implemented the maths and environment and reflectance mapping was born. Here are those images:

Blinn and Newell authored a paper called Texture and Reflection in Computer Generated Images, and delivered it at SIGGRAPH in 1976. A year later Blinn would publish his next paper on considering a surface as ‘facets’. “I had spent a a lot of time poking around in the engineering library trying to figure out better reflection models,” says Blinn. “At the time I got there (University of Utah) Bui Tuong Phong had just come up with a calculation of how light reflects off surfaces, or a simulation of that so you could get highlights and diffuse reflections. And it was mostly constructed on how they knew they worked but it was not based on any very complex physical simulations.”

“So I poked around in the engineering library,” adds Blinn, “and I stumbled on something called the Journal of the Illuminating Engineering Society. In 1920 they actually had an article where someone had actually measured how much light is reflected off a surface in all the different directions, given the light hitting the surface – with a diagram with a bump showing the specular reflection.”

This was the earliest BRDF that Blinn had ever found. He immediately set to work with a photocopier doing enlargements to try and help digitize the data. He also came across other newer articles, including one with a physical model, “by a guy named Torrance and a guy called Sparrow from Minnesota,” says Blinn, “who had a model of micro facets on the surface and a statistical effect it would have on reflection. But their calculation was horrendous – it used spherical trigonometry – I spent weeks doing trigonometric identities trying to figure a simpler way of doing it.”

Blinn would publish this as the Torrance–Sparrow model in 1977, a general approach representing surfaces as distributions of perfectly specular micro facets. The Phong model would be refined by allowing for certain quantities to be interpolated, and thus reducing computational overhead and be thereafter known as the Blinn-Phong model. But it was the next year, 1978, where he amazed everyone with something he thought of while looking at his shoe: bump mapping. “That was pretty cool,” remembers Blinn. “I was pretty happy about that, that was generated because my interest has always been in scientific visualization, and I was trying to make a picture of a water molecule. And I said suppose I could make a texture for the surface of the thing – atoms are always created as a fuzzy blob – how could I make this look like a ball of yarn to simulate the ways the electronics are always whizzing around really fast. So I tried various textures to make it look like a fussy ball of yarn, and they all looked like smooth formica – why is this not looking bumpy?”

Luckily for Blinn and perhaps for the rest of us, he was “looking at my shoes at the time and my shoes were made of leather but they had this embossed pattern in the surface, and I was looking at the highlights on them and what is making these things look bumpy is not the height of the displacement – how far the surface is displaced – but that it changes the angle of the surface from one place to the next. So what if I could nudge the surface normal from one place to the next, plug that into the lighting equation and that would be a good approximation of it.

Blinn then did the maths but as luck would have it before he could render it, the Utah frame buffer died. It was a prototype that E&S had made and it was wire wrapped, – meaning it had loads of hand placed wires across the back to get it working. “Basically the connections on it were flaky and it stopped working,” says Blinn. “So I flew out to New York Tech for Christmas so I could use their frame buffer, and it looked promising. So when they finally fixed the frame buffer at Utah I was able to do the thing right.”

He then produced a small 64 x 64 animation of a sphere turning to see if it was working. At first the sine was wrong and it looked like some kind of an optical illusion, but then he reversed the sine. “I was literally dancing around the room when I got it working.” This was all done on an 8 bit frame buffer. The shoes were left behind at Utah in an odd snow incident (you will need to listen to the podcast for all the details).

Blinn almost joined Xerox PARC, related to writing a book, but while he visited and interviewed at Xerox PARC, as he says, “I was much more commited to space exploration than what Xerox were doing at that time.” Blinn met Dr Alvy Ray Smith during a summer job at NYIT in summer of 1976. When Dr Ed Catmull and Dr Alvy Ray Smith went to Lucasfilm, Blinn briefly joined them. After some time at JPL, Blinn had decided that the 8 bit frame buffer at JPL wasn’t enough. “It was getting really constrained as to what I could do with that, you couldn’t do full color and there was just a PDP-11, and they were getting all this neat equipment at Lucasfilm, so I decided I owed it to myself to go somewhere that had better equipment. So I called up Ed and asked him if he had a place for me, and he said, ‘Sure come up and join us.'”

The year is now 1981, after the flyby simulations of Jupiter, but not yet Saturn. Once NASA and JPL discovered that they were about to lose Blinn, they dropped everything and asked him what it would take to keep him. “Well, a decent frame buffer and a decent computer,” Blinn recalls saying. So the director said that he would immediately spend his entire yearly discretionary funds to keep Blinn. “So here I was, I felt committed to Ed, but JPL had this attractive counter offer. So I was half time at each place for about six months. During that time I co-taught a course at Berkley with Ed, but I was basically commuting back and forth between LA and San Fran and I then finally decided my heart was in space exploration and besides Saturn was coming up, with all this data coming in and if I stayed at JPL I could make all these cool movies of Saturn. I couldn’t pass that up, I thanked Ed and went back to my home at JPL.”

Animation education: The Mechanical Universe

“It was the project I was born to do,” says Blinn of the Mechanical Universe Annenberg/CPB series for which he developed CG sequences. It consisted of over 500 scenes for 52 half hour programs describing physics and mathematics concepts for college students. He also worked with Carl Sagan on the PBS Cosmos series. These videos were revolutionary in their scope, size and ambition. Today we think nothing of 2000 VFX shots in a feature, but in 1985 such a huge amount of visual effects or animation shots was un-heard of. In 1985, Jurassic Park was still eight years away, Luxo Jr. would not be shown until the next year, and yet the series would end up incorporating almost eight hours of computer animation created by Blinn. While any one shot may not be complex by today’s standards, it was, as Blinn explains, “pretty grueling – it was basically seven years, seven days a week, working around the clock, and doing it one shot at a time. The project was run by a physics professor at Caltech who I became pretty close to and we would meet every day at 2 o’clock in the afternoon and I promised myself that I would have something new to show him every day. It was just total focus to get this going and I was just glad I was in the right place at the right time to make that work, because it was pretty stunning.”

Blinn had no off-the-shelf software packages to make these, it was “all home built software”. He relied on a VAX 11-780 (the one JPL had gotten for him) and RGB frame buffer and a 1″ animation rigged BVH 2100 tape machine. Blinn had to work out all the animation, render it all on his own renderers and lay it off to the 1″ machine a frame at a time (standard def NTSC). He wrote the recording program and the machine control for the VTR control. Each frame took about a half to three quarters of a second to write to video tape. But Blinn points out this was easy compared to Voyager – “that was luxury, compared to going to film. You kids today have it easy!” (ed: referencing both Monty Python and Secret Policeman’s Ball in one sentence!). For film, Blinn recorded on 16mm and the film was generated by filming the CRT monitor in the lab. Blinn would normally sit in the dark for 8 to 10 hours while this happened, just in case he saw a dud frame. He could jump at the rig and stop the film camera after the frame should have displayed, but just before the film camera clicked the film frame – often times effectively editing in-camera between good and reloaded ‘bad’ frames.

Blinn would go on to work with Carl Sagan on Cosmos: A Personal Voyage, and a third project, very similar to The Mechanical Universe, called Project Mathematics! While researching for this interview and watching many hours of clips we saw what we assumed was Dr Blinn himself appearing on camera playing a musical instrument. Indeed he confirmed that he did feature. “I have played trombone since junior high school off and on, and enjoyed it,” says Blinn. “I was in the Caltech wind ensemble. They had several bands, a jazz band and a wind ensemble and other things like that. As Caltech isn’t that big, the band is open to faculty and students and even people outside to fill out all the spots. For the 20 years I was at Caltech I was in the wind ensemble or the jazz band and got to be good friends with the director. And when we had this program on sines and cosines, one of the examples was Fourier synthesis and how you can build up different wave shapes. So we got the wind ensemble to come and play with us and that worked out quite nicely!”

Jim Blinn’s Corner

Blinn started writing columns for the IEEE sharing his insight and experience in a regular piece called ‘Jim Blinn’s Corner’. This award-winning column in the technical magazine IEEE Computer Graphics and Applications was where Blinn would publish some of his most useful graphics methods and observations. These columns ended up as a series of three books. Each book is a compendium of about 20 columns or articles, covering maths, graphics pipelines and a wealth of tips and tricks.

In his third book, Notation Notation Notation, he covered the tricky issue of mathematical notation and its role in both maths and allowing understanding of complex subjects. Blinn points out there is almost a key or code in how notion is done. “That’s sort of become my obsession these days a bit, I have always had a hard time understanding maths from mathematicians and I guess ’cause I think of it differently from the mathematicians.” Blinn has spent a lot of time now seeing if there are ways that he can express maths in formulas in a way that would make the mathematical concept easier to understand.

“You might think mathematical notation would be the most consistent and systematic form of representation possible. You’d be wrong! Mathematics is a natural language, made up by humans, and contains just as many inconsistencies and arbitrary rules as any other natural language.”

– From Blinn’s 2002 April IEEE column.

There is both a lack of explicit standardization, and the notation gap can be seen often times at Blinn’s second home: The ACM SIGGRAPH. There can be a gap between the people who research and discover new computer graphics algorithms and those who seek to learn from them and use them. For example, there is a world of ‘codes’ or conventions in maths at SIGGRAPH that if you don’t know it, makes it much harder to understand. Again, from Blinn’s April 2002 column:

When someone says “Consider the x, y, z values,” you immediately think they are talking about 3D coordinates…Coordinates of points usually come from the end of the alphabet (x,y,z). We use the beginning of the alphabet for polynomial coefficients or for the coordinates of lines and planes (ax+ by+ cz +dw)…texture co-ordinates from the middle (u,v).

He then goes on to point out how greek letters are used for angles and special ratios like Pi, subscripts, superscripts, overhead adornments and a ‘host of foreign language fonts’ for other things, a whole world of secret conventions which unless known, make reading maths formulas very difficult. Blinn asks the question:  “Wouldn’t it be nice if we could establish completely consistent rules for the meanings of all these choices? Wouldn’t it be nice if there was Standard Notation?”

Blinn has spent a lot of time on this problem and it is a passion – a passion that once again comes from his massive technical knowledge and his desire to spread that knowledge and help educate people. Today his focus is his website www.jimblinn.com. The site is new and material is only just starting to be posted there, but this will be his new primary medium for communicating and educating.

Blinn’s Law

“As technology advances, the rendering time remains constant.”

While Moore’s law has been beaten with GPUs and other theories come and go, Blinn’s law of rendering seems to be holding. The Law is based on human patience (more than the evolution of technology): as render times reduce, complexity increases and after all the years of advances in the film industry, frames, on average, are not getting faster to render.

Link to the audio version of this story

All images copyright, 2012. Do not reproduce without permission, fxguide, LLC.