COLOR SPACE, BIT DEPTH and FLOAT… some questions

[dudestein]

Hi Everyone!

In advance, thanks for taking the time to read my LONG post and hopefully respond to my questions. Also, sorry if there is any duplication in questions… but sometimes it was hard to illustrate my point that I felt was necessary for the reader to have in order to get a glimpse of my thought process and how I arrived at my assumptions. The questions themselves pertain to compositing.

I believe I have a pretty good understanding of the topics listed in my thread title, but there have always been a few nagging questions that continue to plague me with uncertainty with regards to DIGITAL IMAGE WORKFLOW.

I’ll do my best to paraphrase my concerns and how they relate to each of the topics.

First off, before each set of questions… I’ll give you an overview of what I think I know… and then I’ll ask for clarification concerning that particular area. Perhaps you’ll find that my assumptions are incorrect and therefore are the root of my problems understanding certain areas of the digital image work flow.

Well… here I go….

WHAT I BELIEVE TO BE CORRECT:
1 – a digital image, no matter what BIT DEPTH it is stored at, needs to be ‘normalized’ into the range of 0 to 1 in order to allow for an even playing field within a comp package. This way, an 8 BIT and 16 BIT image can be worked on inside of the same comp… because for each of the R,G and B channels, the pixel value stored will be divided by its maximum allowable for that BIT DEPTH. In other words, an 8 BIT image with its maximum of 0-255 code values per channel allows us to divide each channel’s value by its max of 255 which will result in a number ranging between 0 – 1. The same of course applies to the 16 BIT image, the only difference being that each channel’s value will be divided by 65,535 maximum carried by each channel in this BIT depth. Again, the result will be a number ranging between 0 – 1. Now, with all numbers ‘normalized’ mathematical operators can be applied equally across the board.

THIS LEADS TO MY FIRST QUESTION:
1 – during this ‘normalization’ of the numbers, are we not converting the values into FLOAT SPACE? They are no longer incremental integer based steps of 0 – 255 for 8 BIT or 0 – 65,535 for 16 BIT, but decimal based numbers now. Is this correct?

So, if my understanding of the definition of FLOAT space is correct and this is indeed what the numbers are now in, it leads to my second quesiton:

2 – Is FLOAT SPACE synonymous with accessing SUPERWHITES and SUPERBLACKS within the image or is that simply a result of what COLOR SPACE the image was originally stored in?

BACK TO WHAT I THINK I UNDERSTAND ABOUT COLOR SPACES:
I know that a CINEON 10 BIT file is stored in a NON-LINEAR COLOR SPACE and the curve representing it is LOGARITHMIC. I also believe this means that due to our eye’s superior sensitivity to luminance values over chrominance values that the CINEON file’s NON-LINEAR COLOR SPACE biases the information into the ‘toe’ or shadow portion of the image information and designates the BLACK POINT to be 95 and WHITE POINT to be at 685 within its 0 – 1024 incremental integer based steps, therefore leaving space for the SUPERBLACK and SUPERWHITE values and in essence, makes use of the available storage space of image data much more effective by saving information that our eyes are more sensitive to. What confuses me here is that when this CINEON file is brought into a comp package, the comp package will LINEARIZE the curve.

This leads to my next question:

3a – Is this not cutting off the SUPERBLACKS and SUPERWHITES? Won’t the LINEARIZATION of the image data mean that now the comp package will see the 95 value position as 0 (black) and the 685 position as 1 (white) with all other information now lost? Or is this where FLOAT SPACE comes into play? Do you need to make sure that you’re working space is set to be either 16 BIT half float or full 32 BIT float in order for the comp package to allow you access to those SUPERBLACK and SUPERWHITE values?

3b – And if so, I still find this confusing because what I understand FLOAT SPACE to mean is simply that you’re getting ‘FINER STEPS’ between the image values… NOT extending the RANGE OF IMAGE VALUES. The extension of the image values from my understanding is solely based on the COLOR SPACE your digital image file was saved in. By this train of thought, I would assume that as long as you had access to the SUPERBLACK and SUPERWHITE data that was stored in the original digital file, then it would be the comp package that would determine whether you could see that information or not. If thats the case, you could work in integer based steps (not that you’d want to), but as long as the comp package allowed you to see those values, you could. Is this correct, or even possible? I know that NUKE automatically promotes all images to 32 BIT float which seems to support my assumption in question 3a.

OVERALL – WHAT I THINK I UNDERSTAND ABOUT COLOR SPACES:
An image stored in a LINEAR COLOR SPACE would mean that the image is using its available BIT depth to represent the colors/luminance values it contains with an even distribution from black (0) to white (1). This of course is after the values have been NORMALIZED. Therefore, this would preclude any SUPERBLACK or SUPERWHITE data being included because there would be no data outside of that range. An 8 BIT channel value of 255 divided by itself is 1, nothing more. Therefore, you image data will clamp white at 1 (which is whatever value white was sampled / captured at in that particular image).

Conversely, an image stored in a NON-LINEAR COLOR SPACE (such as a CINEON 10 BIT file encoded logarithmically) would use its available BIT DEPTH to store luminance and chrominance values that the human eye is more sensitive to (in the dark/toe area of the curve and it would include values that are a lot more bright than our eye can normally see (the shoulder portion and above part of the curve). It would take its 10 BIT range of values of 0 – 1023 and map black to 95 and white to 685. This way, there would still be room for data storage of SUPERBLACKS and SUPERWHITES. Of course, once these values were NORMALIZED, you could very easily get a value above 1 due to the fact that white (685) can divide into 1023 more than once. However, in one of the articles I read (http://magazine.creativecow.net/article/cineon-files-what-they-are-and-how-to-work-with-them), it talks about cineon white values clipping at 13…

My Question:

Where is that value of 13 times brighter than white coming from? isn’t the max value over one that you would get when normalizing a potential value something like 1.49 (dividing 1023 by 685)?

MORE ON WHAT I THINK I UNDERSTAND ABOUT COLOR SPACE:
I also think that ‘REGULAR’ digital images (by regular, I mean not originating from film and being scanned into LOG space), such as those rendered out of a CGI package or acquired through a digital camera of some sort have a NON-LINEAR COLOR profile (sRGB gamma correction) embedded in them in order for them to display correctly on our computer monitors and TVs.

Again, I know that the comp package wants to work on the images in LINEAR COLOR space and will generally convert them for you behind the scenes (depending on the package you use). I believe this to mean that the gamma correction (the NON-LINEAR COLOR SPACE) applied to a digital image when it was created so that it would appear correctly on our monitors, is removed and the image is converted to LINEAR in order for the comp operators to work correctly. I know that with packages such as NUKE, this is taken care of by your global settings panel where you tell NUKE how to treat incoming images. It’s default is sRGB for both 8 and 16 BIT images. Generally, the average user is unaware of this as the viewer LUT is set to display sRGB while you work on the image and most would never see the image in its LINEAR version unless you set you viewer to display as such.

My questions concerning this:

4a – Does this mean that standard digital images, whether they originate from a digital camera with an embedded sRGB color profile, or images that are rendered out of a CGI package with an embedded sRGB profile (meaning they’re in NON-LINEAR COLOR SPACE), encode the image data in RGB with white clamped at 1 (no SUPERWHITES) and black set at 0 (no SUPERBLACKS) and the GAMMA CORRECTION / NON-LINEAR COLOR SPACE is only there for the image to appear correct when viewed on a computer or TV monitor?

4b – Does a file type automatically determine this? For example, is an 8 or 16 BIT TARGA or TIFF automatically an image with clamped white and black values?

4c – Isn’t the fact that if the image is in a NON-LINEAR COLOR SPACE then it will automatically be storing SUPERWHITES and SUPERBLACKS? If this is the case, does it mean that the file type would have to be a TIFF 16 BIT half float at least?

4d – Does setting an image BIT DEPTH to 16BIT half float or 32 BIT float when rendered / and or acquired by the camera, make it so that SUPERWHITES AND SUPERBLACKS are recorded?

4e – So I guess basically, the above questions really boil down to:
What determines whether the digital representation of an image will contain SUPERWHITES and SUPERBLACKS? Is it the COLOR SPACE in which it is stored, the FILE TYPE or a FLOAT BIT DEPTH setting when the image is ‘acquired ‘ rendered’, or all three, or something else I am missing completely? If FLOAT SPACE does have something to do with this, I find that confusing because my understanding of FLOAT SPACE is that you’re simply getting ‘finer steps’ between all your stored values regardless of what COLOR SPACE your image is saved in and its really the COLOR SPACE that determines whether or not you’ll have SUPERBLACKS and SUPERWHITES saved.

[claudio antonelli]

generally speaking, float color space is linear. Nuke’s really smart about this, flame a bit less so (but getting better) and I have no experience with Shake, Toxic, AE or Fusion in a float context. When the images are converted to non-float (tiff, dpx, cineon, whatever) they will need to go through a lut to convert them to the appropriate color space. Nuke, again, is the champion here, making the process almost invisible.

in float, 0 is black and 1 is white, so the math for an 8 bit image would be value/255 and for 10 bit would be value/1023 and so on. The images do not get increased steps in color values added to them when converted, but a native float image will have the extra steps.

the bonus of float is that you can have a value of 50, which would be 50 times as bright as white, in a similar way to the sun being brighter than anything else, yet will only ever be displayed as white in a daylight photo. The advantage here is you can move the brightness of the image around without killing the overall light values, so you change the exposure in a way similar to a camera.

cineons have a log curve so they behave in a way that exposes to film correctly while not banding up the way an 8bit image would. The sub-blacks and super-whites there for under and overexposure, giving you a higher dynamic range, though not as high as float.

When you convert them to sRGB or Linear (or whatever color space) in a non-float context you will clip all the values outside of what the lut cares about (95 to 685 in your example). When you convert the log file in a float workflow, all the sub and super blacks and whites are maintained, becoming values under zero or over one.

Nuke’s got a handy exposure slider to look at how a lot of this behaves, and flame has the Shift+E virtual slider that does the same thing.

[Anonymous]

Thank you Andy for replying so promptly to my looong, and I’m sure, somewhat tedious post. I’m still a little fuzzy on some of my questions. If you don’t mind checking back in a few days, I hope that you’ll be willing to answer some more. I’ll do my best to be short and concise this time around.

Thanks again,

Karl

[vamsi krishna]

@dudestein 31857 wrote:

THIS LEADS TO MY FIRST QUESTION:
1 – during this ‘normalization’ of the numbers, are we not converting the values into FLOAT SPACE? They are no longer incremental integer based steps of 0 – 255 for 8 BIT or 0 – 65,535 for 16 BIT, but decimal based numbers now. Is this correct?

You can normalise into float space, but there is no reason to think that’s the only way to do it. If you bring various footage into an 8 bit system, it will all be normalised to 8 bits, etc. When you hear that everything is normalised to 0-1 without any context, think of it as something abstract, rather than a guarantee about how the numbers are stored. 0 means “minimum” and 1 means “maximum” when you aren’t in float, but the exact numbers depend on how many bits you are using.

@dudestein 31857 wrote:

2 – Is FLOAT SPACE synonymous with accessing SUPERWHITES and SUPERBLACKS within the image or is that simply a result of what COLOR SPACE the image was originally stored in?

Float space always understands the concept of superwhites and superblacks, but they will only exist in a particular image if it actually has them.

@dudestein 31857 wrote:

BACK TO WHAT I THINK I UNDERSTAND ABOUT COLOR SPACES:
I know that a CINEON 10 BIT file is stored in a NON-LINEAR COLOR SPACE and the curve representing it is LOGARITHMIC. I also believe this means that due to our eye’s superior sensitivity to luminance values over chrominance values that the CINEON file’s NON-LINEAR COLOR SPACE biases the information into the ‘toe’ or shadow portion of the image information and designates the BLACK POINT to be 95 and WHITE POINT to be at 685 within its 0 – 1024 incremental integer based steps, therefore leaving space for the SUPERBLACK and SUPERWHITE values and in essence, makes use of the available storage space of image data much more effective by saving information that our eyes are more sensitive to. What confuses me here is that when this CINEON file is brought into a comp package, the comp package will LINEARIZE the curve.

You always work on the file in linear because linear mathematics are quite simple. To work natively in log space would require stupidly complicated nonlinear mathematics to be correct. There would be no benefit to making a compositing system that actually works 100% logarithmic internally.

The logarithmic curve has nothing to do with the eye being more sensitive to luminance than chrominance.

@dudestein 31857 wrote:

3a – Is this not cutting off the SUPERBLACKS and SUPERWHITES? Won’t the LINEARIZATION of the image data mean that now the comp package will see the 95 value position as 0 (black) and the 685 position as 1 (white) with all other information now lost? Or is this where FLOAT SPACE comes into play? Do you need to make sure that you’re working space is set to be either 16 BIT half float or full 32 BIT float in order for the comp package to allow you access to those SUPERBLACK and SUPERWHITE values?

There is no reason that linearising a logarithmic file would cut off the superblacks and superwhites. You can cut them off if you want to, but that’s separate from just linearising the data. If you want, you can map 10 bit log -> 16 bit int with the full range. It’ll probably look washed out, but you can keep the whole range when you straighten out the curve if you want to. It’s just a little more idiomatically obvious in floating point.

@dudestein 31857 wrote:

3b – And if so, I still find this confusing because what I understand FLOAT SPACE to mean is simply that you’re getting ‘FINER STEPS’ between the image values… NOT extending the RANGE OF IMAGE VALUES. The extension of the image values from my understanding is solely based on the COLOR SPACE your digital image file was saved in. By this train of thought, I would assume that as long as you had access to the SUPERBLACK and SUPERWHITE data that was stored in the original digital file, then it would be the comp package that would determine whether you could see that information or not. If thats the case, you could work in integer based steps (not that you’d want to), but as long as the comp package allowed you to see those values, you could. Is this correct, or even possible? I know that NUKE automatically promotes all images to 32 BIT float which seems to support my assumption in question 3a.

Floating point has a much higher range of values than an integer format with the same number of bits. Especially if you consider the integer format to only represent 0-1. Floats can represent negative numbers and very large numbers.

@dudestein 31857 wrote:

OVERALL – WHAT I THINK I UNDERSTAND ABOUT COLOR SPACES:
An image stored in a LINEAR COLOR SPACE would mean that the image is using its available BIT depth to represent the colors/luminance values it contains with an even distribution from black (0) to white (1). This of course is after the values have been NORMALIZED. Therefore, this would preclude any SUPERBLACK or SUPERWHITE data being included because there would be no data outside of that range. An 8 BIT channel value of 255 divided by itself is 1, nothing more. Therefore, you image data will clamp white at 1 (which is whatever value white was sampled / captured at in that particular image).

Floating point files are generally in linear space, and can have values greater than 1. BEing linear doesn’t imply in any way that a file is necessarily in 8 bit int. In fact, it’s rare that you would actually have true linear data in an 8 bit file. Gamma corrected images and sRGB space are much more common in 8 bit than true linear.

@dudestein 31857 wrote:

Conversely, an image stored in a NON-LINEAR COLOR SPACE (such as a CINEON 10 BIT file encoded logarithmically) would use its available BIT DEPTH to store luminance and chrominance values that the human eye is more sensitive to (in the dark/toe area of the curve and it would include values that are a lot more bright than our eye can normally see (the shoulder portion and above part of the curve). It would take its 10 BIT range of values of 0 – 1023 and map black to 95 and white to 685. This way, there would still be room for data storage of SUPERBLACKS and SUPERWHITES. Of course, once these values were NORMALIZED, you could very easily get a value above 1 due to the fact that white (685) can divide into 1023 more than once. However, in one of the articles I read (http://magazine.creativecow.net/article/cineon-files-what-they-are-and-how-to-work-with-them), it talks about cineon white values clipping at 13…

Whether something is stored in a linear or nonlinear color space is independent of whether or not there are superblacks or superwhites.

@dudestein 31857 wrote:

MORE ON WHAT I THINK I UNDERSTAND ABOUT COLOR SPACE:
I also think that ‘REGULAR’ digital images (by regular, I mean not originating from film and being scanned into LOG space), such as those rendered out of a CGI package or acquired through a digital camera of some sort have a NON-LINEAR COLOR profile (sRGB gamma correction) embedded in them in order for them to display correctly on our computer monitors and TVs.

sRGB is very common for stuff coming from cameras, but some cameras will shoot to various other color spaces. There’s no guarantee that the image will actually embed an accurate color profile. You may be stuck having to guess. CGI is moving to linear space much more over the last few years.

Most of your Question 4’s are basically unanswerable, as they depend on conflating separate issues of linearity, and the existence of superwhites, etc.

@dudestein 31857 wrote:

4e – So I guess basically, the above questions really boil down to:
What determines whether the digital representation of an image will contain SUPERWHITES and SUPERBLACKS? Is it the COLOR SPACE in which it is stored, the FILE TYPE or a FLOAT BIT DEPTH setting when the image is ‘acquired ‘ rendered’, or all three, or something else I am missing completely? If FLOAT SPACE does have something to do with this, I find that confusing because my understanding of FLOAT SPACE is that you’re simply getting ‘finer steps’ between all your stored values regardless of what COLOR SPACE your image is saved in and its really the COLOR SPACE that determines whether or not you’ll have SUPERBLACKS and SUPERWHITES saved.

This is a better question. 🙂 In the end, what you want to know is when an image has data that is brighter than white or darker than black. The only full answer is that there is no universal guarantee. Ask whoever put data in the file. Only they can be sure what the bits actually represent. Files get written in strange ways with odd, inaccurate headers all the time.

That said, floating point formats always have the ability to store data greater than 1. Cineon files have a standard convention to only use part of the range for “normal” image data. The fact that it is possible to have superwhites in either of these formats is no guarantee that a particular image actually will contain any superwhite data.

It’s also fairly normal in some 8 bit video formats to consider anything below 16 to be superblack. Again, it’s a convention, not a guarantee.

Hopefully, some of this will clear up some of your misconceptions.

[Author]

Posts