From: tflash, tflash@earthlink.net
Date: Thu, Jul 26, 2001, 5:16 PM
RE: [colortheory] Dynamic Range, Bit Depth, and the Histogram
I'm rather confused about the interaction/relationship of these concepts:
Dynamic Range, Bit Depth, and the Histogram.First off, when in 16 bit mode in PS does the histogram indicate how the file maps into 16 bit space, or an 8 bit space?
More specifically, when I capture a raw unmanipulated HDR file from my Leaf scanner, which I believe scans in 14 bit, and bring it into 16 bit space in Photoshop, the tones in the histogram are bunched up. I know, I haven't set endpoints or gamma yet, but is this showing that the dynamic range of the 14 bit data is not as large as that which is allowable in a 16 bit space? Would it look any different if the same film were captured on a similar device that captures at a true 16 bit level?
Or, Does it indicate that the dynamic range of the film scanned was just not as large as that possible even in an 8 bit space? Is it a function of the dynamic range of film itself, the dynamic range of the capture device, or the bit depth of the capture device relative to the space?
Basically, I am confused overall as to how the histogram works in 16 bit mode, in that anything over 8 bits is handled as 16 bit. Thus, does it display 10 bit data differently relative to say, 14 bit, or any other possible comparison of that sort?
Thanks,
Todd Flashner
From: Andrew Rodney, andrew@digitaldog.net
Date: Thu, Jul 26, 2001, 6:01 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
on 7/26/01 3:15 PM, tflash at tflash@earthlink.net wrote:> First off, when in 16 bit mode in PS does the histogram indicate how the > file maps into 16 bit space, or an 8 bit space?
In a way both. The numbers in the Histogram (0 to 255) remain the same. Technically that1s not really correct as there are a whole lot more numbers in a high but file. But consider that Photoshop wisely treats 8 bit files as one thing, ANY file more than 8 bits (10, 12, 14, 16) bits as another. The Histogram would be a mess if we had numbers for 12 bit verses 14 bit verses 16 bit. What Adobe did was just leave everything the same as far as numbers are concerned. The Histogram you see is accurate to the bit depth you are currently in. Turn off cache for Histogram in preferences.
> More specifically, when I capture a raw unmanipulated HDR file from my Leaf > scanner, which I believe scans in 14 bit, and bring it into 16 bit space in
> Photoshop, the tones in the histogram are bunched up.That1s because Leaf provides you untonede (linear) raw data. Newer scanners can provide high bit files that are toned. The Leaf is pretty old stuff so at the time, it was about the only scanner that even provided you high bit data but they just gave it to you untoned. You can pull the histogram and make it look nice and lovely and you1ve just toned it. Since it1s high bit, no damage done. Once you convert to 8 bits, you1ll see a lovely resulting Histogram with no data loss.
> Would it look any different if the same film were captured on a similar device
> that captures at a true 16 bit level?
>
If said scanner also provided you untoned data, no. If it toned the data, it could look a whole lot different. With my Imacon Flextight, I can tone and capture 16 bit data. The software allows me to pull curves and set endpoints and then provides me 16 bit scans. Your Leaf doesn1t provide that option so you get that dark looking file with the Histogram pushed to one side.> Or, Does it indicate that the dynamic range of the film scanned was just not > as large as that possible even in an 8 bit space?
>
Dynamic range has nothing to do with bit depth! They are completely different spec1s. You can have a scanner with 16 bits per color and a dynamic range of 3.3 and you can have a scanner with 12 bits and a dynamic range of 3.8. Bit depth is the number of steps. Dynamic range is the height of the star case. You can have a staircase that1s 20 feet high and have 40 steps. You can have a staircase that1s 30 feet high and have 30 steps.> Is it a function of the > dynamic range of film itself, the dynamic range of the capture device, or > the bit depth of the capture device relative to the space?
>
Film has a dynamic range, so do scanners and digital cameras. It1s the device that plays a role here.> Basically, I am confused overall as to how the histogram works in 16 bit
> mode, in that anything over 8 bits is handled as 16 bit. Thus, does it
> display 10 bit data differently relative to say, 14 bit, or any other
> possible comparison of that sort
>
No, Photoshop deals with 8 bits pre color in one fashion and then everything higher (what I like to call 3High Bit2) as 16 bits pre color. At least that1s what you see in the mode menu. There is simply no reason to break down a high bit file into categories.Andrew Rodney
From: "Maris V. Lidaka, Sr.", mlidaka@ameritech.net
Date: Thu, Jul 26, 2001, 6:33 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
>First off, when in 16 bit mode in PS does the histogram indicate how the
>file maps into 16 bit space, or an 8 bit space?Into 16-bit space.
>More specifically, when I capture a raw unmanipulated HDR file from my Leaf
> scanner, which I believe scans in 14 bit, and bring it into 16 bit space in
> Photoshop, the tones in the histogram are bunched up. I know, I haven't set
>endpoints or gamma yet, but is this showing that the dynamic range of the 14
>bit data is not as large as that which is allowable in a 16 bit space? Would
>it look any different if the same film were captured on a similar device
>that captures at a true 16 bit level?If by bunched up you mean left and/or right ends of the histogram are clean, this is not a function of the bit level but of the range of data - the setting of the black and white points in the Leafscan software.
The 16-bit space results only in smaller gaps between spikes - meaning that the colors captured are closer together than with 8-bit (and with all due respect to Dan Margulis, in the end result this may not matter at all despite what Bruce Fraser says).
>Or, Does it indicate that the dynamic range of the film scanned was just not >as large as that possible even in an 8 bit space? Is it a function of the >dynamic range of film itself, the dynamic range of the capture device, or >the bit depth of the capture device relative to the space?
This is not a function of the dynamic range of the film, even though different film may have different dynamic ranges, or of the capture device, or of the bit depth. This is a function of setting the endpoints in the scan process - whatever dynamic range the film has, and whatever dynamic range the capture device (scanner) has, if the endpoints are set appropriately the histogram will show end-to-end spikes. The spikes will be closer together in 16-bit mode, but the beginning and end of the spikes will be the same.
>Basically, I am confused overall as to how the histogram works in 16 bit >mode, in that anything over 8 bits is handled as 16 bit. Thus, does it >display 10 bit data differently relative to say, 14 bit, or any other
>possible comparison of that sort?This I don't know.
Maris V. Lidaka Sr
From: Raphael Bustin, rafeb@channel1.com
Date: Thu, Jul 26, 2001, 8:14 PM
RE: [colortheory] Re: Dynamic Range, Bit Depth, and the Histogram
>At 05:15 PM 7/26/01 -0400, Todd Flash wrote:
>>I'm rather confused about the interaction/relationship of these concepts:
>>Dynamic Range, Bit Depth, and the Histogram.
>>
>>First off, when in 16 bit mode in PS does the histogram indicate how the
>>file maps into 16 bit space, or an 8 bit space?
>>
>>More specifically, when I capture a raw unmanipulated HDR file from my Leaf
>>scanner, which I believe scans in 14 bit, and bring it into 16 bit space in
>>Photoshop, the tones in the histogram are bunched up. I know, I haven't set
>>endpoints or gamma yet, but is this showing that the dynamic range of the 14
>>bit data is not as large as that which is allowable in a 16 bit space? Would
>>it look any different if the same film were captured on a similar device
>>that captures at a true 16 bit level?
Todd, as I recall, our buddy Austin claims that the Leaf can do perfectly fine scans over a wide range of exposures. This seemed odd to me. But I did say, at the time, that this could only be true if the A/D was using only a small portion of its available input range. So in a way -- your observation about bunched- up histograms makes some sense.
Rafe Bustin
From: tflash, tflash@earthlink.net
Date: Fri, Jul 27, 2001, 3:37 AM
RE: Re: [colortheory] Re: Dynamic Range, Bit Depth, and the Histogram
Thank you Andrew and Maris. You guys seem to be in accord, and what you expressed was in agreement with my assumptions.Rafe, you seem to feel other wise, so I would like to address this to you.
>>> More specifically, when I capture a raw unmanipulated HDR file from my Leaf
>>> scanner, which I believe scans in 14 bit, and bring it into 16 bit space in
>>> Photoshop, the tones in the histogram are bunched up. I know, I haven't set
>>> endpoints or gamma yet, but is this showing that the dynamic range of the 14
>>> bit data is not as large as that which is allowable in a 16 bit space? Would
>>> it look any different if the same film were captured on a similar device
>>> that captures at a true 16 bit level?
> > > Todd, as I recall, our buddy Austin claims that the
> Leaf can do perfectly fine scans over a wide range
> of exposures. This seemed odd to me. But I did say,
> at the time, that this could only be true if the A/D
> was using only a small portion of its available input
> range. So in a way -- your observation about bunched-
> up histograms makes some sense.Could you say more about what makes sense?
This question does in fact stem from a difference of opinion between myself and Austin. He is of the mind that the histogram of the raw (linear) data appears the way it does because it is 14-bit capture mapped into/displayed in 16-bit space. I am of the opinion that it is due to the dynamic range of the data, whether it be due to the DR of the film, or the scanner. He believes the DR of the data is in part a function of the bit depth. He referenced the way higher bit A/D converters allows scanner manufacturers to make extended DR claims.
At least that's the way I interpreted his statements.
You seem to be the only other person that may share his view. Could you expand a bit more as to your reasoning. I think your explanations tend to stay a bit more on point than his.
Todd Flashner
From: Raphael Bustin, rafeb@channel1.com
Date: Fri, Jul 27, 2001, 6:35 AM
RE: Re: [colortheory] Re: Dynamic Range, Bit Depth, and the Histogram
On Fri, 27 Jul 2001, tflash wrote:> Thank you Andrew and Maris. You guys seem to be in accord, and what you
> expressed was in agreement with my assumptions.
>
> Rafe, you seem to feel other wise, so I would like to address this to you.
>
> This question does in fact stem from a difference of opinion between myself
> and Austin. He is of the mind that the histogram of the raw (linear) data
> appears the way it does because it is 14-bit capture mapped into/displayed
> in 16-bit space. I am of the opinion that it is due to the dynamic range of
> the data, whether it be due to the DR of the film, or the scanner. He
> believes the DR of the data is in part a function of the bit depth. He
> referenced the way higher bit A/D converters allows scanner manufacturers to
> make extended DR claims.
>
> At least that's the way I interpreted his statements.
No, I'm not trying to take sides in your dispute with Austin. As you recall, Austin claims (and cites the Leaf manual as proof) that the Leaf can render perfectly fine scans over a wide range of exposures at the scanner-driver level (on the Leaf, these exposures would be measured in milliseconds per scan line, I presume.) This didn't make a lot of sense to me.As a (former) designer of instrumentation circuitry, my goal was generally to set analog gains so that one used as much of the A/D input-voltage range as possible, while also ensuring that over-range and under-range conditions would not occur.
To may way of thinking, the only way that Austin's claim could be true was if the Leaf was only using a small part of the available A/D input voltage range on any given scan. And, voila -- this is what your "narrow" histograms seem to show.
I'm only trying to put "2 and 2 together" here, that's all.
Rafe Bustin
From: Dan Margulis,
Date: Fri, Jul 27, 2001, 8:58 AM
RE: [colortheory] Dynamic Range, Bit Depth, and the Histogram
Todd writes,>>I'm rather confused about the interaction/relationship of these concepts:
Dynamic Range, Bit Depth, and the Histogram.>>Most people would say that these are three totally different concepts, but in fact they are linked by a couple of common factors: 1) the effect of random noise; 2) each one has managed to give birth to a myth that has caused a lot of trouble to the color community.
What's random noise? Well, suppose you have two digital thermometers on your wall. One of them tells you that it is currently 73 degrees. The other one has more LEDs, and tells you that it is currently 72.86473 degrees.
The second one *sounds* more impressive, but in reality no thermometers are that accurate. The last four digits for sure, and probably the last five digits, are meaningless, empty numbers, of no validity at all. And yet they are there.
This example relates to your three terms as follows.
*Dynamic Range* means the ability to discern detail at extremes of lightness, darkness, or color. Positive film, for example, holds detail so subtle in the deepest shadows that no scanner or digital cameras can see it. Drum scanners do slightly better at picking up this detail than even the most expensive CCD devices, so we say that the drum scanners have more dynamic range, though less dynamic range than there is in the film.
The myth is that when the camera or scanner encounters something that's outside of its dynamic range it hits a brick wall and just clips. IOW, assume a digital photograph, or a scan from film, of a black cat at night. If the scanning device really can't resolve the darkest area of the cat because of inadequate dynamic range, some people suppose that what will show up there is pure black. Not so. The scanner won't fail altogether, it will *think* it sees something there and will try its damndest to show it to you. Unfortunately, what it will show you is the last four digits of the thermometer--just random pixels, meaningless noise, not black, but not a cat either.
*Bit Depth* is a measure of how many distinct shades might theoretically appear in a single channel. Each bit that's added doubles the number of potential shades. The norm, 8-bit depth, has 256 possible shades, 9-bit 512, and 10-bit 1,024. When Photoshop encounters a file with more than 8-bit depth, it converts it to 16-bit, which has 65,536 possible shades.
The myth is that expanding a file to 16-bit somehow makes it more accurate. Assuming for the sake of argument that your scanner really can record 1,024 accurate values per channel, going to 16-bit just packs it with the last four digits of the thermometer: useless data, random numbers, mere noise, of no statistical validity, of no benefit to the image. Furthermore, the more I've studied it, the more I doubt that any current scanning device is capable of getting even 500 accurate shades per channel, let alone 65,000.
The *Histogram* is an invention of the evil one for the express purpose of retarding progress in the graphic arts. More people have been deluded by it than by any other feature of Photoshop.
The myth is that a smoother histogram equates to a better-looking image. Since the histogram doesn't reveal anything about the *significant* areas of an image as opposed to the background, it's of extremely limited utility in evaluating image quality. Nevertheless, if you have to guess at which of two versions of an image will look better based solely on a histogram, you'll be dollars ahead in the long run if you bet on the one that's less smooth. The reason, again, is noise. The more random the image, the more it resembles the last four digits of the thermometer, the smoother the histogram will be. In the case of the image of the black cat discussed above, the histogram for the shadows will definitely look better in the CCD scan that is nothing but noise as compared to the drum scanner that captures a certain amount of detail.
Dan Margulis
From: tflash, tflash@earthlink.net
Date: Fri, Jul 27, 2001, 12:29 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
on 7/27/01 8:54 AM, Dan Margulis wrote:> The *Histogram* is an invention of the evil one for the express purpose of
> retarding progress in the graphic arts. More people have been deluded by it
> than by any other feature of Photoshop.
>
> The myth is that a smoother histogram equates to a better-looking image.
> Since the histogram doesn't reveal anything about the *significant* areas
> of an image as opposed to the background, it's of extremely limited utility
> in evaluating image quality. Nevertheless, if you have to guess at which of
> two versions of an image will look better based solely on a histogram,
> you'll be dollars ahead in the long run if you bet on the one that's less
> smooth. The reason, again, is noise. The more random the image, the more it
> resembles the last four digits of the thermometer, the smoother the
> histogram will be. In the case of the image of the black cat discussed
> above, the histogram for the shadows will definitely look better in the CCD
> scan that is nothing but noise as compared to the drum scanner that
> captures a certain amount of detail.Dan,
Surely there is a point of diminishing returns in a combed histogram, no?
I'm thinking of the banding/posterization I see in the smooth tones of my overly manipulated images, and that *adding* noise (dither) is the standard recommendation to ameliorate such a condition.
Lastly, is your claim based solely from the point of view of a image going to press, or does it hold true even through LVT (sorry, I don't know the long form of the acronym) output?
Thanks,
Todd Flashner
From: tflash, tflash@earthlink.net
Date: Fri, Jul 27, 2001, 12:29 PM
RE: Re: [colortheory] Re: Dynamic Range, Bit Depth, and the Histogram
Rafe,My intent was not to ask you to take sides in a dispute. I was looking for a coherent explanation of the basis for his claim. As I am as far from an engineer as an apple is to an orange, I like to put these things into the context of visual references (which can be verbal) and metaphors, while the engineers like to stick with math and the jargon of electronics. I was just looking to get a better handle on Austin's notions from you.
So, while I understand what Maris, Andrew, and Dan have said, I still don't understand what Austin is suggesting. The downside is that I don't know if it's contradicting what the others say or not. This isn't about right and wrong, so much as understanding.
Thanks
Todd
> To may way of thinking, the only way
> that Austin's claim could be true was
> if the Leaf was only using a small part
> of the available A/D input voltage
> range on any given scan. And, voila --
> this is what your "narrow" histograms
> seem to show.
>
> I'm only trying to put "2 and 2 together"
> here, that's all.
From: "J Walton", j.walton@mindspring.com
Date: Fri, Jul 27, 2001, 1:20 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
I don't think Dan's assertion about histograms takes into account overly manipulated images. He was attacking the overuse of the histogram in the context of scanning. Since (from what I've seen) CCD scanners tend to advertise >8 bit input, perhaps that has added to the reliance of histograms (and 16-bit editing) on the low end. Of course, I'm making the assumption that CCDs dominate on the low end.I would have to say that histograms would not be nearly as popular if "Levels" didn't include one by default. Once again, assuming Levels is reserved for the less-experienced (and it is), would explain the popularity of the histogram relative to its benefit.
As to Dan's claim based on output device, I don't think he had one in mind. He's talking about quality of image, not output.
Now let's hear Dan defend himself, wherein all my assertions are reversed....
J
From: Raphael Bustin, rafeb@channel1.com
Date: Fri, Jul 27, 2001, 1:36 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
On Fri, 27 Jul 2001, J Walton wrote:> I don't think Dan's assertion about histograms takes into account overly
> manipulated images. He was attacking the overuse of the histogram in the
> context of scanning. Since (from what I've seen) CCD scanners tend to
> advertise >8 bit input, perhaps that has added to the reliance of histograms
> (and 16-bit editing) on the low end. Of course, I'm making the assumption
> that CCDs dominate on the low end.Histograms are useful, IMHO, for knowing, at a glance, whether your scan is hopeless.
They won't necessarily tell you if it's a good scan, however.
I don't waste much time looking for spikes or combs in the histogram, but if I see pixels stacked up against the wall, at either side, I redo the scan.
Having accepted the scan as "useable," I personally don't bother with the histogram after that. Determination of "usable" generally involves a subjective review of the composite RGB image, and each of the three color channels individually.
rafe b.
From: "Maris V. Lidaka, Sr.", mlidaka@ameritech.net
Date: Fri, Jul 27, 2001, 3:11 PM
RE: Fw: [colortheory] Dynamic Range, Bit Depth, and the Histogram
Agreed, but it is useful in setting the endpoints.Maris Lidaka
>The myth is that a smoother histogram equates to a better-looking image.
>Since the histogram doesn't reveal anything about the *significant* areas
>of an image as opposed to the background, it's of extremely limited utility
>in evaluating image quality.
From: David.Clark@Walsworth.com, David.Clark@Walsworth.com
Date: Fri, Jul 27, 2001, 3:48 PM
RE: [colortheory] Re: Histograms
Dan,I understand what you are saying about histograms being a distraction but I find them to be like monitor color.
Both are pretty good at telling you when something is wrong, but not so good in evaluating correct results.
From: Dan Margulis, Date: Fri, Jul 27, 2001, 7:29 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
Todd writes,>>Surely there is a point of diminishing returns in a combed histogram, no?>>
I'm sure there is at some point, but one would never get there in a real-world photograph. It depends a lot more on how good the data is than on how many points it occupies. Even if we only have 30 possible tones per channel, that's a palette of 27,000 colors, more than enough to create an undetectably smooth image.
>>I'm thinking of the banding/posterization I see in the smooth tones of my overly manipulated images, and that *adding* noise (dither) is the standard recommendation to ameliorate such a condition.>>
Natural photographs carry enough noise to defeat banding. However, if you have done enough retouching so that parts of the picture count as computer art rather than a photograph, particularly if you've completely blurred out an area or added a gradient, and then you intend to apply curves or convert to CMYK later on, sure, convert to 16-bit and you'll get better results.
>>Lastly, is your claim based solely from the point of view of a image going to press, or does it hold true even through LVT (sorry, I don't know the long form of the acronym) output?>>
Well, film recorder then. Assuming there's enough resolution in the file, there wouldn't be a problem *unless* somebody is motivated to scan the resulting film at >500% magnification, in which case the scan might have what seem to be sharpening artifacts. However, in the year 2001 this is not a particularly likely workflow.
J writes,
>>Now let's hear Dan defend himself, wherein all my assertions are reversed....>>
No, you got them right.
Rafe writes (and David similarly),
>>Histograms are useful, IMHO, for knowing, at a glance, whether your scan
is hopeless.>>$&*(%%@#!!! You mean you can't tell whether an image is no good by *looking* at it? You mean that if you have a scan that looks good on screen, but the histogram says it's hopeless, you throw it away???
Dan Margulis
From: Bob Tyson, bobicho@earthlink.net
Date: Fri, Jul 27, 2001, 8:25 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
> >>Histograms are useful, IMHO, for knowing, at a glance, whether your scan
>is hopeless.>>
>
>$&*(%%@#!!! You mean you can't tell whether an image is no good by
>*looking* at it? You mean that if you have a scan that looks good on
>screen, but the histogram says it's hopeless, you throw it away???What, you mean you actually LOOK AT AN IMAGE, Dan?????? Shocking! And so is your implication . . . he said, muttering to himself as he picked up yet another PRINT . . . . .
Bob Tyson
From: Raphael Bustin, rafeb@channel1.com
Date: Fri, Jul 27, 2001, 10:47 PM
RE: Re: [colortheory] Dynamic Range, Bit Depth, and the Histogram
>>>Histograms are useful, IMHO, for knowing, at a glance, whether your scan
>is hopeless.>>
>
>$&*(%%@#!!! You mean you can't tell whether an image is no good by
>*looking* at it? You mean that if you have a scan that looks good on
>screen, but the histogram says it's hopeless, you throw it away???
Ummm. Isn't it you who says "don't trust your monitor?"The histogram is at least a "by the numbers" approach. Damn straight, if even 2% of the pixels have gone to pure black or pure white, I re-scan.
Or are you being facetious in this reply?
rafe b.
From: David.Clark@Walsworth.com
Date: Mon, Jul 30, 2001, 9:46 AM
RE: [colortheory] Re: Histograms
>>>You mean that if you have a scan that looks good on
screen, but the histogram says it's hopeless, you throw it away???<<<Well, your book did teach me better than that but I need to work with a lot of neophyte users. Heavy reliance on the histogram is a valuable crutch for beginners, especially if you need to work with them over the telephone or email.
From: Dan Margulis,
Date: Tue, Jul 31, 2001, 7:10 PM
RE: [colortheory] An 8-bit, 16-bit test
Folks,Todd Flashner has kindly contributed a set of files that cast light on a topic that has gotten a fair amount of discussion on this list.
Photoshop allows us to work on files either in 8-bit (256 levels per channel) or 16-bit (65,536 levels). Files are often delivered from a scanner in some kind of intermediate mode which theoretically makes 16-bit a more favorable way to correct. Curves, levels, and certain filters work in 16-bit, but many other functions, such as layers, don't. Also, files have to be converted to 8-bit before they can be output.
For some years a number of persons have been assuring us that doing as much editing as possible in 16-bit pays quality dividends, in spite of the fact that 16-bit files take up twice as much disk space and twice as long in computing time. To demonstrate this superiority, they show us a bunch of histograms, but never actual images that support the claim.
I have experimented a good deal with this and have never been convinced that there's any merit to it. So, for several years, I've been asking those who claim that it works to provide me a few sample images that they think edit better in 16-bit. Mysteriously, the people who are most vociferous about 16-bit have never been able to come up with such samples.
The supporters have always theorized that the advantage of 16-bit would be greater as the moves made to them were more drastic (or as successive changes were applied to the same image). This has posed a problem in testing the hypothesis, because if the pictures are good to start with it's hard to come up with real-world sorts of changes that would really torture the image. OTOH if the images are bad to start with, it's likely that the source of the scan is so bad that the extra bits would be worthless, which would be unfair to those who believe that 16-bit works.
This is where Todd's files were so helpful. They were 16-bit files scanned from film in native mode from a high-quality Leaf scanner. Anyone who is sane uses the Leaf software to adjust the range of the scans before sending them to Photoshop, but Todd deliberately did not do that and gave me the raw scans, which were excruciatingly dark and flat. The corrections needed to bring them to normalcy were therefore much, much more drastic than one would expect to see in the real world. For example, in real-world images, I've sometimes applied more than one set of curves, but never because it was *impossible* to do otherwise. Two of the images here were so flat that I was forced to use Auto Curves to set highlight and shadow before applying a second curve, because otherwise parts of the curve would have had to be more vertical than Photoshop will allow.
The subjects were two B/Ws, two color negs, and one chrome. In each case, I started with two copies of the 16-bit file. One of them I corrected entirely in 16-bit (saving all curves along the way) and then converted to 8-bit as if for printing. The other I converted immediately to 8-bit and repeated all of the curves. Then I compared the two, composite and channel by channel, at several magnifications on screen.
Bottom line: even in these very extreme cases, there's no benefit to working in 16-bit (although there's no loss either). Of the five images (I didn't bother to work one of the chromes) only one had a difference significant enough that I thought someone might notice it on output. In that case, the 8-bit file was certainly better than the one done in 16-bit. Two others had minor differences that wouldn't affect output, but that conceivably could play a role if there were further major changes. Of these two, I preferred the 16-bit version in one case and the 8-bit in the other. The final two files had differences that were negligible and in my view no amount of further correcting would reveal any quality difference between them.
What I learned from the exercise was that the impact of the extra bits is somewhat analogous to (although not nearly as pronounced) as scanning at higher resolution. Scanning in high resolution emphasizes smoothness and consistency; lower resolutions create more action and variability. Which one is preferable will depend on the image. Some fashion shops, for example, routinely scan female faces at higher resolutions than male ones.
In the case of Todd's files, the versions corrected in 16-bit always wound up (at least if examined at high magnification) a little smoother than their 8-bit counterparts. The trouble is, that frequently isn't what you want. The only one of the five that I felt the 16-bit worked better on was an image that was fairly noisy even in 16-bit. Correcting it in 8-bit aggravated the problem slightly.
Similarly, the image that worked marginally better in 8-bit was a fairly smooth capture to begin with, with a lot of superfine detail (grass, for example) that can't be fully resolved. In such cases, the extra action in the 8-bit file is perceived as more detail in the grass and will be preferred by almost anyone.
This image, incidentally, was a great illustration of what a crock the "combed histogram is bad" idea is. After correction in 8-bit, the green channel (which supplies more than half the contrast in an image) had only around 25 values in it.
I have no problem with sharing these images and my results if Todd doesn't, and if Todd doesn't object I'll probably print one or two of them in Professional Photoshop 7. However, the files are 20-50 mb apiece, so they can't go on-line (JPEGging would destroy the minor differences between versions). If somebody seriously wants them I'd have to burn CDs. I'm not, however, willing to do that or ship them to anyone for free. If there's interest and Todd's OK with it I'll figure out the lowest sane cost for duplicating and shipping.
In a second post I have a technical description of what was done to the images and why I concluded that some were better than others. Thanks to Todd for making the test possible.
Dan Margulis
From: Dan Margulis,
Date: Tue, Jul 31, 2001, 7:48 PM
RE: [colortheory] 16-bit test technical
The following is the technical information on the 8-bit vs. 16-bit test I wrote about in the accompanying post. If that post didn't interest you,this one certainly won't either.Dan Margulis
*********************************
The B/W original images were around 20 mb apiece. The color originals varied from 28 to 52 mb.Except as noted, I always started with two files, one the original 16-bit and one converted immediately to 8-bit. I saved all modification settings and applied the same ones to both images. At the end of all corrections I converted the 16-bit file to 8-bit and compared it to the one in which all corrections were done in 8-bit. All corrections in color files were done in RGB. I was using Apple RGB as the working space but do not believe the choice of RGB space would have affected the result.
*********************************
1) TREES Image:Description: This is a B/W negative of a very detailed tree, devoid of leaves, against a cloudy sky.
Procedure: Once the file is inverted, the maximum shadow is 42%. I applied a single curve with a single point, raising 15% to 42%, drastically accentuating highlight contrast while bringing the shadow to around 95%. At this point I made two extra copies of the image, converted the 16-bit one back to 8-bit, and compared it carefully to the one done only in 8-bit. After this, I went to the copies, sharpened the 16-bit one in 16-bit and then converted it to 8-bit to compare to the 8-bit file similarly sharpened.
Result: In the unsharpened version, the two were indistinguishable at 100% magnification on screen. At 200% magnification in certain parts of the sky one could see that the files were not identical, but not distinguish which was which. At 300% in certain parts of the sky but not in others, a very slight additional texture identified the version done in 8-bit. No variation could be detected in the tree at any magnification. Comparing the two sharpened versions gave similar results.
Evaluation: The two sets of files would print equally well under any conceivable output condition and with any conceivable set of future corrections.
*********************************
2) SNAKE image:Description: This is a scan of positive color film of a piece of jewelry with a background that gradates from light gray to black. The scan is very dark, but it's easy to see what it's supposed to represent.
Procedure: The curves used to open this up were drastic but seemed to yield satisfactory results and a smooth transition in the background. This was the highest-quality original scan of the five.
Result: In composite mode there was no discernible difference between the two versions at magnifications up to 400%. At 300% one could see that individual channels were different but the difference seemed to be random, neither version sharper or noisier than the other.
Evaluation: The two files aren't pixel-for-pixel identical but they're even closer than in example 1. For any conceivable purpose they are the same file.
*********************************
3) OW2 imageDescription: This is a color negative of a cement path leading through a garden with lots of greenery and bright flowers. The negative image is so dark and flat that I cannot tell what the subject of the image us until later in the correction.
Procedure: In order to torture the image I deliberately used an inferior method to correct it. I first inverted the image. As no compensation had been made for the orange mask of the negative, the blue channel of the inverted image was nearly nonexistent (darkest point: 242B) and the green channel not much better. The image was so light (measured shadow at 86L; pure white is 100L and pure black is 0L) that it was impossible to make a curve steep enough to bring out the highlight to quartertone area. It was necessary instead to run Auto Curves followed by a second curve to correct the color.
Result: After the second curve, according to the histogram, the version done in 8-bit had around 25 actual levels in the green channel and around 14 in the blue. Naturally the version done in 16-bit had 256 values per channel. At 100% magnification I could detect no difference in the composite file on screen, but at 200% it was fairly obvious. At 100% the difference could be seen in the green and blue channels but not the red. The blue channel was extremely choppy in the 8-bit version but the green channel seemed to show extra texture in the cement path and to a lesser extent in the greenery.
Evaluation: The version corrected in 8-bit is technically superior. If they were printed at the appropriate size for this resolution nobody would see the difference. However, under certain circumstances the 8-bit version might give a better result. Given that the green channel is the most important one for establishing contrast, this is a shocking verification of the combed-histogram-means-nothing rule. The main reason for the success of the 8-bit correction here, in my view, is that the original image was fairly soft. The relatively harsh variation of tone in the green channel therefore seems to the viewer to be additional detail, whereas if the original had been more sharply focused, the viewer might take it as noise.
*********************************
4) MANHATTAN SHADOW image:Description: This is scanned from negative B/W film. It shows some skyscrapers against a background of a cloudy sky. Even when inverted, the original was so flat that I couldn't discern what it was a picture of until applying curves. Todd supplied a thumbnail indicating that he wanted to have the quartertone particularly darkened so that there would be more contrast between the clouds and the sky.
Procedure: The highlight move is so drastic that it could not be done in a single curve. I had to use Auto curves followed by a second, very steep curve, far more inclined than the one used in example 3. Because I expected the two versions to be drastically different, I did a third version as follows: I converted the 16-bit file to 8-bit, and reconverted it immediately to 16-bit. This move threw away all but 256 values in each channel. I then rotated the file slightly twice before returning it to its normal orientation. (This causes each pixel to get a certain percentage of the values of its neighbors and thus insures that there will again be 15,000+ separate values in the channel; however, anything other than the original 256 can be considered incorrect.) I then applied the same moves to this faux-16-bit file as to the other two.
Result: When the range was opened up, this image was fairly noisy in all three versions. The 8-bit version seemed crisper but the noise was worse. The faux-16 bit version had less noise than either.
Evalution: If absolutely sure that this were to be printed with the conventional rule of resolution=2x screen ruling, I'd expect the 8-bit version to print slightly better. But too many other things might happen to the image, so I'd prefer the 16-bit version. The faux-16-bit version, however, was best of all.
*********************************
5) OW STEPS image:Description: This is a color negative of a garden scene with a concrete staircase in the foreground. It is similar to #3 and again the negative is extremely dark.
Procedure: Rather than invert the image as in #3, I did what I would ordinarily do, I converted the RGB to LAB, applied a set of curves there to eliminate the orange mask, reconverted to RGB and applied a second curve to try to regularize the color. Once again, these curves were considerably more drastic than one would ever expect to see in real life.
Result: I am the proud discoverer of a bug. Although the curves were exactly the same, there was a pronounced color variation between the one done in 16-bit throughout as opposed to 8-bit throughout, particularly along the yellow-blue axis. Because I was sure this was impossible, I redid the test four times with the same result. The luminosity of the image was unaffected. The 8-bit file was slightly grainier but the image was not particularly noisy to begin with. This had the effect of making the 8-bit file seem sharper.
Evaluation: This is the only one of the five images that I think there is a chance somebody might distinguish a difference (other than color) if the two were printed. The 8-bit correction was very slightly, but discernibly, better than when it was done in 16-bit. The extra variation shown in the 8-bit file fools the viewer into imagining extra detail.
From: Bob Tyson, bobicho@earthlink.net
Date: Tue, Jul 31, 2001, 7:38 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Wow! Thank you Todd and Dan. I'd been curious about this 8/16 bit stuff, too.I haven't read your post detailing your procedure, Dan, but let me ask one question before I do. You say, here, that you applied curve(s) to the 16-bit files, saved the curves, and processed the 8-bit files the same way.
Does that mean you applied the curves saved from the 16-bit files, to edit the 8-bit ones? Just for a clarification.
Your work confirms an odd hunch that has been crawling up my spinal cord for some time, and I can't claim to have investigated nor understood the intricacies of scanning so much. I just try to USE the damned thing, as-is.
Folks, with this great clearing of the air (a premature conclusion, I'm sure!) I have this comment, based on my experience over the past almost ten years:
After using an HP ScanJet IIcx for six years, at 300 DPI maximum and 8/24 bits, I moved "up" to a PL-III, 1200 DPI and 14/42 bits. But the REAL difference I have found is in ONE single area: the capability of the PowerLook to read far greater densities in negatives or transparencies than the HP ever could.
Within the limitations of each, either can and has produced excellent scans, some of which have been used for very successful offset printing, but most, in my work, for inkjet prints.
Hey-- free at last! 8-bit and 600 DPI, maybe even 430 or (gasp) 720, here I come.
Bob Tyson
From: Chris Brown, cb@chrisbrownphoto.com
Date: Tue, Jul 31, 2001, 8:36 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
> What I learned from the exercise was that the impact of the extra bits is
> somewhat analogous to (although not nearly as pronounced) as scanning at
> higher resolution. Scanning in high resolution emphasizes smoothness and
> consistency; lower resolutions create more action and variability. Which
> one is preferable will depend on the image. Some fashion shops, for
> example, routinely scan female faces at higher resolutions than male ones.Well, what I learned proved to me that this list is an excellent and valuable resource.
Many thanks to Dan and Todd.
Chris
Chris Brown Photography
http://www.chrisbrownphoto.com
Vox: (217) 356-0540 * Fax: (217) 356-1394
From: tflash, tflash@earthlink.net
Date: Tue, Jul 31, 2001, 8:42 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dan,Thanks for the report.
I look forward to seeing what you did to the images, because in my experiments I reached a different conclusion, but with caveats: I was viewing the same files at 2x the resolution I provided you, I made my evaluations onscreen, rather than in print (what did you output them on?), and everyone should know, these raw scans look horrible before adjustment. It is a torture test for 8-bit manipulation!
As you know, I only found one image that I could see no difference in, and all the others I thought looked better in 16-bit. But I was looking for smoothly graduated tones; you may have been looking for "drama" or "energy". In any case, these were probably far from real world examples to judge by, and I too concluded that for real world situations there should be no benefit to using 16-bit files. Certainly not enough benefit to justify their added size and lack of PS support.
Speaking of Real World, since giving you those images, I learned a trick from Bruce Fraser, who himself had a Leaf once. I think you might find it interesting. It follows below.
*********
Your problem stems from the way the Leaf software writes out 16-bit files -- as you've noticed, all the data is bunched up in the shadow end.
Here's what I do with my legacy Leaf 16-bit images: it works fairly well.
1.) Create a linear-gamma profile. I use Adobe RGB as a starting point. With Adobe RGB set as the working space, in Color Settings, choose Custom RGB, set the gamma to 1.0, call it something sensible, then save it using Save RGB. Then restore your working space of choice.
2.) when you open the Leaf image, assign the linear gamma profile.
3.) Go into Curves, set the input value to 4, and the output value to 14.
4.) Use Convert to Profile to convert the image to working RGB.
5.) Edit as necessary.
It's a bit of a kludge, but it works quite well.
Bruce Fraser
********
Dan, I'm not sure I am comfortable having my images distributed to all takers. Perhaps a better solution would be to provide me with a few images, which I will gladly scan for you. The book would be fine however, provided copyright credit is given.
Thanks again,
Todd Flashner
From: "Maris V. Lidaka, Sr.", mlidaka@ameritech.net
Date: Wed, Aug 1, 2001, 4:48 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Todd,It would be helpful if you would reconsider and release the images, and that Dan and you would provide records of the adjustments you respectively applied, to help in laying to rest this perennial issue of 16 v 8 bits.
I doubt that he has the time or inclination to repeat the exercise with new images that he would have to provide to you - his challenge historically has been for someone (else) to provide an image and show an improved result using 16-bit adjustments as his own experiments have shown no improvement:
" I have for several years asked here and elsewhere for those who advocate these methods if they might not be able to provide me, say, two or three sample original high-bit images, with a record of what moves were applied to them, so that I could verify that there is a quality gain, however slight, in applying them to a 16-bit image as opposed to an 8-bit one."
Maris Lidaka
From: "Maris V. Lidaka, Sr.", mlidaka@ameritech.net
Date: Wed, Aug 1, 2001, 2:38 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
I'm interested - please let me know when you have Todd's OK and the price.Maris Lidaka
From: Robin Forslund, rforslund@mckaypress.com
Date: Wed, Aug 1, 2001, 10:09 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
One of the issues we have encountered with our CTP device (Creo Trendsetter driven by Delta RIP with IS screening) relates to 16-Bit Screening.Since the Creo Trendsetter runs at 2400 dpi, and the majority of what we run through it runs at 175lpi, we're only getting 188 shades of gray. As you might imagine, this sometimes complicates the printing of gradients.
Heidelberg has sent us a PostScript Header File to attach to the print ques that enables 16-Bit Screening. Since the available number of grays is so much higher for 16-Bit Screening, the loss of a few shades of gray off the top is not as significant. At least, that is the theory they are pitching to us.
So far, we have not been able to get the 16-Bit Screening PostScript Header file to be recognized by the Delta RIP. The upgrade to 7.0 is due to be installed this week or early next week, and they hope it will be friendlier to the 16-Bit screening.
My question: since you have seen that 16-Bit mode does NOT help with regard to photoshop files, is there any hope for the "16-Bit Fix" they have proposed for our 188-Shades-of-Gray problem? Or are they grasping at straws here? i.e., Will it be worth the trouble to install and test this feature?
I appreciate any input you can provide.
Thanks! :-)
--Robin
From: Dan Margulis,
Date: Wed, Aug 1, 2001, 11:15 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Todd writes,>>I look forward to seeing what you did to the images, because in my experiments I reached a different conclusion, but with caveats: I was viewing the same files at 2x the resolution I provided you, I made my evaluations onscreen, rather than in print (what did you output them on?),>>
No need for any output--the differences can be evaluated accurately on screen.
>>As you know, I only found one image that I could see no difference in, andall the others I thought looked better in 16-bit. But I was looking for smoothly graduated tones; you may have been looking for "drama" or "energy".>>
More precisely, the appearance of sharpness or detail. The 8-bit images look very marginally sharper, the 16-bit ones marginally softer. The differences are irrelevant in the real world since one can compensate in either direction.
>>Speaking of Real World, since giving you those images, I learned a trick from Bruce Fraser, who himself had a Leaf once. I think you might find it interesting. It follows below.>>
Using a false profile is a valuable trick but its real utility is not for the occasional Leaf user but rather for the rest of us who from time to time get presented with work from clients who are not professional photographers and have managed to create a picture just as dark as any of yours without the aid of Leaf software. This is the whole topic of my column this month, which is called "Fate and the False Profile."
Dan Margulis
From: tflash, tflash@earthlink.net
Date: Wed, Aug 1, 2001, 1:17 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
> Todd,
>
> It would be helpful if you would reconsider and release the images, and that
> Dan and you would provide records of the adjustments you respectively
> applied, to help in laying to rest this perennial issue of 16 v 8 bits.Maris,
I appreciate your point, but I provided Dan the images for his own use, not to distribute freely. These are large files. I am amenable to small JPEGs on a website, or book reproduction, but not distribution. Please understand my point of view. I might have chosen different images to begin with, had I known this would be Dan's intention.
However, I extend my offer to any (not all) of you. Provide me with up to 6 pieces of film, and some Zip100 disks, or one DVD-RAM disk (sorry, I don't have a CD burner), and I'll return your film with the scans on the disks, and you can distribute them. Everybody can do their own moves, and see what they think. At full resolution, color HDR scans can take 45 - 60 minutes ea., so this is not a proposition I take lightly, but I think it's the best solution.
While I trust the difference in our Photoshop manipulations made a difference in our judgements, as well as the fact that I worked with files about twice the size, I think the difference comes from what we were looking for, and Dan's much more learned eye. If there is ever a dispute between Dan's judgement and mine, go with Dan's - I'm fairly new to the digital world.
My moves consisted of duplicating the file and converting one to 8-bit, then doing the following moves to each. Invert (negatives), auto levels, a levels gamma slider adjustment, and a gray eyedropper sampling (for color negs and chromes). That's all - down and dirty.
What I found was that details did hold up well, whereas smooth gradients broke apart and posterized. In my product shots, the jewelry held up well, but the graduated light on the seamless sweep broke apart. In a prior post Dan wrote that the broken histogram doesn't take into account what is the subject of the photo, vs the background. This is such a case, where the subject held up well, and the background not.
In landscapes, the foliage held up well, but the subtle transitions of depth in the sky broke up. In some cases, I see how this "breaking up" could be appreciated as a reduction of noise (i.e., grain flattened out).
In the end, I think the reason Dan and I drew different conclusions was, A) different methodologies, but more importantly, B) Dan was looking for which looked better (smart), while I was primarily looking to see if the 8-bit image looked *different* from the 16-bit version.
My own conclusion was that in one case I could see no difference, but where I did see a difference, I preferred the 16-bit version. This may have been more out of playing safe, than sound judgement. But I also concluded that on most anything resembling a real world scenario, little difference would be noticed.
If anybody would like to provide me with images for distribution, please contact me and I will provide you with my mailing info off-list.
Sorry for the inconvenience.
Todd Flashner
tflash@earthlink.net
From: Dan Margulis,
Date: Wed, Aug 1, 2001, 1:48 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Robin writes,>>Since the Creo Trendsetter runs at 2400 dpi, and the majority of what we run through it runs at 175lpi, we're only getting 188 shades of gray. As you might imagine, this sometimes complicates the printing of gradients. Heidelberg has sent us a PostScript Header File to attach to the print queues that enables 16-Bit Screening. Since the available number of grays is so much higher for 16-Bit Screening, the loss of a few shades of gray off the top is not as significant. At least, that is the theory they are pitching to us.>>
That isn't a particularly good explanation on their part, since nothing about the input gives you more gray shades on output. Nevertheless, there is some chance that this could be effective.
>>My question: since you have seen that 16-Bit mode does NOT help with regard to photoshop files, is there any hope for the "16-Bit Fix" they have proposed for our 188-Shades-of-Gray problem? Or are they grasping at straws here? i.e., Will it be worth the trouble to install and test this feature?>>
The real question is whether your gradients are rough-looking in general or you are getting irregular banding. I suspect the latter, in which case the 16-bit input might help.
This is an entirely different issue than the color correction one. Assume for a moment that you are running 150 lpi, meaning that you have a full 256 shades available. The input customarily has 256 shades also, but unfortunately they won't be quite the same 256, and two different input values may round off to the same output one.
IOW, it's like shooting 256 marbles in the general direction of 256 holes. The fact that two marbles may wind up in the same hole isn't an issue; the problem is that one of the 256 holes will therefore wind up empty. In photographs that isn't a catastrophe but in a gradient it's a killer, and will cause banding.
Heidelberg's idea is to prevent empty holes from occurring by shooting 65,536 marbles at the 256 holes. That will definitely work--provided there were empty holes to begin with. If it was 256 input values recomputed to 256 output values, there would be empty holes for sure. If, as in your case, it's 256 input values recomputed to only 188 output values, there may not be any empty holes for these marbles to fill up.
It's a bit more complicated than that when we consider gradients that don't fill the entire range, but the idea will still be the same. The 16-bit input has a chance of helping you, but it would help somebody who was running at 150lpi screen a lot more.
Dan Margulis
From: Gordon Pritchard, gordon_pritchard@creoscitex.com
Date: Wed, Aug 1, 2001, 4:08 PM
RE: RE: [colortheory] An 8-bit, 16-bit test
RE: Robin writes,
> > >>Since the Creo Trendsetter runs at 2400 dpi, and the majority of what
> we run through it runs at 175lpi, we're only getting 188 shades of
> gray. As you might imagine, this sometimes complicates the printing
> of gradients. Heidelberg has sent us a PostScript Header File to attach to
> the
> print ques that enables 16-Bit Screening. Since the available number
> of grays is so much higher for 16-Bit Screening, the loss of a few
> shades of gray off the top is not as significant. At least, that is
> the theory they are pitching to us.>>
>
> That isn't a particularly good explanation on their part, since nothing
> about the input gives you more gray shades on output. Nevertheless, there
> is some chance that this could be effective.Robin,
The CreoScitex Trendsetter running at 2400 dpi has no problem generating all the screen tint values that you require at all supported screen frequencies (currently 450 lpi depending on media). The number of shades avaliable to you is dependent on your front end and choice of screening method. Many people believe that the absolute number of tones that can be reproduced with a digital halftone screen is equal to the number of spots in a halftone cell, plus one -- represented by the formula: (dpi/lpi)2+1. As a result of this misconception, it is widely thought that a lower-resolution (e.g. 2400 dpi) imaging device cannot reproduce a full range of tones at higher linescreens (e.g. (2400dpi/300lpi)2+1=65 tones). Under such constraints tonal reproduction can be inaccurate and uneven causing visible shade-stepping in gradients. This is also known as banding or contouring, where the color "steps" abruptly from one shade to the next without a smooth transition. Many people also believe that PostScript only supports an upper limit of 256 greylevels. As a result of this misconception, some people believe that it is not possible to achieve more than 256 greylevels in a PostScript workflow, and simply do not believe claims to do otherwise. Reality: Some modern RIPs use the supercell, a grouping of many halftone cells, to build screens with more accurate angles and screen rulings. Supercells typically contain many more pixels than an individual halftone cell, and subsequently can be used to represent many more greylevels or tones than a single halftone cell. By activating pixels in neighbouring halftone cells at different grey levels, a supercell can be built with greylevels far in access of 256. Even our base level Allegro RIP, for example, has the capability to render 4096 greylevels, regardless of addressability and screen ruling. However, for practical purposes the RIP is shipped with this feature set to 1024 as this meets the tonal needs of the highest quality printers. CreoScitex TurboScreening solves the grey-levels issues by increasing the addressablity in the fast scan direction of the output device to deliver extra grey levels at the halftone dot level.
In any case the Trendsetter imaging resolution is not the limitation.
thx, gordo
Gordon Pritchard
Commercial Print Specialist
CreoScitex
Vancouver Canada
T: 604.451.2700 ext 2870
C: 604.351.2437
gordon_pritchard@creoscitex.com
http://www.creoscitex.com
From: "Maris V. Lidaka, Sr.", mlidaka@ameritech.net
Date: Wed, Aug 1, 2001, 5:22 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
I appreciate your forthright response.I won't be taking you up on your offer, though, as for purposes of this
challenge I don't think a lengthy HDR scan is necessary, but I'll see if I
have some images I can scan and submit myself somewhere down the road.My thanks to you for submitting the images to Dan, as even his and your
descriptions of the results are of value.Maris Lidaka
>I appreciate your point, but I provided Dan the images for his own use, not
>to distribute freely. These are large files. I am amenable to small JPEGs on
>a website, or book reproduction, but not distribution.
From: Dave Badger, dbadge@worldnet.att.net
Date: Wed, Aug 1, 2001, 9:35 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
From: tflash
>
>... and I too concluded that for real world situations there should be no
> benefit to using 16-bit files. Certainly not enough benefit to justify their
> added size and lack of PS support.After reading all the post I am now too convinced that 16 bit files are of
no great use. But...Bob Tyson writes:
> After using an HP ScanJet IIcx for six years, at 300 DPI maximum and
> 8/24 bits, I moved "up" to a PL-III, 1200 DPI and 14/42 bits. But the
> REAL difference I have found is in ONE single area: the capability of
> the PowerLook to read far greater densities in negatives or
> transparencies than the HP ever could.Could someone please explain why all scanners need to capture in 10-16 bits these days. The SilverFast manual seems to say that the extra bits allow you to set the tonal range in the software while "throwing away" the bits (tonal areas?) that you don't need to capture. This apparently allows the tonal range you do capture to have the full 256 steps per channel imported into Photoshop.
Hows does Photoshop know how to map all those extra gray tones from high-bit scanners (or is the mapping always done in the scanning software) and do more scanner bits translate into greater shadow detail? Or does it just add more digits (noise) to each step. If so, why the need for high bit scanners and cameras?
Dave Badger
From: "SUSAN & JOHN OPITZ", jas10286@earthlink.net
Date: Wed, Aug 1, 2001, 11:04 PM
RE: [colortheory] RE: 8 bit vers.16 bit
Maris V. L., Sr. wrote:> would be helpful if you would reconsider and release the images, and that Dan and you would provide records of the adjustments you respectively applied, to help in laying to rest this perennial issue of 16 v 8 bits.
I doubt that he has the time or inclination to repeat the exercise with new images that he would have to provide to you - his challenge historically...ect.ect. >
I second that.
John Opitz
From: Dan Margulis,
Date: Thu, Aug 2, 2001, 9:12 AM
RE: RE: [colortheory] An 8-bit, 16-bit test
Gordo writes,>>Many people believe that the absolute number of tones that can be reproduced with a digital halftone screen is equal to the number of spots in a halftone cell, plus one -- represented by the formula: (dpi/lpi)2+1. As a result of this misconception, it is widely thought that a lower-resolution (e.g. 2400 dpi) imaging device cannot reproduce a full range of tones at higher linescreens (e.g. (2400dpi/300lpi)2+1=65 tones).>>
There is indeed a common misconception that if one constructs a dot to a maximum of 1/300" width across a grid of spots of 1/2400" width, that one can only fit eight spots across the dot. Further, many operate under the delusion that 8*8=64.
>>Reality: Some modern RIPs use the supercell, a grouping of many halftone cells, to build screens with more accurate angles and screen rulings. Supercells typically contain many more pixels than an individual halftone cell, and subsequently can be used to represent many more greylevels or tones than a single halftone cell.>>
Not reality, but illusion. Nothing wrong with that: lots of what we do is illusion, ranging from the "grays" that we manage to portray with only black ink and white paper to the "detail" shown in my 8-bit manipulations that's "missing" in the 16-bit versions.
In one of the images in my test I started with an 8-bit file, changed it to 16-bit, then rotated it a couple of times to create more apparent values in the channel. Thus every pixel was slightly affected by the values of its neighbors. A mung-and-blur method to be sure, but it worked: the result was better than either an 8-bit or 16-bit original, because apparent noise was reduced.
Something highly similar has historically been necessary in PostScript workflows because the traditional way: 256 levels input, 256 output causes banding in smooth gradients unless countermeasures are taken. Munging and blurring is a good way. It eliminates the banding and probably isn't noticeable in other contexts. This is particularly so at a very fine screen, where the impact of individual dots isn't that big.
Scitex has been the traditional leader in developing such undetectable illusions, but other vendors have competing products. They are customarily called something like Precision Irrational Turbotangent Accurate Rendering Technology, so that we will realize it isn't just munging and blurring.
>>CreoScitex TurboScreening solves the grey-levels issues by increasing the addressablity in the fast scan direction of the output device to deliver extra grey levels at the halftone dot level. In any case the Trendsetter imaging resolution is not the limitation.>>
Perhaps in the case where only CreoScitex equipment is used, its imaging resolution is not a limiting factor. One suspects that Heidelberg might take a different view of the whether it is limiting otherwise.
Dan Margulis
From: "Bruce J. Lindbloom", blind@picto.com
Date: Thu, Aug 2, 2001, 10:30 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dan,A crucial piece of information missing from your tests has to do with the number of meaningful image bits vs. the number of noise bits in the original 16-bit scans. Imagine the hypothetical case where there are 8-bits of image data and 8-bits of noise. This case could easily lead an investigator to the conclusion you drew: 8-bits is just as good as 16-bits. I'm not saying that's the case with your test, but in order for the conclusion to be meaningful, this measurement must be made. And you cannot simply rely on the scanner vendor's spec sheet to provide it.
Since the original images will not be made publicly available, I guess we need to rely on you to make this determination (of the 16-bits of data from the scanner, how many bits contain actual image information and how many are either empty or contain just noise?). I would be interested in learning what you find, and also in learning the method you use to find it.
--
Bruce J. Lindbloom, Pictographics Intl. Corp.
From: Dan Margulis,
Date: Thu, Aug 2, 2001, 11:13 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dave Badger writes,>>How does Photoshop know how to map all those extra gray tones from high-bit scanners (or is the mapping always done in the scanning software) and do more scanner bits translate into greater shadow detail? Or does it just add more digits (noise) to each step.>>
This is a question of dynamic range, not bit depth. If the scanner can't differentiate the darkest shadows it won't matter whether it has 8 bits to describe its noisy findings or 8,000. If a scanner does see the distinctions 6 bits will do fine.
>>If so, why the need for high bit scanners and cameras?>>
It's traditional for higher-end devices to be able to do this, even if it's not obvious that it's effective. It's conceivably useful in certain cases, and it's no inconvenience to us to have them use high-bit internally even if we want them to deliver 8-bit files.
Dan Margulis
From: Martin Orpen, orpy@idea-digital.com
Date: Thu, Aug 2, 2001, 11:21 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
on 2/8/01 2:08 pm, Dan Margulis at wrote:> Something highly similar has historically been necessary in PostScript
> workflows because the traditional way: 256 levels input, 256 output causes
> banding in smooth gradients unless countermeasures are taken. Munging and
> blurring is a good way. It eliminates the banding and probably isn't
> noticeable in other contexts. This is particularly so at a very fine
> screen, where the impact of individual dots isn't that big.PostScript as a page description language has never been limited to 256 gray levels.
Admittedly Adobe's own Level 1 RIPs did have this limitation, but the language itself does not impose any limits. (Improvements to the language incorporated in PostScript Level 2 included the ability to move data with 12 bits per channel directly to the "image" operator.)
A large amount of pre-press work in the UK and US is done using the Harlequin RIP engine (always hidden behind somebody else's logo). All versions of the Harlequin RIP are able to handle (or generate) additional gray levels.
Scitex are the latecomers here - I wonder if they have an estimate of how many pages of film were wasted after Quark XPress added the capability to generate graduated tints? :-)
--
Martin
Idea Digital Imaging Ltd
http://www.idea-digital.com
From: varis@varis.com, varis@varis.com
Date: Thu, Aug 2, 2001, 11:51 AM
RE: [colortheory] Mung-and-blur?
Dan wrote:> A mung-and-blur method to be sure, but it worked: ... 256 levels input, 256
> output causes banding in smooth gradients unless countermeasures are taken.
> Munging and blurring is a good way.This is what I love about this list!
I've never heard this term (Mung-and-blur) before but I suspect that it is an older printing technique. Dan could you elaborate? What was the original mung-and-blur method?
--
regards,Lee Varis
varis@varis.com
www.varis.com
888-964-0024
From: tflash, tflash@earthlink.net
Date: Thu, Aug 2, 2001, 12:00 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dan,In thinking more about your experiment and conclusions, I'm confused.
You start making your 8-bit moves on these scans when they are in a very tonally compressed state, i.e., "bad scans", and you prefer their end result over 16-bit files which could undergo much more expansion before the loss of data.
This raises two points to mind:
A) Doesn't this also leave you arguing that working from a bad 8-bit scan will give a "better" result than working from a good 8-bit scan?
B) As the control freak I suspect you are (as are the rest of us when it comes to images, I presume) aren't you giving away a lot of control over what data and/or noise gets trashed in a file, relative to working with a high-bit file, or even a good 8-bit scan?
It just seems to me that your preference for the 8-bit versions was either coincidence, or that you manipulated each file differently, each in a way that would favor the 8-bit version. But in neither case because it could preserve the information of the original.
Doesn't the test then break down along aesthetic lines, rather than lines of capability? IOW, those who might prefer smooth tonal transitions over the sharpness which results from dropped tones, will not agree with your conclusions? Nor anyone who wishes to preserve the look of the original, for whatever reason?
Todd Flashner
From: "Bruce J. Lindbloom", blind@picto.com
Date: Thu, Aug 2, 2001, 1:35 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dave Badger wrote:
> Could someone please explain why all scanners need to capture in 10-16 bits
> these days.The sensors used in scanners and digital cameras respond in a manner that is proportional to the amount of incident light energy. So the "raw" data from the sensor may be thought of as having a gamma value of 1.0, which results in the uncorrected image looking "excruciatingly dark" (as Dan put it) on a monitor. In order for the image to look anything close to normal on a monitor (whose gamma is more like 2.2), the pixel data must go through a curve. If the scanner passed 8-bit data through this curve, the 256 possible original levels get reduced to 184 levels (using a gamma 2.2 curve):
Before curve:
0 1 2 3 4 ... 253 254 255 (256 different levels)After curve:
0 21 28 34 39 ... 255 255 255 (184 different levels)In order to "fill in" the gaps and increase the number of levels back up to 256, you need more bits of precision going into the curve.
So for a scanner or digital camera to deliver "good" 8-bit, gamma 2.2 data, its sensor must digitize to more than 8-bit precision.
--
Bruce J. Lindbloom, Pictographics Intl. Corp.
From: Dan Margulis,
Date: Thu, Aug 2, 2001, 10:46 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Bruce writes,>>A crucial piece of information missing from your tests has to do with the number of meaningful image bits vs. the number of noise bits in the original 16-bit scans. Imagine the hypothetical case where there are 8-bits of image data and 8-bits of noise. This case could easily lead an investigator to the conclusion you drew: 8-bits is just as good as 16-bits.>>
That's right. As I pointed out in my initial post, most captures as flat as Todd's come from devices of such questionable quality that one would have to suspect that what you say is true, 8 bits or even 10 bits is noise. If so, working those files proves nothing about the merits of 16-bit vs. 8-bit. Therefore, I said that having a scanner like Todd's was an important plus.
>>I'm not saying that's the case with your test, but in order for the conclusion to be meaningful, this measurement must be made. And you cannot simply rely on the scanner vendor's spec sheet to provide it.>>
Of course not, but on what *would* you rely? Surely you do not suggest that there is any instrument that can verify whether individual pixels of a scan are accurate? And even if there were, how would we know that any inaccuracy was due to the scanner, and not some poor contact, dust, voltage fluctuation, or defect in the film itself? What if parts of the scan are more accurate than others? And are you measuring composite, or channel by channel? As to doing it by eye, who is going to be so bold as to say he can evaluate whether a scanner is accurately capturing 256+ tone levels per channel based on high-magnification screen previews?
That's why the only solution is to ask for something "real-world", yet as extreme as possible, trying to bend over backwards to accommodate the hypothesis that 16-bit may actually be better. Professionally shot film is surely the most advantageous case for the 16-bit advocates; prints, older film, or amateur work is likely to be noisier. Extremely flat scans requiring massive moves are clearly more advantageous for 16-bit advocates. Conceivably it would be better for the 16-bit cause if I had taken the original film, mounted it on a drum scanner and battened down every setting to produce the flattest possible scan. That might give better data than Todd's, but since I would probably be the very first person in the world ever to have used a drum scanner that way, the results would be highly suspect.
Whether any current scanner produces more than 8 bits of valid data is a question for theologians more than ourselves. Personally, I think Todd's scanner does; I'm going to guess that typically it records between 200 and 350 meaningful values in the green channel, which is the most significant one. But I have no way of proving this, any more than anyone else would have a way of disproving it.
Dan Margulis
From: "Darren Bernaerdt", bernaerdt@telus.net
Date: Thu, Aug 2, 2001, 11:14 PM
RE: [colortheory] An 8-bit, 16-bit test
> That's why the only solution is to ask for something "real-world", yet as
> extreme as possible, trying to bend over backwards to accommodate the
> hypothesis that 16-bit may actually be better. Professionally shot film is
> surely the most advantageous case for the 16-bit advocates; prints, older
> film, or amateur work is likely to be noisier. Extremely flat scans
> requiring massive moves are clearly more advantageous for 16-bit
> advocates.So if the fine grain of a professionally shot chrome is going to provide smoother areas to tone in the final scan, what about a high end professional digital capture? I sometimes see areas of very smooth tones graduating in density in my digital captures. (ie, seamless paper in a product shot) This does not have the noise (grain) that a film scan would have. A relatively minor move in curves can introduce banding in that grad, however it never shows on press due to the limited number of tones our target can reproduce. If my end destination was a LightJet print, then I may want to consider 16bit?
Darren Bernaerdt
From: "Bruce J. Lindbloom", blind@picto.com
Date: Fri, Aug 3, 2001, 11:13 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Yesterday I wrote the following regarding Dan's 8-bit vs. 16-bit test:
>> A crucial piece of information missing from your tests has to do with the
>> number of meaningful image bits vs. the number of noise bits in the original
>> 16-bit scans. Imagine the hypothetical case where there are 8-bits of image
>> data and 8-bits of noise. This case could easily lead an investigator to
>> the conclusion you drew: 8-bits is just as good as 16-bits. I'm not saying
>> that's the case with your test, but in order for the conclusion to be
>> meaningful, this measurement must be made. And you cannot simply rely on
>> the scanner vendor's spec sheet to provide it.To which Dan replied:
> Of course not, but on what *would* you rely? Surely you do not suggest that
> there is any instrument that can verify whether individual pixels of a scan
> are accurate? And even if there were, how would we know that any inaccuracy
> was due to the scanner, and not some poor contact, dust, voltage
> fluctuation, or defect in the film itself? What if parts of the scan are
> more accurate than others? And are you measuring composite, or channel by
> channel? As to doing it by eye, who is going to be so bold as to say he can
> evaluate whether a scanner is accurately capturing 256+ tone levels per
> channel based on high-magnification screen previews?I don't think it matters so much which specific components are contributors to the noise. What is important is the aggregate noise. I also believe that the number of significant image bits is a measurable quantity. I was curious whether you or anyone else on this list had ever attempted to make this measurement, since IMO it is a very useful piece of information.
A couple of years ago, I wrote a program that attempts to determine the number of real bits in each of the three channels of a digital image. It works by slicing each channel into eight or sixteen separate bit planes, depending upon the pixel depth. Each plane is then subjected to a statistical analysis to determine how similar or different its properties are from pure noise. As you progress from the most significant bit to lesser bits, there comes a point where the bit plane statistics are indistinguishable from pure noise. This tells you where the image data stops and the noise starts. There may be better ways of measurement, but this was the best I could think of at the time. If you think it would be a useful or helpful contribution to your tests, I would be happy to analyze one or more of the raw original scans (subject to Todd's approval, of course).
My intent is not to be critical -- I think your test was absolutely fascinating, but before I can accept your conclusions, I need to be comfortable with the testing methods. Presently, I'm a bit skeptical because the results of your test cannot be independently verified (the images -- the "proof" or "evidence" -- have not been made publicly available).
> Whether any current scanner produces more than 8 bits of valid data is a > question for theologians more than ourselves.
Hmmm... ok, I give up. How can theology help? Shall we pray for an answer?
:-)
--
Bruce J. Lindbloom, Pictographics Intl. Corp.
From: Dan Margulis,
Date: Fri, Aug 3, 2001, 9:11 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Martin Orpen writes,>>PostScript as a page description language has never been limited to 256 gray levels. Admittedly Adobe's own Level 1 RIPs did have this limitation, but the language itself does not impose any limits.>>
The technology Gordo was referring to developed in the days of Level 1, although it presumably has gotten better since. In those benighted days, bombarding the imagesetter with extra gray levels wasn't an option, hence some kind of horseplay with the screens was in order.
>>Scitex are the latecomers here - I wonder if they have an estimate of how many pages of film were wasted after Quark XPress added the capability to generate graduated tints? :-)>>
I don't know, but it made the stockholders of Kodak and du Pont very, very happy.
Dan Margulis
From: Dan Margulis,
Date: Fri, Aug 3, 2001, 4:31 PM
RE: [colortheory] Mung-and-blur?
Lee Varis writes,>>I've never heard this term (Mung-and-blur) before but I suspect that it is an older printing technique. Dan could you elaborate? What was the original mung-and-blur method?>>
I don't know when the term originated but it may antedate me. Before electronic retouching when it was necessary to clone certain areas in film or knock out some noise, there were some dot etchers who upheld high quality standards (sign on the wall: "people whose hands aren't steady enough to work here become brain surgeons") while others munged and blurred with copies of the film.
Mung and blur nowadays can be quite sophisticated but usually seems to involve adding some very slight noise or dither in conjunction with some means of having each pixel slightly affected by the values of neighboring ones.
The kinkiest example of it I ever saw was in the days when a lot of folk had 1024 spi imagesetters and so had real difficulties with finer screens. Some geek came up with a method that created not a stochastic screen but a weird hybrid that imaged a 120-line and and a 240-line screen simultaneously, making a grid pattern of alternately large and small dots. Naturally the tiny dots could only handle about 16 gray levels, but he munged and blurred and damn if the thing didn't look pretty good. The name of this product, and what became of it, I have no idea.
Dan Margulis
From: Dan Margulis,
Date: Fri, Aug 3, 2001, 9:45 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Todd writes,>>A) Doesn't this also leave you arguing that working from a bad 8-bit scan will give a "better" result than working from a good 8-bit scan?<<
No, nor does it mean that I advocate nuclear war or poisoning stray cats. It means that I see no quality difference between working on a photograph in 16-bit as opposed to just converting it to 8-bit immediately.
>>B) As the control freak I suspect you are (as are the rest of us when it comes to images, I presume) aren't you giving away a lot of control over what data and/or noise gets trashed in a file, relative to working with a high-bit file, or even a good 8-bit scan?>>
No more than when I crop an image, res it down, apply a curve, a filter, or use a tool. All of these things trash data. If I think there's a significant chance that I might want to revert to an earlier state I save an intermediate copy. Some of your images captured the sprocket area of the film. If this were a real job I would have cropped those areas out immediately without any attacks of conscience. Similarly, having concluded that there are no real-world scenarios in which the extra 8 bits are useful in a natural photograph, I have no compunction in discarding them.
>>It just seems to me that your preference for the 8-bit versions was either coincidence, or that you manipulated each file differently, each in a way that would favor the 8-bit version.>>
I do not have a general preference for 8-bit and do not criticize those who work in 16-bit, I merely say there is no difference of any consequence.
The files were manipulated with exactly the same curves, saved and reloaded. Granted that the differences were totally trivial, I preferred the 8-bit versions twice, the 16-bit once, and two I rated a tie. So, yes, it is coincidence; any other file I might prefer the 16-bit slightly, but it would be irrelevant, because when the images are this close, if you want one to match the other more precisely it takes around three nanoseconds to make it happen.
>>Doesn't the test then break down along aesthetic lines, rather than lines of capability? IOW, those who might prefer smooth tonal transitions over the sharpness which results from dropped tones, will not agree with your conclusions?>>
If they couldn't tell the difference when the files were output in spite of the enormous curves that were applied, they would perforce agree that correcting in 16-bit is of extremely limited utility if any.
Dan Margulis
From: "Ron Bean", rbean@execpc.com
Date: Sat, Aug 4, 2001, 5:39 AM
RE: [colortheory] Histograms and digital cameras
Last week Dan wrote:>Rafe writes (and David similarly),
>
>>>Histograms are useful, IMHO, for knowing, at a glance, whether your scan
>is hopeless.>>
>
>$&*(%%@#!!! You mean you can't tell whether an image is no good by
>*looking* at it? You mean that if you have a scan that looks good on
>screen, but the histogram says it's hopeless, you throw it away???A few digital cameras offer a histogram, which photographers use to evaluate exposure in the field. The alternative is to try to draw some kind of conclusion from the tiny (usually 1.5") LCD on the back of the camera, or just bracket the shot and evaluate it later on the computer.
I've never seen a digital camera with an info palette, but the Nikon D1X has a feature that highlights any clipped pixels, so you can see where they are. Either of these sounds potentially more useful than the histogram, but we'll have to wait until the manufacturers are convinced (or until we have cameras that can run 3rd party software).
A while back we discussed profiles for digital cameras-- the author of Qimage sells profiles for a number of mid-range digital cameras (for about $13 ea), and has some before-and-after samples on his website at http://www.ddisoftware.com/qimage/plugins/.
The new Minolta Dimage 5 and Dimage 7 come with profiles, but for some reason they don't work in Photoshop, so you have to use Minolta's software to convert to a standard colorspace. These cameras have some very nice features but also some strange design flaws, so read the reviews before buying either of them (they're identical except for the size of the CCD).
http://www.dpreview.com/reviews/ http://www.steves-digicams.com/hardware_reviews.html
From: Dan Margulis,
Date: Sat, Aug 4, 2001, 11:33 AM
RE: [colortheory] An 8-bit, 16-bit test
Darren writes,>>So if the fine grain of a professionally shot chrome is going to provide smoother areas to tone in the final scan, what about a high end professional digital capture?<<
That would be even smoother.
>>I sometimes see areas of very smooth tones graduating indensity in my digital captures. (ie, seamless paper in a product shot) This does not have the noise (grain) that a film scan would have. A relatively minor move in curves can introduce banding in that grad, however it never shows on press due to the limited number of tones our target can reproduce.>>
A limited number of tones *encourages* banding. So, what makes you think there's banding? Looking at the monitor isn't a valid way of finding out; the settings you've used to calibrate with are very apt to suggest banding when there isn't any in reality.
If there ever were a case for 16-bit being better, it would have to be because it adds smoothness to the image. High-end digital captures are already the smoothest things on the market. For similar reasons, one can get by without as much resolution in digital captures than would be the case with film that has been scanned.
Dan Margulis
From: Dan Margulis,
Date: Sat, Aug 4, 2001, 2:50 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Bruce writes,>>A couple of years ago, I wrote a program that attempts to determine the number of real bits in each of the three channels of a digital image. It works by slicing each channel into eight or sixteen separate bit planes, depending upon the pixel depth. Each plane is then subjected to a statistical analysis to determine how similar or different its properties are from pure noise. As you progress from the most significant bit to lesser bits, there comes a point where the bit plane statistics are indistinguishable from pure noise. This tells you where the image data stops and the noise starts. There may be better ways of measurement, but this was the best I could think of at the time. If you think it would be a useful or helpful contribution to your tests, I would be happy to analyze one or more of the raw original scans (subject to Todd's approval, of course).>>
I don't know of any statistical method that would reliably distinguish between noise and valid data deeper than about three bits, i.e. about as well as a person who is on the threshold of being legally blind would do. I don't say it can't be done, though, before I've heard a description. I'd be interested in hearing that off-list (the question of whether a computer can measure bit depth that may or may not be necessary anyway seems to me to be getting into the area of "color management minutiae" that should be avoided here.)
>>My intent is not to be critical -- I think your test was absolutely fascinating, but before I can accept your conclusions, I need to be comfortable with the testing methods. Presently, I'm a bit skeptical because the results of your test cannot be independently verified (the images -- the "proof" or "evidence" -- have not been made publicly available).>>
It's odd that you aren't equally skeptical of those who have been foisting their theories that 16-bit is mandatory for quality work for all these years, considering that they have offered exactly zero in the way of examples or proof to back up the proposition. I've shown enlarged examples of the "differences" in my book since 1998, and would anticipate showing one or more of Todd's files in the next edition. Nobody on the other side as shown anything but histograms, AFAIK.
Nevertheless, there would seem to be a solution. Since you would be doing a computer analysis that presumably doesn't take overall appearance into account, you wouldn't need a complete image, would you? Todd has kindly consented to allow use of a *segment* of the images for this purposes. For my tests I cropped out some extraneous information, but even so they are around 3000 pixels in the long direction. If I give you a slice of a representative area, say 500 pixels wide, that's upwards of a million pixels, which ought to be enough for your program to analyze.
If you would like to try this, normal scientific procedure would be to attempt to verify how accurate your methodology was before accepting what it told us. The obvious way to do this would be for me to supply a dozen numbered image slices, including some of Todd's and some that are not, without indicating how they were produced. Then your program could report what was found.
When your results are ready, you will give me 24 hours notice, and I will print out the truth about the images and give it to Todd in a sealed envelope. You will then post the results to this group and as soon as possible thereafter I will respond with the facts about the images. Todd will then open his envelope and be able to verify that I hadn't changed anything after finding out what your results were.
If you wish to go through this exercise, I'd need to know whether you desire to analyze 8-bit files or 16-bit only, and whether you can analyze CMYK, LAB, and grayscale as well as RGB. Obviously I would need to know where to send the CD, and how much time you would need thereafter. It's rather a busy month for me so I would need about 30 days before I could get this to you.
Dan Margulis
From: lights@onr.com, lights@onr.com
Date: Sat, Aug 4, 2001, 5:10 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
On Sat, 4 Aug 2001 14:48:46 -0400, you wrote:>I don't know of any statistical method that would reliably distinguish
>between noise and valid data deeper than about three bits, i.e. about as
>well as a person who is on the threshold of being legally blind would do. I
>don't say it can't be done, though, before I've heard a description. I'd be
>interested in hearing that off-list (the question of whether a computer can
>measure bit depth that may or may not be necessary anyway seems to me to be
>getting into the area of "color management minutiae" that should be avoided
>here.)Just out of curiosity, I wrote a quick filter that reveals each bitplane of an image. It only works on images with 8 bits per channel, but for those images, I can see patterns at all bit levels. If you would like to see the images, they are available at http://the-light.com/Photography/Bitplanes/
Note, though, that the total size of all images on that page is about 1.3 Mb. The individual bit plane images are not very compressible.
If patterns can be seen visually, they can be recognized by statistical analysis.
-----------------------------------------------------------
Victor Engel lights@onr.com http://the-light.com
From: Raphael Bustin, rafeb@channel1.com
]Date: Sun, Aug 5, 2001, 10:47 PM
RE: Re: [colortheory] An 8-bit, 16-bit tes
At 08:43 PM 8/5/01 -0400, Todd wrote:>I'm wondering if it's similar to the way many an image will look the same on
>screen, whether your monitor is set to thousands, or millions, of colors,
>but others won't. I suppose the same rules apply too; if there will be a
>difference, it will show up as banding in smooth gradations.Todd, there's an easy way to experiment with this.
In Photoshop, try Image->Adjust->Posterize. You're asked to enter an integer for the number of color levels.
What I find interesting is how small a number you can get by with, on most images.
And, as you've observed, one of the primary effects of too-small a value is that the image gains in contrast.
rafe b.
> A limited number of tones *encourages* banding. So, what makes you think
> there's banding? Looking at the monitor isn't a valid way of finding out;
> the settings you've used to calibrate with are very apt to suggest banding
> when there isn't any in reality.The banding is appearing on the monitor and also in a print done on a Fuji Frontier printer (a printer using silver based photographic paper for those not familiar with it).
> If there ever were a case for 16-bit being better, it would have to be
> because it adds smoothness to the image. High-end digital captures are
> already the smoothest things on the market. For similar reasons, one can
> get by without as much resolution in digital captures than would be the
> case with film that has been scanned.So maybe when taking shots (digitally) against a graduated background, throwing in a bit of noise may be an appropraite move? Would working in a lower gamma/smaller RGB space like ColorMatch be more appropriate than AdobeRGB? (The banding I'm referring to is particularily with head & shoulders business portraits shot on a white to mid/dark grey seamless background. The banding tends to show up in about a 60% value.)
Darren Bernaerdt
From: "Dave King", kingphoto@mindspring.com Date: Mon, Aug 6, 2001, 10:14 AM RE: Re: [colortheory] An 8-bit, 16-bit test
> Todd writes, >
> >>I look forward to seeing what you did to the images, because in my
> experiments I reached a different conclusion, but with caveats: I was
> viewing the same files at 2x the resolution I provided you, I made my
> evaluations onscreen, rather than in print (what did you output them
> on?),>>Dan writes:
> No need for any output--the differences can be evaluated accurately on > screen.
>
> >>As you know, I only found one image that I could see no difference in, and > all the others I thought looked better in 16-bit. But I was looking for
> smoothly graduated tones; you may have been looking for "drama" or
> "energy".>>
>
> More precisely, the appearance of sharpness or detail. The 8-bit images
> look very marginally sharper, the 16-bit ones marginally softer. The
> differences are irrelevant in the real world since one can compensate in
> either direction.I haven't been keeping up with the list of late, and haven't finished reading this thread to date yet, but like a fool I'm going to jump in here anyway. :)
I have one image, a beauty portrait shot on medium format color neg, that I manipulate into a "blown out" final image from a Pro Photo CD scan that is too dark. Even though the Pro Pho CD scan is too dark, I lit and exposed the neg with the hi key result in mind. It takes two successive curve manipulations to get it where I want it. My first attempt was with PS5, which could not pull hi-bit from Pro PhotoCD, and even though the monitor image didn't look particularly posterized (looking for creamy smooth hi value skin tones), the Epson output certainly did. In short, an unusable result.
I did subsequently get a very usable result editing the same file in hi-bit, but I'll admit, I didn't use the same corrections (previous were long forgotten), and my PhotoChops had improved in the interim considerably. When I get a chance I'll do an exact comparison this time, and print the results on an Epson. If there is a significant difference I'll send the result to Dan for him to see. If there isn't I'll eat my hat on this list. It's a nasty old sweaty one too. :)
Dave King
From: Corpricom/JC, corpricom@earthlink.net
To: Dan Margulis, 76270,1033
Date: Mon, Aug 6, 2001, 11:44 AM
RE: More 16 bit
Hi Dan,There is a thread running on the Adobe Photoshop forum named "16 bit and the histogram." A few of the posters (names you would recognize) promoted the benefits of 16 bit, so I asked how they would answer your 16 bit challenge. I thought it was an innocent enough question and I really wasn't interested in starting a fight, but there were a couple of answers that you may want to be aware of, if not respond to.
By the way, no one volunteered to supply any images.
Regards,
Jonathan Clymer
From: Dan Margulis,
Date: Mon, Aug 6, 2001, 12:06 PM
RE: RE: [colortheory] An 8-bit, 16-bit test
Darren writes,>>The banding is appearing on the monitor and also in a print done on a Fuji Frontier printer (a printer using silver based photographic paper for those not familiar with it).>>
I'm one of those, but I assume it wants 8-bit, RGB data. If so, its internal driver that converts to CMY(k) is the likely suspect, although that shows up in dark blues more commonly than in neutral colors. Experimentation is going to be necessary.
>>So maybe when taking shots (digitally) against a graduated background, throwing in a bit of noise may be an appropriate move?>>
That's the ultimate solution, but not a particularly desirable one. Perhaps your captures are so smooth that it's necessary. A way to test this, if you happen to have a copy of Photoshop 4 or earlier around, is to generate some gradients and run those to see what happens (Photoshops 5.0 and up generate small amounts of noise in gradients to defeat banding).
If you conclude that there has to be noise added, you need to run tests to see how much. If it bothers you to have that much noise in the fleshtones, there are various ways of writing actions to get a more sophisticated kind of noise that will work the dark neutrals more prominently than the rest of the image.
>>Would working in a lower gamma/smaller RGB space like ColorMatch be more appropriate than AdobeRGB? (The banding I'm referring to is particularily with head & shoulders business portraits shot on a white to mid/dark grey seamless background. The banding tends to show up in about a 60% value.)>>
It's not impossible--if a higher-gamma RGB caused banding, 60% is about where it would appear. Again, test it. Set up a phony RGB workspace with gamma 1.0, run a few files, and see what happens.
Dan Margulis
From: Dan Margulis,
Date: Mon, Aug 6, 2001, 12:06 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Todd writes,>>Just for the record, Blatner/Fraser show the same example in at least two books: Real World Photoshop 4, Pg. 154, and Real World Photoshop 6, pg. 216. They show a grayscale image that had three successive gamma moves applied to both an 8-bit, and 16-bit, version. They also include each variants histogram.>>
I don't have either of these editions, so can't comment. I have RWP5 but unless their index is very defective no such demonstration appears.
>>When I compare the histograms of some of the images we tested, even those which looked near identical, they had wildly different histograms. Some of those 8-bit histograms looked ridiculous, but the images themselves look good.>>
All the more reason not to look at the histogram at all. The test proposed by Rafe is a good one that lurkers might try. With any final image, do Image: Adjust>Posterize>128. This creates, in effect, a 7-bit image. Then do a series of Command-Zs and see if you can see the difference. When you can't, see how high you have to magnify the files to see that they aren't identical.
What most people will find is that 128 is usually identical for all useful purposes and 64 is likely to be as well. Even 32 is probably OK on certain images.
>>I redid some of my tests and in some of the images that I saw distress in the first time out, did look better this time. I don't know why. Perhaps the first time I was looking at some parts of the images at a poor screen magnification, like 33%, or 67%.>>
That would do it.
Dan Margulis
From: Andrew Rodney, andrew@digitaldog.net
Date: Mon, Aug 6, 2001, 12:17 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
on 8/6/01 10:04 AM, Dan Margulis wrote:> I don't have either of these editions, so can't comment. I have RWP5 but
> unless their index is very defective no such demonstration appears.Try page 206, 207 and 208 of RWP5.
Andrew Rodney
From: "Dave King", kingphoto@mindspring.com
Date: Mon, Aug 6, 2001, 1:02 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
I wrote:> When I get a chance I'll do an exact comparison this
> time, and print the results on an Epson. If there is a significant
> difference I'll send the result to Dan for him to see. If there isn't
> I'll eat my hat on this list. It's a nasty old sweaty one too.
Munch munch -- that hat doesn't taste all that good either.Just ran a quick test, and was surprised not to see any significant differences between the worst case 8-bit file and the "optimum" 16 bit file on either monitor or print. I guess the differences I have seen previously regarding this issue are down to inexact comparisons, or workflows that have subsequently been improved.
Very interesting! :)
Dave King
From: "Bruce J. Lindbloom", blind@picto.com
Date: Mon, Aug 6, 2001, 1:25 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Dan wrote:
> It's odd that you aren't equally skeptical of those who have been foisting
> their theories that 16-bit is mandatory for quality work for all these
> years, considering that they have offered exactly zero in the way of
> examples or proof to back up the proposition. I've shown enlarged examples
> of the "differences" in my book since 1998, and would anticipate showing
> one or more of Todd's files in the next edition. Nobody on the other side
> as shown anything but histograms, AFAIK.Let me make it very clear: I do not have a predisposed position on the 8-bit vs. 16-bit issue. I would like to have a position on this subject, but so far I have not seen enough evidence, one way or the other, to form a valid opinion. Thus my interest in your tests. But it is also my opinion that an assertion (i.e. a statement made WITHOUT proof or evidence) carries much less significance than does an argument (i.e. a statement made WITH proof or evidence). I am not disagreeing with the conclusion you came to. All I am saying is that providing some evidence would give your conclusions much more impact.
Regarding my lack of criticism of those holding the pro-16-bit view, I'm not aware of who you are, but if you don't provide evidence, then shame on you, too! :-)
> Nevertheless, there would seem to be a solution. Since you would be doing a > computer analysis that presumably doesn't take overall appearance into > account, you wouldn't need a complete image, would you? Todd has kindly > consented to allow use of a *segment* of the images for this purposes. snip
I think a test is a great idea, but let's be sure to keep it friendly and objective. My offer to attempt to measure the number of significant image bits in a scanner was intended to contribute helpful information to this discussion. I think it is intimately related to the experiment you performed. In an earlier post, I stated briefly the method I used, and have asked if anyone knows a better way (no suggestions so far). I will fully disclose the method I use and welcome alternative methods.
You can imagine a hypothetical case where the analog output of a scanner sensor is wired up to a 128-bit A/D converter (if there even is such a thing), and sure enough, 128-bit data comes out. Sounds great on the scanner spec sheet, but of course, any thinking person might be just a little doubtful that all 128 bits are meaningful. It seems to me that it should be possible to objectively measure the number of "real bits". It is this entity that I attempt to measure. For the test being discussed in this thread, this would be the raw, 16-bit RGB data from Todd's scanner, which was the source data for all following experiments.
Although I don't think all the "cloak and dagger" involving 24 hour notices and sealed envelopes is really necessary, I am open to any valid methods of cross-checking. When you say that you will "print out the truth about the images", what do you mean? Since all I plan to do is measure and report the number of significant image bits in original, raw scans, and your printed "truth" presumably will either affirm or refute my findings, the implication is that you already know the answers. But how do we know that YOUR answers are correct? And what method will we use to verify THAT?
> Obviously I would need to know where to send the CD, and how much time you > would need thereafter. It's rather a busy month for me so I would need about > 30 days before I could get this to you.
Forget CDs (we live in an electronic age) and forget 30 days (this thread will be stone cold dead by then). Here is a counter proposal that does not take time from your busy schedule:
a) Todd emails me one or more, complete or partial raw 16-bit scans (I'll take what I can get). I will agree not to distribute or make copies of any of these files, and further agree to immediately delete them all upon notice from Todd.
b) I will measure the number of significant bits of each channel of each file.
c) I will post my findings to this list along with a detailed description of the method I used.
I can post the result within 24-hours of receipt of the images. Since I presume all images were scanned on the same scanner under the same conditions, I don't think we need lots of large files to make the measurement. So perhaps a smaller cropped piece of one or more images might serve our purposes, thus allowing the images to be quickly e-mailed. (If so, I need to discuss with Todd off-list what image content would be most appropriate from the set he has available.) Unfortunately, some amount of Todd's time would be required to prepare the image pieces, and we should not impose on him any more than necessary.
Others wishing to "take the challenge" to independently make the measurement may contact Todd and make similar arrangements and we can compare both results and methods. Hopefully we all can learn something from this exercise (myself included).
--
Bruce J. Lindbloom, Pictographics Intl. Corp.
From: Andrew Rodney, andrew@digitaldog.net
Date: Mon, Aug 6, 2001, 3:00 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
on 8/6/01 12:33 PM, Todd Flashner at tflash@earthlink.net wrote:>> > a) Todd emails me one or more, complete or partial raw 16-bit scans (I'll
>> > take what I can get). I will agree not to distribute or make copies of any
>> > of these files, and further agree to immediately delete them all upon
>> > notice from Todd.
>
> I accept to your conditions above.
>Guys, if you want a high bit (16 bits per color) of an Imacon Flextight, let me know. The main difference is that with this scan, the high bit will be toned unlike the Leaf. I can set the scanner to provide a flat, full range scan that produces toning a lot closer to a final goal than the Leaf which produces linear data. I think the Leaf will provide a lot of the best challenges to editing the file but newer high bit scanners don1t1 have the limitation of only supplying un-toned high bit data. If you think that would aid in the science, let me know.
Andrew Rodney
From: Martin Orpen, orpy@idea-digital.com
Date: Mon, Aug 6, 2001, 6:51 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
I carried out an experiment on some 16 bit and 8 bit images while commuting home on the train this evening.The original image was generated in Photoshop -- so fails Dan's specification of being a "real" image. However, it is restricted to colours found in a common CMYK gamut and certainly less than the gamut of a high end scanner.
For those that don't want to ready about the process, the results and conclusions were as follows:
Results:
1. The 16 bit images show little noticeable quantization after applying heavy curve transformations
2. The same curves applied to an 8 bit image in Colormatch RGB showed more quantization than the 16 bit sample.
3. Applying the same curves to an 8 bit image in a wide gamut RGB space (Ekta) provided the best colour match to the 16 bit file. The quantization was slightly less than the Colormatch version, but in some hi-light areas appeared worse.
Conclusions:
1. For best results in scanning and manipulating negatives I will continue to scan in 16 bit. The limited density range and massive exposure range make them more prone to quantization during retouching than transparency.
I don't think the data overheads and the loss of many of Photoshop's features are worth it when it comes to scanning transparency and reflective originals.
2. I was expecting the quantization to actually appear worse in the 8 bit "wide gamut" RGB version due to rounding errors in the maths during the transforms. In these examples the results for Ekta space were better than those in the Colormatch version.
I'll post the images to my FTP server tomorrow and post and address and password so that those interested can download the stuffed images.
The method:
1. Create a new document, 1000x1000 pixels in Lab format.
2. Fill the L channel with 50 per cent black
3. Fill the a channel with a linear gradation from black to white from left to right
4. Fill the b channel with a linear gradation from black to white from top
to bottomThis gives you a range of colours far outside any printing or screen gamut.
5. Apply gamut warning (I am using Euroscale V2 as my CMYK profile) so that you can see the area that is within your CMYK gamut.
6. Crop the image so that you don't include anything much outside your gamut.
7. Save the image as a L*a*b master file.
Now you have a Lab image that covers the most saturated colours in your CMYK gamut. All we have to do now is apply some hi-light and shadow detail.
8. Select the L channel and apply: filter>render>clouds.
9. Change the mode to 16 bit.
10. Apply a Guassian blur of 10 to the L channel.
11. Save your master file.
Now we have something to experiment on...
12. Convert the file to RGB.
13. Start recording in the Actions palette and apply 4 or 5 sets of curve corrections. I applied two sets of different, steep "S" curves in each of the channels and then applied a further 2 sets of roughly correcting inverted "S" curves. Save your Actions.
14. Save this file as your 16bit RGB master.
15. Open the original file (11). Convert to RGB, dump down to 8 bit and then apply the saved actions (13). Save this images as your 8 bit master.
16. I carried out 1 further variation -- my RGB colour space was set to Ekta, so for the 3rd variation I opened the master file (11), converted it to Colormatch RGB, dumped it down to 8 bit and then applied the Actions.
--
Martin
Idea Digital Imaging Ltd
http://www.idea-digital.com
From: Dan Margulis,
Date: Mon, Aug 6, 2001, 7:59 PM
RE: Re: [colortheory] An 8-bit, 16-bit test
Todd wrote,>>Just for the record, Blatner/Fraser show the same example...They show a grayscale image that had three successive gamma moves applied to both an 8-bit, and 16-bit, version. They also include each variants histogram.>>
Andrew Rodney kindly gave a page reference to the edition that I have. Assuming that this is correct and the image is an vertical shot of one light and one dark building, labeled "Data loss due to tonal correction" it does not show what Todd is describing.
The two versions are not "three successive gamma moves applied to both an 8-bit and 16-bit version". They are three moves applied to an 8-bit version compared to *an untouched original*.
The idea was to get the right-hand (corrected) version to match the original tonally, but it didn't quite work out; the midtone is slightly darker, and naturally that means it has better highlight detail and worse shadow detail.
All that this section has is the usual, a few ratty-looking histograms that purport to demonstrate "data loss", many assurances that 16-bit correcting is the way to go, but absolutely nothing in the way of visual proof.
The advocates of this workflow are asking us to double our file sizes. It doesn't seem too much to ask that they should provide side by side examples of a real photograph, at a large enough size that we can really see what's going on. If they want to apply three curves to an image, fine; let them apply the same curves to a 16-bit version on the left and an 8-bit version on the right, and let the readers decide whether they see a significant difference.
Dan Margulis
From: Brad McMullin, mcmullin@studiod-online.com
Date: Tue, Aug 7, 2001, 9:47 AM
RE: [colortheory] stupid question 8bit, 16bit
Sometimes I see pictures of commercial products and models having a very
smooth surface or skin. It almost look like a cartoon or rendering is this
the result of a smooth 16bit tonal range or just excessive retouching.--
Brad McMullin
Digital Imaging Expert
mcmullin@studiod-online.com
Studio D Photographers
864.294.0060 fax 864.294.0061
http://www.itsyourimage.com
From: Dan Margulis,
Date: Tue, Aug 7, 2001, 11:24 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Bruce writes,>>Although I don't think all the "cloak and dagger" involving 24 hour notices and sealed envelopes is really necessary, I am open to any valid methods of cross-checking. When you say that you will "print out the truth about the images", what do you mean?>>
I meant that, since I don't believe that a statistical method can identify real data that deep, I would have put in some legitimate 16-bit scans and some that were produced by methods that clearly create invalid information. I suspect, but don't know, that your program would not have been able to tell them apart.
>>Since all I plan to do is measure and report the number of significant image bits in original, raw scans, and your printed "truth" presumably will either affirm or refute my findings, the implication is that you already know the answers. But how do we know that YOUR answers are correct? And what method will we use to verify THAT?>>
The unhappy part of dealing with people with significant production experience is that we tend to be rather cynical and feel free to call things as we see them. The good part is that we always have reliable backups.
>>a) Todd emails me one or more, complete or partial raw 16-bit scans (I'll take what I can get). I will agree not to distribute or make copies of any of these files, and further agree to immediately delete them all upon notice from Todd.>>
Although this leaves your program's validity untested, I'd say it's a reasonable way to go at this point. Not to belabor the obvious but these files can't be JPEGged before they get to you.
Dan Margulis
From: Dan Margulis,
Date: Tue, Aug 7, 2001, 10:13 AM
RE: Re: [colortheory] An 8-bit, 16-bit test
Martin Orpen writes,>>The original image was generated in Photoshop -- so fails Dan's specification of being a "real" image. However, it is restricted to colours found in a common CMYK gamut and certainly less than the gamut of a high end scanner.>>
The stipulation that it be a real photograph and not computer-generated art was not just a whim. The two are different animals, and the results you got are what one would expect.
When you generate your own art each pixel is by definition perfect. Any change you make to it--such as by converting fr