Dan Margulis Applied Color Theory

Differences Between TIFF and PSD


What is the difference on Tiff and PSD
Posted by:Joel Hinojosa
Wed Feb 28, 2007 11:38 am

What is the difference on Tiff w / layers and PSD or PSD will disappear any time sun I believe is most be some differences on this two formats

thanks

Joel Hinojosa
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Louis Dina
Wed Feb 28, 2007 4:50 pm

Joel, At one time, TIFF was a standard file format (tagged image file format) that was and still is used widely in many applications, platforms, and extensively for print work. The 'standard' format allows for clipping paths and some other extras, but Adobe has added a lot of new capabilities to Tiff files within Photoshop that are very non-standard, such as multiple layers. In general, I don't consider this a good thing, because it leads to confusion and you can't tell by the file extension whether you have a 'standard' Tiff or a 'augmented" Tiff with features not supported by other programs.

So, my normal rule is that I only save a file as a Tiff when it is a single layer document meeting ALL the original requirements and limitations. If I want to save a file with multiple layers and other features that are not standard in the Tiff format, I save as PSD. This is my 'flag' to myself that I probably have multiple layers or other such stuff in the file. Since Photoshop is the 800 lb gorilla and king of the mountain, I'm not too concerned about PSD format being abandoned any time soon.

Lou Dina
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Andrew Rodney
Thu Mar 1, 2007 10:35 am (PST)

-"Joel" wrote:

What is the difference on Tiff w / layers and PSD or PSD will
disappear any time sun I believe is most be some differences on this
two formats

Summed up nicely here with additinal links:

http: //luminous-landscape.com/forum/lofiversion/index.php/t13479.html

Andrew Rodney
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Bob Smith"
Thu Mar 1, 2007 10:36 am (PST)

In Julianne Kost's Lightroom presentation at the PPA convention a few weeks ago, she mentioned there had actually been some discussion of leaving PSD out as an export option. They didn't, but it sounds as if PSD would be more likely to be devalued before TIFF. There's zero functional difference between PSD and TIFF now... and as such little need for PSD. TIFF offers more compression options if you want them. TIFF always includes a composite (flattened) image for compatibility with older programs. Its sort of like turning on Maximize compatibility in PSD files. I'm finding I pretty much have to do that with PSD files to maintain full compatibility with finder previews and cataloging apps anyway... so again... no advantage to PSD.

I had recently saved a heavily retouched image as a layered TIFF in CS3. It included smart filters, several smart objects... numerous special elements. The file became corrupt and could not be fully opened by CS3. It could however open it as a flat image with all effects in place. That same corrupt file (over 1GB in size) was also able to be opened by numerous other programs as a simple flat image... no problem. That speaks well to how compatible TIFF is.

Bob Smith

Accurate Image o Bob Smith Photographer o Waco Texas USA
http://www.accurateimage.org
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Pylant, Brian"
Thu Mar 1, 2007 10:43 am (PST)

So, my normal rule is that I only save a file as a Tiff when it is a
single layer document meeting ALL the original requirements and
limitations. If I want to save a file with multiple layers and other
features that are not standard in the Tiff format, I save as PSD.
This is my 'flag' to myself that I probably have multiple layers or
other such stuff in the file.

Ditto here, and I do the same thing with Illustrator as well -- I can assume that any .eps file I've saved from Illustrator has all the text converted to paths, and any other element that need simplification are simplified, so the file can be handed off to anyone else. The .ai version of the file would be the one with live text, effects, invisible working layers, etc.

Poor organizational working habits is one of the common problems I see with customer files -- the design itself might be nice, the specs all met (resolution, color mode, etc.) but the files are a complete trainwreck.

BRIAN PYLANT
Manager, Electronic Prepress

:::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::

Disc Makers
7905 North Route 130, Pennsauken, NJ 08110
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Stephen Marsh"
Thu Mar 1, 2007 4:29 pm (PST)

Ditto here, and I do the same thing with Illustrator as well -- I
can assume that any .eps file I've saved from Illustrator has all the
text converted to paths, and any other element that need
simplification are simplified, so the file can be handed off to anyone
else.

Agreed Brian, a good and common approach with one important catch for my setting (which may not be concern to all) - live text/fonts outlined to vector paths will RIP slightly fatter, even more so with digital printing resolutions over the higher ones found in film/platesetters where the issue is less noticeable.

This can be enough of a concern in my setting that sometimes live fonts must be used.

Sometimes with transparency flattening/stitching it is best to outline all fonts so that there are no minor differences in live vs. vector outlines.

Poor organizational working habits is one of the common problems I
see with customer files -- the design itself might be nice, the specs
all met (resolution, color mode, etc.) but the files are a complete
trainwreck.

Agreed, as RIPs are no longer as fussy as in past times, more shocking input can be pushed through them without the problems found in the past (this still does not excuse poor electronic housekeeping IMHO).

Regards,

Stephen Marsh.
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Todd Shirley"
Thu Mar 1, 2007 4:30 pm (PST)

Andrew

Added functionality like layers can make other programs (QuarkXpress) choke on the so-called "TIFF". Although Quark is supposed to support importing layered tiffs, I have seen many instances where it does not handle them properly and either a proof cannot be output or if it does output, the proof is bad. I realize that Adobe "owns" the TIFF format, and that really this is a Quark bug, but that doesn't really make my life any easier when I'm trying to output proofs from Quark. So I do the same as Lou Dina and "I only save a file as a Tiff when it is a single layer document meeting ALL the original requirements and limitations. If I want to save a file with multiple layers and other features that are not standard in the Tiff format, I save as PSD."

Outside of the Quark bug, the main problem I have with layered TIFFS is how huge they can become. I've seen 100MB 8-bit images with ten layers become 600MB tiffs and only 150MB PSD files, so for me it is more efficient to save both a flattened TIFF and a PSD and both save space and have confidence that the TIFF will output correctly.

-Todd Shirley
Urban Studio
New York, NY
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Andrew Rodney
Fri Mar 2, 2007 9:42 am (PST)

On 3/1/07 4:33 PM, "Todd Shirley" wrote:

Although Quark is supposed to support
importing layered tiffs, I have seen many instances where it does not
handle them properly and either a proof cannot be output or if it
does output, the proof is bad.

Seems someone should be yelling at Quark, not Adobe, no?

How©ˆs InDesign handling them?

Andrew Rodney
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Dan Margulis
Fri Mar 2, 2007 9:46 am (PST)

Todd Shirley writes,

Added functionality like layers can make other programs (QuarkXpress)
choke on the so-called "TIFF". Although Quark is supposed to support
importing layered tiffs, I have seen many instances where it does not
handle them properly and either a proof cannot be output or if it
does output, the proof is bad. I realize that Adobe "owns" the TIFF
format, and that really this is a Quark bug, but that doesn't really
make my life any easier when I'm trying to output proofs from Quark.
So I do the same as Lou Dina and "I only save a file as a Tiff when
it is a single layer document meeting ALL the original requirements
and limitations. If I want to save a file with multiple layers and
other features that are not standard in the Tiff format, I save as
PSD."

The Photoshop team has a history of going to great effort to design things that break Quark and then declaring that they are Quark bugs. The addition of layering to TIFF (Photoshop 6) was a big mistake--they should have designated it as a different format. But it was inoffensive enough, because we could check off a preference that would absolutely prevent us from saving layered "TIFF"s. Most people did that. As this thread has shown, overwhelmingly users still reserve the layering and other options for PSD, as do I.

What *was* offensive, and inexcusable, was the removal of this preference in Photoshop 7. This was a disgraceful effort to sabotage the workflows of those using Quark, Corel, and various others, by inviting people to save Adobe-only compatible "TIFF"s by mistake. For some time Adobe denied the obvious (what other reason could there be for deleting a safety feature, in the face of loud advance warnings of its necessity?) but some two years later a prominent member of the Photoshop design team did confirm it .

This conduct has recently come back to bite Adobe in an unfortunate way. The DNG format that they have proposed is sensible and meets the needs of the community. Unfortunately, camera vendors are very reluctant to cooperate, with the result that the format is now in big trouble. My information is that the Photoshop team's conduct in degrading the TIFF format for the purpose of damaging a competitor is a big factor in the camera manufacturer's distrust of Adobe to manage DNG.

Outside of the Quark bug, the main problem I have with layered TIFFS
is how huge they can become. I've seen 100MB 8-bit images with ten
layers become 600MB tiffs and only 150MB PSD files, so for me it is
more efficient to save both a flattened TIFF and a PSD and both save
space and have confidence that the TIFF will output correctly.

That's correct also. PSD contains mild compression capabilities, so that things with large areas of similar color save to much smaller sizes than TIF does.

Back to vacation.

Dan Margulis
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Andrew S. Webb"
Fri Mar 2, 2007 10:21 am (PST)

On Mar 1, 2007, at 4:33 PM, Todd Shirley arranged some pixels so they looked like this:

I've seen 100MB 8-bit images with ten
layers become 600MB tiffs and only 150MB PSD files

I expect this is because you aren't using LZW compression in your TIFFs, which Photoshop does automatically, OR you have the compatibility option in Photoshop turned off. The latter would eliminate the composite version of the file, which results in a large filesize reduction.

Cheers,

_andrew webb
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "dbernaerdt"
Fri Mar 2, 2007 12:07 pm (PST)

Andrew,

If a company is going to promote a "standard", then it should be exactly that. Not a moving target. A revision shouldn't break an existing workflow. There were too many "gotchas" when TIFF gained new abilities such as layers and JPEG compression.

As for InDesign's handling, it's hardly a fair comparison since it is another Adobe product.

Darren Bernaerdt
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by Andrew Rodney

Oh so standards means stagnation? Silly. If Quark or anyone else wants to comply with the open and published standards, they can.

I can©ˆt open a Quark doc in Photoshop, what©ˆs Quarks problem? Can©ˆt open an Excel doc either.

Andrew Rodney

___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "dbernaerdt"
Fri Mar 2, 2007 3:21 pm (PST)

Andrew,

Your analogy doesn't fly. Photoshop has never opened Quark documents, nor Excel.

Darren Bernaerdt
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Ron Kelly"
Fri Mar 2, 2007 5:20 pm (PST)

Andrew:

It's no secret you're in close touch with Adobe, but this is a bit much. If you want to be a shill for Adobe, please do it somewhere else.

Ron Kelly
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "MARK SEGAL"
Fri Mar 2, 2007 7:49 pm (PST)

A technical discussion about the relative merits and features of two file formats has now degenerated into conspiratorial speculations first spun by Dan relative to Adobe and then spun by Ron relative to Andrew's alleged relationship with Adobe. Frankly, I don't see what any of this contributes to an understanding of the technical issues regarding the file formats or indeed to the fundamental purposes of the Colortheory List. For whatever the reasons or the motives the formats do what they do and don't do what they don't do, for better or for worse, as now elsewhere clarified in this thread.

Mark Segal
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Arye P Rubenstein"
Fri Mar 2, 2007 11:42 pm (PST)

...and I just discovered there is a 2GB limit for PSD and not on TIF in PhotoShop. I just tried saving a 2.84GB file as a PSD and no go, massage said to big - but as a TIF it was ok.

my 50¢ (2¢ adjusted for inflation)

Arye P. Rubenstein
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Stephen Marsh
Fri Mar 2, 2007 11:46 pm (PST)

Arye, this is one reason why Adobe introduced the Large Document Format in CS (or was that v7) - which has the file extension of .psb

Regards,

Stephen Marsh.
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Lee Clawson"

Oh so standards means stagnation? Silly. If Quark or anyone else wants to
comply with the open and published standards, they can.

If you're outside and needing to use the "standard" for making nuts, bolts and screws (an early important standard that we depended on being upheld) then I suspect things look different.

Put me down as "not silly" to hold the standard. Offer other versions as tools and techniques evolve.

Lee Clawson
2/\V/\7 Studio
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Larry Wangelin"
Sat Mar 3, 2007 4:28 pm (PST)

It never ceases to amaze me how quickly the goon squad appears on this list.

Larry Wangelin
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "colorman042000"
Sat Mar 3, 2007 4:28 pm (PST)

A guest showing the door to another guest?

Ron, your message doesn't serve any useful purpose, quite the contrary ...!!! I think you should have sent this as a private message to Andrew.

I don't understand why it was published (?)

André Dumas
___________________________________________________________________________

Disagreements
Posted by: "Howard Smith"
Sat Mar 3, 2007 4:29 pm (PST)

Whoa, folks! While I don't always agree with Andrew Rodney, I have come to respect his knowledge, his ideas, and especially his assistance when I have a problem to which I cannot find an answer anywhere else. There's nothing to be gained by taking cheap shots at him. If we start doing that to everyone with whom we disagree, we'll soon be talking to ourselves because no one is going to be willing to offer new ideas or express opinions that might not be popular with the majority. My own strongly held feeling about criticism is that it should always be welcomed when it's constructive. Criticism posted here usually is objective, which is why I feel at ease in expressing my own feelings even when they're almost certainly not going to be popular, especially with those who know enough about the subject to be able to shoot me down when I'm wrong. But if my posts begin to invite name-callling and put-downs, I'm going to be looking for a Forum that is more interested in new ideas than in feuding. Let's please not go in that direction. We'll all be losers if this catches on.

As a disclaimer to avoid any wrong impressions, this post is not aimed at any individual. If I want to call any of you down, I'll do it in a private communication. This is no place for it.

Howard Smith
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "markds0"
Sun Mar 4, 2007 1:11 pm (PST)

Looking through the Lightroom Help material for answers on another issue altogether, I came accross the following, which would seem to suggest that Adobe sees merit for TIFF over and above Photoshop's native PSD format, especially in respect of its portability; here is what they say:

"Tagged-Image File Format (TIFF, TIF) is used to exchange files between applications and computer platforms. TIFF is a flexible bitmap image format supported by virtually all paint, image-editing, and page-layout applications. Also, virtually all desktop scanners can produce TIFF images. Lightroom supports large documents saved in TIFF format (up to 100 million pixels with pixel dimensions of no more than 10,000 on a side). However, most other applications, including older versions of Photoshop (pre-Photoshop CS), do not support documents with file sizes greater than 2 GB.

"The TIFF format provides greater compression and industry compatibility than Photoshop format (PSD), and is the recommended format for exchanging files between Lightroom and Photoshop. In Lightroom, you can export TIFF image files with a bit depth of 8 bits or 16 bits per channel."

Mark Segal
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: Stephen Marsh
Mon Mar 5, 2007 1:20 am (PST)

Mark Segal wrote:

Looking through the Lightroom Help material for answers on another
issue altogether, I came accross the following, which would seem to
suggest that Adobe sees merit for TIFF over and above Photoshop's
native PSD format, especially in respect of its portability; here is
what they say:

For me it is more in what is *not* said.

Tagged-Image File Format (TIFF, TIF) is used to exchange files
between applications and computer platforms. TIFF is a flexible
bitmap image format supported by virtually all paint, image-editing,
and page-layout applications.

As long as the compression scheme used in the TIFF file is understood.

No compression is a "bullet proof" option.

LZW can cripple the file for many apps. If I recall correctly, older versions of Xpress required an xtension (plug) to handle LZW TIFF files.

On XP, the OS can generate a thumbnail view in OS Windows and open/save dialog boxs - even with LZW compression. But it chokes with a ZIP compressed TIFF.

Also, virtually all desktop scanners can produce TIFF images.

Many apps can create TIFF files, not just scanners. Scanners often create uncompressed TIFF files - but some may have the option for LZW compression. Years ago, I wanted to open up the TIFF's that the platesetter used, that were generated by the RIP. These were 1-bit images using some unknown compression scheme that Photoshop or other common image editors could not open.

Lightroom supports large documents saved in
TIFF format (up to 100 million pixels with pixel dimensions of no
more than 10,000 on a side). However, most other applications,
including older versions of Photoshop (pre-Photoshop CS), do not
support documents with file sizes greater than 2 GB.

Adobe offer the Large Document Format which is PSD without the size limitations.

The TIFF format provides greater compression and industry
compatibility than Photoshop format (PSD)... “

This is true, as long as there is no problem for the software in question decompressing the image. Then other users mention layers being a problem. What about Image Pyramid? What about Mac or PC byte order? This format has always had and is gaining more variables. This is fine, as long as all play equally and that there are good ways to avoid using the features that break various software.

MS Office will accept TIFF files, which is great for providing high quality logos for clients that do not artifact on their laser printers (like JPG does). The only problem is, MS Office is too "smart" and attempts to do things based on any alpha channels in the RGB file - so it is critical that no alphas are presented to say MS Word. This is one reason why I requested in the Bridge CS3 beta for the ability to display alpha channels along with other metadata inclusions if possible (more info about the file being a good thing, even more so without opening files).

Regards,

Stephen Marsh.
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Lee Clawson"
Mon Mar 5, 2007 11:42 am (PST)

on 3/5/07 5:08 AM, Stephen Marsh wrote:

LZW can cripple the file for many apps. If I recall correctly, older
versions of Xpress required an xtension (plug) to handle LZW TIFF files.

Stephen,

I don't recall any plug-ins for this. We did pre-check this with any printer we use and especially any packaging work being printed in Asia. Some RIP's seemed to "choke" when encountering LZW.

Lee Clawson
2/\V/\7 Studio
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "RJay Hansen"
Mon Mar 5, 2007 2:57 pm (PST)

Quark 4 (and 5) has an extension named "LZW Import".

RJay
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Todd Shirley"
Mon Mar 5, 2007 2:58 pm (PST)

Lee

Prior to version 4, Quark required an Xtension to import LZW compressed images. Also required an Xtension for jpegs.

-Todd Shirley
Urban Studio
New York, NY
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Lee Clawson"

On second look-- I just looked back at v.4.11 and as you recall we used an extension to read/place the LZW TIFF's.

Lee Clawson
2/\V/\7 Studio
___________________________________________________________________________

Re: What is the difference on Tiff and PSD
Posted by: "Kevin"
Mon Mar 5, 2007 9:20 pm (PST)

LZW can cripple the file for many apps. If I recall correctly, older
versions of Xpress required an xtension (plug) to handle LZW TIFF
files.

Stephen

One of Quark's selling points was always its ability to handle LZW- compressed TIFFs at least as early as v.2 (not sure about v.1; I only recall using it for plain typesetting). This was important back in the days of 1.4K modems and floppy discs. The problems came with the extra processing overhead required to decompress the files, which occasionally upset some RIPs. It only happened once in a blue moon, but when a man with a Komori is breathing down your neck, once in a blue moon is once too often. It was PDFs that needed a plug-in (and still didn't work with some RIPs/imposition software).

This is true, as long as there is no problem for the software in
question decompressing the image. Then other users mention layers
being a problem. What about Image Pyramid? What about Mac or PC byte
order? This format has always had and is gaining more variables. This
is fine, as long as all play equally and that there are good ways to
avoid using the features that break various software.

One of the problems with the TIFF format is that the standard has been regularly revised, then many software developers have added their own interpretations on top of that. There have been some alleged TIFF formats, particularly from older, more obscure Windows software, that gave trouble with anything other than the originating software. I recall a number of years ago reading an interview with some software developer who reckoned he'd identified over 40 variants of TIFF file format. Even these days I occasionally get lumbered with cleaning up TIFF scans of engineering diagrams for publication that use oddball compression formats, etc. They sometimes give Photoshop grief, but Graphic Converter usually copes. Uncompressed TIFFs make life less exciting, which is a good thing in this respect at least.

Kevin Crosado
___________________________________________________________________________

Re: TOPICWhat is the difference on Tiff and PSD
Posted by: Stephen Marsh
Mon Mar 5, 2007 11:36 pm (PST)

Folks, it would appear that the thread has run it's natural course and the moderation team requests that this thread close. Laurentiu and Arye are welcome to take their separate part of the discussion offline, if desired. Further posts on this separate submatter of the TIFF and PSD thread will be rejected.

John Ruttenberg has posted some thoughtful points on downsizing and USM, are there any takers?

Sincerely,

Stephen Marsh
List Co-Moderator
___________________________________________________________________________

File formats
Posted by: "Mike Davis"
Tue Mar 6, 2007 9:35 pm (PST)

In doing some comparison of file formats for the purpose of choosing an archive default, I took a RAW image from my Canon 20D (8.2MP) into the CS3 beta and saved it in a number of formats. I know Dan has commented on the changes in TIFF formatting, but I thought the following table might raise some eyebrows, if in fact I performed the process fairly and properly. The image was a sunset with minimal "busy" detail to allow for compression algorithms to do their work. Here are my results:

Original RAW image saved directly from the camera 7.47M

Saved as a TIFF file 16 bit, no compression 46.8M
Saved as a TIFF file 16 bit, LZW compression 57.3M (no, that's not a typo!)
Saved as a TIFF file 8 bit, no compression 23.4M
Saved as a TIFF file 8 bit, LZW compression 9.22M
Saved as a Photoshop (.PSD) file 16 bit 46.8M
Saved as a Photoshop (.PSD) file 8 bit 22.9M
Saved as a DNG file 16 bit, no compression 16.4M
Saved as a DNG file 16 bit, lossless compression 6.98M
Saved as a DNG file 8 bit, no compression 16.4M
Saved as a DNG file 8 bit, lossless compression 6.98M

Then the JPEG levels. Note that JPEG does not support 16 bit, only 8 bit
files:

Saved as a JPEG file level 12 - 22.9M
Saved as a JPEG file level 10 - 1.51M
Saved as a JPEG file level 8 - 0.87M
Saved as a JPEG file level 6 - 0.65M
Saved as a JPEG file level 4 - 0.43M
Saved as a JPEG file level 2 - 0.32M

There are several things that would seem apparent from this. First is that keeping a file as an original RAW file is arguably the best format for saving a file. The down sides are that you don't have the edits saved with the file (unless you save it after editing in Lightroom or ACR), and you risk having the format becoming obsolete in the years to come for your camera's current version of RAW.

Another lesson is that a TIFF file is no bargain in terms of file size. Note that it makes absolutely NO difference if you save the image as a 16 bit or 8 bit file, the filesize in TIFF if HUGE, even compared to the original RAW file. This is due in part to the fact that the camera only captures 12 bits of data, and the TIFF algorithm apparently inflates the data to fill out 16 bits worth. Those who argue that 16 bits of data is better than 8 bits need to realize that they are working with 12 bits of real data and 4 bits of synthesized data or space, not 16 bits of pure data. There ain't no free lunch, and it costs a bunch of dead file space to save it.

Yet another lesson is that LZW compression in a TIFF file does not work with 16 bit data, only 8 bit data, at least the CS3 version of LZW.

Moreover, using the Adobe DNG format (an attempt at a universal Digital NeGative format which is sputtering), can be a file size saver. But note that both the 16 bit and 8 bit sizes are identical, so there is no advantage to saving DNG files in 8 bit, and the DNG compression algorithm is pretty darn good compared to LZW.

Then we come to the much maligned JPEG. Note that level 12 JPEG has NO compression whatsoever! That suggests that level 12 is lossless! Then note the tremendous compression dropping just to level 10 which is still "maximum" in JPEG terminology.

More complex images will yield differing amounts of compression, and most of them will not show the file size savings that this image does. I realize this deviates a bit from color correction,

but I'd appreciate comments on the apparent discrepancies noted above.

Mike Davis
mldavis2 AT sbcglobal DOT net
<http://www.pbase.com/mldavis2/> http: //www.pbase.com/mldavis2/
___________________________________________________________________________

Re: File formats
Posted by: "jann lipka"
Tue Mar 6, 2007 11:32 pm (PST)

Mike ,

my guess would be tiff files with zip compression gives smallest size of all lossless formats ( and it supports layers as well ) even for 16 Bit.

I know zip compression is somewhat controversial, but it has been around for a while now ..

IMHO it is at least good for keeping files in storage . The only downsize I©ˆm aware of is longish save times .

Jann Lipka
___________________________________________________________________________

File formats
Posted by: "Andrew Rodney"
Thu Mar 8, 2007 2:39 am (PST)

Mike, it sounds like you©ˆre embedding the original Raw in the DNG hence it©ˆs larger file size. Personally I find that doing this brings nothing to the party. Yes, if you wish to use the camera vendors converter and it doesn©ˆt support DNG, that©ˆs an issue and a reason to embed Raw original into DNG but since I find that these products usually don©ˆt work all that well for me, I just produce a DNG. The file size should be smaller than the original RAW and you have an open format and ability to embed metadata instructions inside the container. You may want to add that to the matrix if for anything, feedback on file size of strictly a DNG.

Moreover, using the Adobe DNG format (an attempt at a universal Digital
NeGative format which is sputtering

Sputtering? You©ˆre sounding like Dan. What makes you say this? What statistics is this based on? Are you quite sure that less files are being converted or even directly captured into DNG today then a year ago? With more users having tools to convert to DNG (Lightroom for one) and more people educating photographers to the benefits of DNG, I©ˆd like to know how you©ˆve come up with this opinion (on your own).

Those who argue that 16 bits of data is better than
8 bits need to realize that they are working with 12 bits of real data and 4
bits of synthesized data or space, not 16 bits of pure data. There ain't no
free lunch, and it costs a bunch of dead file space to save it.

No there isn©ˆt but storage space is getting less expensive each month and my images never become less valuable. 12 bits of my camera data is far better data wise than 8-bits (the math is indisputable). My last concern is how much space a file takes up. I have the option to trash it if I wish (and I do). For the keepers, there©ˆs no way I©ˆd ever think of trashing the high bit DNG for a rendered 8-bit JPEG.

IF storage space is big deal and image integrity isn©ˆt, then by all means, go JPEG and trash the camera originals. I have no problem with that if the choice is an informed one. But this message isn©ˆt going to fly for photographers who are serious about their image archives. You©ˆd be hard pressed to recommend a photographer make a big print from their neg, then trash the neg. Its for this reason I fully support photographers who want to embed the original Raw into a DNG (you have additional advantages but as you point out, the bite out of the free lunch is file size).

If you©ˆre an inhouse photographer working in a printing house, shooting widgets on a white background all day, the idea of archiving the images in Raw or Raw+DNG could certainly be an issue with costs and storage. That©ˆs a far cry from a guy like me that will shot a good 100 gigs or more on a two week expedition down the Amazon river! I©ˆll be spending my idle time making multiple back-ups to multiple drives to do my best to ensure the data is coming home!

Andrew Rodney
___________________________________________________________________________

Re: File formats
Posted by: "Richard Wagner"
Thu Mar 8, 2007 2:40 am (PST)

On Mar 6, 2007, at 10:35 PM, Mike Davis wrote:

[snip] but I thought the following table might raise
some eyebrows, if in fact I performed the process fairly and properly.

Mike, it looks like you performed all the file saves "fairly and properly." However, I disagree strongly with your conclusions.

It is not news that a 16-bit file saved with LZW compression will be larger than the original, nor that your 8-bit uncompressed TIFF will be half the size of an uncompressed 16-bit TIFF, nor that LZW compression will make 8-bit TIFF files smaller. This is all old news.

Your DNG is a GRAYSCALE image that has not had Bayer de-mosiacing applied to generate all three RGB colors, so by its nature it is much smaller than a TIFF. It is also possible to losslessly compress 12- bit RAW files for storage on-camera, where storage space is at a premium, since even by bit-packing you can fit two 12-bit words into three bytes (24 bits) instead of four (32 bits). The downside is that the operation takes time. Thus for several reasons it is no surprise that DNG is smaller than TIF.

There are several things that would seem apparent from this. First is that
keeping a file as an original RAW file is arguably the best format for saving a file.

This conclusion is based only on file size. If for whatever reason your RAW converter drops from existence, or if the RAW converter changes how it renders files (Nikon Capture did this), then your RAW file will not be able to generate the image that you thought you "archived." A TIFF is the best format for archiving a RENDERED file. A DNG (or the native RAW file from your camera) is the best way to archive the "digital negative."

Another lesson is that a TIFF file is no bargain in terms of file
size.

No one (who knew what they were talking about) ever said it was!

Note that it makes absolutely NO difference if you save the image
as a 16 bit or 8 bit file, Actually, it makes a lot of difference.
the filesize in TIFF if HUGE, even compared to the original RAW file.

As I described above, this is an invalid comparison. The 16-bit TIFF will be twice as large as the 8-bit TIFF (both uncompressed), and both will be larger than the RAW file.

This is due in part to the fact that the camera only
captures 12 bits of data, and the TIFF algorithm apparently
inflates the data to fill out 16 bits worth.

This is pure manure!!! 12 of 16 bits contain "data," and the other 4 bits contain zeros. You can't store 12 bits of data in an 8-bit word, nor do computers generally operate with 12-bit words. The choices are 8, 16, or 32 bit words with all current processor architectures. "Bit-packing" is not a standard method of storage for computer file systems.

Those who argue that 16 bits of data is better than
8 bits need to realize that they are working with 12 bits of real
data and 4 bits of synthesized data or space, not 16 bits of pure data. There
ain't no free lunch, and it costs a bunch of dead file space to save it.

You don't understand data formats and how 12 or 14-bit data is represented within a 16-bit word. There is NO "inflation of data" or "synthesized" data - that statement is pure garbage. The higher bits are simply zeros. NOTHING is "synthesized." The "TIFF algorithm" does not "apparently inflate the data to fill out 16 bits worth." 16- bit data has nothing to do with the file format that the data is saved in, and most people working with "high-bit data" realize how many bits of "real data" they have, be it 12, 14, or 16 bits. The number represented by a 12-bit word will be EXACTLY the same in a 16- bit word! . If you convert an 8-bit image to 16 bits, all upper bits are zeros. No "data synthesis," nothing to do with TIFF. You need to realize that you're throwing away 4 out of 12 (or more) bits worth of data when you work with 8-bit files!

Here's the biggest number (4095, base 10) that you can represent in "12-bit data" shown in 12-bit binary:
1111 1111 1111
and in 16-bit binary:
0000 1111 1111 1111

And here's the number 1 in 12-bit:
0000 0000 0001
and in 16-bit:
0000 0000 0000 0001

How about that magic number, 255?
1111 1111 in 8-bit,
0000 1111 1111 in 12-bit, and
0000 0000 1111 1111 in 16-bit.

See the pattern? If you have a Mac, you can do this with the Calculator in Programmer's mode.

Yet another lesson is that LZW compression in a TIFF file does not
work with 16 bit data, only 8 bit data, at least the CS3 version of LZW.

Lempel-Ziv-Welch compression does not work well with 16-bit data, because there are few data *patterns* in 16-bit data that can be used by the algorithm. It is well-known that LZW is NOT useful for 16-bit image files. Notice, for example, that ACR does not even give the *option* for LZW when saving 16-bit files. Do you think the "stupid engineers at Adobe" simply forgot to include it? They "remembered" it for 8-bit files.... It was left out because it is well-known that LZW will INCREASE the file size of 16-bit images!

But note that both the 16 bit and 8 bit sizes are identical, so there is no
advantage to saving DNG files in 8 bit, and the DNG compression algorithm is
pretty darn good compared to LZW.

You're comparing a GRAYSCALE image to a RENDERED RGB image - apples and oranges!

Then we come to the much maligned JPEG.

Yea, poor JPEG...

Note that level 12 JPEG has NO compression whatsoever!

JPEG by its nature "has compression." I just took a 13.3 MB 8-bit TIFF and compressed it to 5.6 MB using JPG compression at 100%. Are they identical? No!!! JPEG is lossy. But is DOES compress images!

That suggests that level 12 is lossless!

What does? You have a misunderstanding about JPG compression.

More complex images will yield differing amounts of compression,
and most of them will not show the file size savings that this image does. I realize this deviates a bit from color correction, but I'd appreciate comments on the apparent discrepancies noted above.

Mike, I think you have some basic misunderstandings about how data is stored in 8- or 16-bit words, about the TIFF file format, and about JPG compression. None of what you presented is "new" or unpredictable. ZIP is a much better compression algorithm for 16-bit images - that's why it's included in ACR when saving 16-bit TIFs (and LZW is not). Unfortunately, there is a lot of software out there that cannot read ZIP-compressed TIFs, as ZIP-compressed TIFs are relatively new. Even the latest version of ImagePrint, an expensive RIP, cannot read ZIP-compressed TIFFs.

Uncompressed TIFF is still the gold standard for archiving RENDERED images. It is lossless, and every pixel is defined. It is not dependent on a program to correctly interpret "editing instructions" like RAW files are. High-bit TIF files contain the maximum amount of information for a rendered file. Period. There may be a significant loss of information when saving a 12 or 14-bit scan or digital capture as an 8-bit file. Whether there is or not depends on the image data.

--Rich Wagner
___________________________________________________________________________

Re: File formats
Posted by: "Chris Brown"
Thu Mar 8, 2007 2:42 am (PST)

When archiving, I used to compress all my files using Stuffit (and OS 9) but as storage became cheaper I stopped. It was cheaper to use more disk space than the time it took to compress GBs of files. In addition, when a client or stock request came in, the time taken to decompress became annoying.

Nowadays, everything is saved to external hard drives and archival DVDs. Both media is catalogued using iView.

Now that iView is a MS product, I believe its feature set will become bloated and its performance will become slow and buggy in future releases. Nothing in MS's software product line indicates otherwise. I'm now looking into other ways to catalog my digital library. I want something like mySQL or FileMaker Pro to handle my library but, at the moment, those interfaces are for geeks.

Cheers ~
Chris Brown
___________________________________________________________________________