Dan Margulis Applied Color Theory Archive: What Should Proper Sharpening Do?

Dan Margulis Applied Color Theory

What Should Proper Sharpening Do?

proper "sharpening" at all scales, ideas (and rants) about the unsha

Posted by: Jacob Rus

Fri Dec 5, 2008 8:32 pm (PST)

Hi all,

I thought I'd send along some thoughts about "sharpening", because I hope that others either know where to look for improved tools or how to bend Photoshop into a more useful form, or will perhaps have ideas of their own in response to my rambling. Hopefully the length and assumed experience here aren't too off-putting: improving local/edge contrast of images is a large subject, which could probably fill a book, so it takes a bit of explanation to get my thoughts across.

Motivation

----------

Whenever I have the time to spend on an image, and after having made global contrast adjustments through curves (or usually the "Jacob's ladder" technique) I use the unsharp mask tool (I'd rather call this "Gaussian sharpen", since it does not behave quite the same way a real photographic unsharp mask would, and because that would give a much better indication of how it operates and its relation to the Gaussian blur tool--the two could incidentally be combined into a more useful tool, more on this later--but at this point we're probably stuck with the name) repeatedly on the L* channel, at several radii, starting with a very large radius (e.g. 250 pixels--the inability to push this higher is unfortunate) at some extremely low amount (usually no more than 4 or 5 percent, but enough to be a larger effect than ultimately desired), and then repeatedly stepping radius down by half or so & amount up, as I apply the filter again & again, each time aiming for a somewhat excessive effect. I do this all the way down to 0.2 pixels or so. Then, I can dial down the opacity of my "sharpened" layer to an appropriate amount (I usually do this about half), and the result is that I end up with an image which has the acutance (local contrast) I desire at every scale, and which is therefore as "punchy" as I can manage (or sometimes it has intentionally subtle distinctions, &c.), while avoiding creating garish halos or excessively accentuating noise.

A bit more about my technical thoughts about this below, but first a practical question: the best image quality is clearly achieved if as many radius steps as possible are included in the process (with substantially diminishing returns in time spent after five or six applications of unsharp mask), because this allows for details at every scale to be properly highlighted. Unfortunately, I've had some amount of trouble coming up with appropriate actions to automate the process: I adjust the amounts by eye, and the properties of different images require different treatments. I admittedly haven't done *that* much experimentation along these lines though, so my first question is whether anyone has tried to design actions or scripts which facilitate improving local contrast at every scale in a simple straight-forward way. Anyway, this takes rather more time than I'd like it to; more discussion about that below too.

Preliminaries: How unsharp mask operates

----------------------------------------

Unsharp mask essentially decides the value of a particular pixel (call it p) by multiplying the existing value of that pixel by (x + 1), where x is the "amount" of the pixel, and then subtracting x times a weighted average of all the pixels in the image, where the weighting function weights pixels using a Gaussian or normal distribution based on their distance from the pixel p--a distribution with standard deviation equal to the "radius" setting.

This "works" (i.e. increases acutance) because it drives the value of the pixel *away* from the local average value in the region, making the pixel more extreme. Several nearby pixels with originally identical values, against a dissimilar background, will all be moved in the same direction, especially at their edges, and the result is that the values at the edges of shapes move apart. The effect is strongest on the pixels of delicate features & sharp corners, but also significant along straight edges, and weakest (predictably) in the pixels which most closely match the average of their neighbors. Along any boundary between shapes (of two solid colors), an "amount" of 100% will double the difference in values between pixels on opposite sides of the edge (unless radius is extremely small); if the edge is straight, each pixel will be pushed half of that difference away from their average; if a foreground shape is extremely thin, then its pixels will be pushed 100% of the difference away from the background, while the background pixels will be nearly unaffected.

Hopefully the above explanation all makes reasonable sense. It's a bit complicated and would probably best be illustrated with diagrams. Anyway...

Okay, so what does radius mean practically? Well, we can think of the Gaussian distribution ("bell curve") which makes our weighting function as diminishing the effect of each pixel on our pixel of interest as the distance between them increases, but alternately, and more practically, we can imagine that as we move away from the edge, the overall effect of the filter on pixels in a shape falls off with distance from the edge, so that once all of the pixels within two or three standard deviations (multiples of the radius) of our pixel average to the same value as our pixel itself, the effect becomes negligible.

Aside: T he Gaussian distribution makes computation fast for 1980s-era machines, because as it can be decomposed easily into horizontal & vertical components, and then is a convolution of the image with a simply computed kernel which can be done in several ways, but is (at least) fast & simple math if FFTs are used (look convolution up in wikipedia if interested).

What's the matter with bell curves?

-----------------------------------

Unfortunately, the Gaussian distribution is *never actually* what we want to use in "sharpening" an image. It can easily create garish halos, because we want strong effects at tight edges, but also want to take into account the contribution of medium-scale shapes. (The unsharp mask tool has other problems, discussed below, but I'll focus on this one for now.) So what happens is, either we split the difference, picking a radius & amount which negotiates that trade/off between sufficient edge contrast and unattractive side-effects, or we do multiple passes with the unsharp mask tool, at different radii, with each one targeting different scale shapes. Dan oddly names our increase of local contrast at different scales by two different names,† respectively "hiraloam sharpening" & "conventional sharpening", which is unfortunate for three reasons: (a) neither name gives much indication of what is being attempted; (b) the distinction between the two is completely artificial, as they simply operate at different scales, so that a viewer stepping back a few paces, or examining the image as a whole would notice one and a viewer peering closely at details would notice another; but mostly (c) it implies that the additive combination of two Gaussian distributions is the proper shape for enhancing local contrast, an idea not quite but nearly as incorrect as the idea that a single Gaussian distribution can cut it.

Why is the Gaussian distribution bad for sharpening? Because it is relatively flat for a while in the middle, and then drops off precipitously to almost zero. This characteristic "shoulder" is the halo created by the unsharp mask tool which is so distasteful when overdone. The distribution has one benefit that I can see, which is that it makes the radius simple to understand conceptually, and limited to a single slider, and is therefore beginner friendly. But really, Gaussian sharpening as it exists in the unsharp mask tool is designed for 80s hardware & unsophisticated users, and I am astounded that it has not been improved in 20 years of Photoshop ("smart sharpen" is some kind of tragic ironic joke).

How can we improve our "sharpening"?

------------------------------------

In any image, we must make critical choices about how much contrast to allocate to each region, each detail, &c.: this is in a nutshell the art of photography and of image-making more generally: it's the idea of Ansel Adams's zone system, and the main point of every book about color correction in Photoshop, though some of them haven't necessarily considered it out in those terms, and the inspiration for the various HDR tone-mapping algorithms which have been developed over the past few years, including Photoshop's shadow/highlight tool, &c. In an existing image, what makes up details (shapes and textures) is contrast at different scales/spatial frequencies, in different parts of the image--this is the fundamental insight behind Fourier decomposition of images or other signals into a frequency domain, and we can think of "sharpening" and similar operations as operations in the spatial frequency domain in order to change the visual effect in the spatial domain. Anyway, I'll cut the math jargon so I don't start losing less technical readers. Basically, our image has different amounts of contrast at different scales, and that makes up the shapes that we see. If we start taking up more contrast in the small scale, we run up against overall limits in the amount of contrast we have available to us, and we either have to reduce overall ("global") contrast in the image, or we start to create nasty artifacts.

Okay, so when we apply unsharp mask to an image, what we're trying to do is bring out (i.e. make visually distinct and interesting) features: shape details, textures, and so forth. These features have different sizes, and contrast to different degrees. Humans generally find images with more distinct features at every perceivable scale to be the most interesting. So what we want to do is usually bump up the edge contrast for shapes of every size, so that, for instance, a tree contrasts with the grass behind it, and then individual leaves in the tree contrast with each-other, and then, at the smallest scale, the texture of the leaves is also apparent, and their edges have sharp definition and tiny detail.

If we use a Gaussian function for weighting our sharpen, we can increase the contrast in one such distance range, but instead of making razor-sharp-looking edges, we actually make somewhat wide edges, with definite halos with their own distinct shapes. This is because the Gaussian function has horizontal slope at 0, and then a "bell top" region of significant effect between 0 and 1 standard deviations from 0, and then very quickly falls off towards zero from there, and by 3 standard deviations has almost no effect whatsoever. Okay, so what happens is our pixel moves away in value from other pixels which lie at particular distances--mostly between 0 and 2 standard deviations. This has the effect of especially heightening any details which are overall 1 or 2 standard deviations in size. Anything bigger will have its edge contrast increased with a halo extending about 1 or 2 standard deviations away from the edge, and anything smaller will be effectively un-enhanced.

Because we want to highlight delicate details as well, and avoid halos, we want to instead use a weighting function which immediately starts decreasing at 0 (unlike the horizontal slope at 0 of the Gaussian function), and which does not go to zero quite so fast as we get away from 0. Perhaps some function which looks like `y = e^(-x)`. To construct such a function out of the unsharp mask tool involves its repeated application. We need to carefully eyeball the image after each step, and try to make sure that we overdo our sharpen by about the same relative amount each time, so that when we then cut global opacity on our newly-sharpened copy of our image, we'll end up with various distance ranges sharpened appropriately. This takes practice to learn, and time and finesse to perform; it's tedious and easy to screw up. Overall, we'd really prefer to just use a better tool. Still though, it's usually worth running unsharp mask at least four or five times, if not more, because the improvements to images are quite dramatic, and the result is significantly better than only one or two applications of the tool can accomplish.

What do we want instead?

------------------------

What would I prefer? Well, first and most simply, I don't understand why Gaussian blur and unsharp mask need to be separate tools, or why "sharpen", "sharpen more", "sharpen edges", "blur", or "blur more" even exist (they're utterly pointless). Gaussian blur is exactly the unsharp mask tool, where "amount" has been set to -100%, and threshold has been set to 0. I also can't really imagine uses for "box blur", "shape blur", "radial blur", but I can see that someone might imagine such a use; all three are nonetheless extremely simplistic, and radial blur in particular could be fairly easily beefed up for sophisticated users without sacrificing comprehensibility for novices. I think having all of these myriad menu options brings Photoshop down for everyone; many of these tools are so simplistic even novices have no use for them, but their existence bothers every advanced user who must visually scan past them every time Photoshop's menu is examined. Anyway, that rant is a bit off-topic.

What I'd like to see in an ideal world is a tool which provides something like a curve of some sort, with radius (probably in logarithmic scale) along the horizontal axis, and amount (maybe also logarithmically) along the vertical axis. I'd be able to adjust the amount of effect at every scale by drawing a curve in a box, and in a single interface with a real-time preview perform what now takes infinite (or to quite roughly approximate, at least 5 or 6) invocations of the current unsharp mask tool. This would have several benefits: it would let us sharpen different scales using a single tool, and would take a lot of the guess-work out of the process. It would allow the various inputs to the process to be figured out, and then applied selectively, with some deferred for the end--for instance, this would allow fine detail sharpening to happen very late in the image processing cycle, so that small exaggerations to noise wouldn't be further amplified by intermediate retouching steps. My ideal-world tool would allow the "amount" axis to range from at least -100% to +500%, so that blurring and sharpening at different scales could be achieved.

To be especially sophisticated, the tool would also have another curve by which I could set how much the effect should be at different amounts of contrast, so that I could accomplish the same thing the current threshold slider does, but with more nuance, and also could cut or cut out the sharpening effect at the top (already high contrast) end. We could, in other words, break down the linear transformation of edge contrast provided by the current tool. And the tool would maybe even have a way of separating the effect in highlights, midtones, and shadows.

This would require some amount of serious investment in experimentation and testing, to figure out the proper parameters to expose to the user, and possibly even some basic research in human vision, or at least the consideration of previous basic vision research. Designing a decent interface which didn't overly confuse new users would take considerable usability testing and implementation effort.

What can we build for ourselves?

--------------------------------

Here's where I'm hoping for some advice from the list. I still need to experiment, but I've been having trouble constructing actions to make manifest my desires for a sharpening tool. If I want to counter effects at different lightness levels, in different difference ranges, and so forth, I need to get really creative, with, for instance, putting curves on specially-constructed high-pass filter layers (I am not precisely sure what Photoshop's high-pass filter does--I should do some more experiments, but if anyone knows the technical details I'd be obliged), with masks built to hit mainly highlight or mainly shadow regions, etc. Some of this may be figure-out-able and generalizable, and scripts or actions could maybe be constructed to perform the desired movements. But I haven't figured anything out yet, particularly. Alternately, maybe there's a good tool outside Photoshop or in a plugin that does a better job at improving local contrast at various scales from small to big, and gives the operator lots of control. I'd like to hear about it.

I'm going to do some experimentation in about a month, during the finals period of this semester when I have a bit more time, and I'll report back with any progress I make. I'm hopeful that Photoshop can be hacked into shape, but am still looking for inspiration.

Two other problems with unsharp mask

------------------------------------

Unsharp mask also simply multiplies the value difference between adjacent shapes, in a linear way, and then has a single lower-bound cut-off point (called "threshold"), which prevents noise, and mostly this means color noise in RGB images, and especially in the R & B channels (not to suggest that Fats Domino or Ray Charles just makes noise!!), from being amplified, when overall there is not so much difference between pixels. First, the operation of this cut-off is quite arbitrary from the perspective of what the user needs it for, and indeed is mostly only needed because unsharp mask computation is performed in a brain-dead color space like RGB or CMYK, where such disproportionate color noise can "infect" channels which also contribute to overall value/lightness. But more importantly, this linear effect is bad because it undesirably excessively accentuates already strikingly contrasting shapes. That is, because the multiplication of contrast is linear, two strongly contrasting regions are spread apart with *even more* distinction than two less contrasting regions. What we actually want is to increase the contrast more among similarly valued pixels, so as to drive them apart. Two regions of extreme contrast only need a slight contrast boost, if any. In other words, we get really ugly excessive halos and we often over-sharpen fine details with already high contrast, or already rough textures, when all we really wanted to do was bring out edges and bring blurry shapes into focus. A truly sophisticated tool would provide some kind of curve or set of sliders for determining the shape of the fall-off at both low and high ends, so that a more accurate low-bound "threshold" could be determined, and also so that the effect could be reduced or eliminated (or perhaps sometimes even reversed) for very high contrast shapes.

Unsharp mask has the same effect at every absolute value level. Often, highlights or shadows call for differing strength effects, or even for different scale effects. This can be done through masking & the changing of blend modes, in various combinations, but that's a giant bother, and somewhat hard to get right at the edges of the masked regions. The "smart sharpen" tool does not properly address this set of concerns, though I guess at least they're trying.

Conclusion

----------

I think there's considerable remaining space for experimentation, both within Photoshop and without, first for the creation of algorithms which sharpen the image as we would like, and second for novel user interfaces for controlling their parameters. Even in the absence of new tools, I imagine that some of Photoshop's tools are hackable to give us some of what we want. At some point, I'll try to do some more careful and exhaustive experimentation. If others on this list have pointers though, or really any kind of responses, I'd be much obliged.

Do you think this is a useful way to discuss local contrast enhancement? Is there any good way of explaining the concept to novice users? Any diagrams which would do the trick? At some point I'd like to write some of these ideas up with example images and a bit of a tutorial bent, as I think even most experts don't think carefully about what sharpening really means.

Anyway, I should get back to real work now, as I've spent a whole afternoon on this email that I didn't really intend to spend. Apologies for its length and any excessive repetition or confusing description, or for anything I inadvertently left out.

Cheers,

Jacob Rus

† I hope that Dan doesn't take this as personal criticism. The oddity here extends to a whole industry of image experts and the whole software market, and Dan's peculiar terminology is far from the worst of it, though to be honest, I'd have expected a catchier term than "hiraloam" (what a mouthful!) and a more descriptive one than "conventional" even if a particularization is needed to denote local contrast improvement for, respectively, medium- & small-scale features.

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: George Machen

Sat Dec 6, 2008 10:55 am (PST)

Jacob Rus wrote:

... Dan oddly names our increase of local

contrast at different scales by two different

names,† respectively "hiraloam sharpening" &

"conventional sharpening", which is unfortunate ...

I actually take a complete opposite stance here. Indeed, I go so far as to object to even Dan's use of "hiraloam" and "sharpening" in the same breath, because I consider the two effects on images as qualitatively different enhancements:

- I regard sharpening as increasing edge definition, or acutance.

- I regard hiraloam or local contrast enhancement as shaping or modeling.

Even though both involve the same operational process at different scales, I think their effect on the image to better simulate human perception are in two distinct realms. I see use of the USM tool to attain shaping or modeling of image subject matter as a clever "abuse" of the tool outside the scope of its normal application, akin to use (abuse) of "false profiles" or "false separations" as swashbuckling methods to obtain different results unrelated to their intended purposes.

I think Jacob's subsequent remarks actually bear me out, but where he sees a continuum, I see a sharp conceptual break. I fear that not seeing them separately may lead to yet another flavor of a long-standing problem that has for so long plagued the color correction field: While Jacob's descriptions of his process of correcting images do indicate that he duly & thoughtfully considers their significant content subjectmatter, nevertheless my "Calibrationist radar" went off while reading his remarks (possibly knee-jerk; I hope I'm wrong), and I feel like I can already see it coming: a whole new initiative by the Pixel Mafia to take over an entire industry with "appearance modeling" involving automatic adjustment algorithms that ignore the individuality of image subjectmatters that only human judgement can discern.

I do like Jacob's proposed sharpening tool that uses curves (not Lightroom-type sliders!) to adjust various parameters. I can see how it could simplify our present cumbersome use of blend mode layers, masks, and such to accomplish our task.

Digression: Inasmuch as we're at the moment stuck with the cumbersome use of blend mode layers, masks, and such, there's an interesting thread over on DPReview about sharpening through a type of edge mask I've not heard of before:

http://forums.dpreview.com/forums/readflat.asp?forum= 1006&thread=29402337&page=1

It's a method for microdetail enhancement to sharp edges without introducing halos. Whereas Photoshop's USM presumably depends on some difference of blurs for edge detection (note: I have been led to understand that Photoshop's USM does not use Gaussian Blur, but one still gets those halos!), this guy's procedure uses an edge detection of a prior edge detection. A "second order" edge detection, if you will. Or a *rate of change* of edges.

- George Machen

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: "Francis Corvin"

Sat Dec 6, 2008 2:22 pm (PST)

Jacob,

I can't yet answer your questions, but I wanted to say I have rarely read such a cogent, clear and insightful explanation of the shortcomings of USM. I find it accurate in that it does reflect my own limited experience.

The idea that the series of stepped USMs attempts to approximate e^-x makes a lot of sense in theory (I haven't tried it in practice). However, using e^(-x) makes an arbitrary balance between sharpening at all scales that may not suit everyone (at some point, when I'm not happy with the overall contrast, I prefer to play with an S curve than a vast and unwieldy USM). Therefore you might want to consider an additional slider n such that you can work on e^(-x^n)... which would give you Gauss if you so wished.

Of course, the very advantage of the fast drop-off in a Gaussian blur is that it imposes a limit on the computation and therefore is a fast algorithm. So e^(-x) --- let alone a^(-x^n) --- may be too computationally intensive for today's time-sensitive workflows.

Best regards,

Francis Corvin

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sat Dec 6, 2008 7:51 pm (PST)

George Machen wrote:

Indeed, I go so far as to object to even Dan's

use of "hiraloam" and "sharpening" in the same

breath, because I consider the two effects on

images as qualitatively different enhancements:

- I regard sharpening as increasing edge

definition, or acutance.

- I regard hiraloam or local contrast enhancement

as shaping or modeling.

Even though both involve the same operational

process at different scales, I think their effect

on the image to better simulate human perception

are in two distinct realms. I see use of the USM

Well, the question I suppose is whether it's actually helpful to draw a bright line, and where? Between 1 px and 10 px somewhere? Between 10 px and 100? The visual effect depends on the distance of the viewer, and even what he's looking for when he examines the image. There's a clear trade-off between large-scale and small-scale contrast, because we have only a fixed amount of contrast to allocate (namely, between off and full brightness for a computer display, or between substrate and solid ink coverage for reflective media). So I like to think of the choices to "blur" or "sharpen" contrast at various scales as being significant artistic choices. Some of my favorite photographers have images which look very flat from a distance, because they instead focus on maximally emphasizing local details and textures. Others push whole regions of their photographs into near complete obscurity, because they want to emphasize dramatic large-scale contrast shifts between shapes. Even increasing acutance at the smallest scale to prevent an image from appearing blurry or out of focus has some (often substantial) impact on this trade-off.

tool to attain shaping or modeling of image

subjectmatter as a clever "abuse" of the tool

outside the scope of its normal application, akin

to use (abuse) of "false profiles" or "false

separations" as swashbuckling methods to obtain

different results unrelated to their intended purposes.

Hmm. I'd be somewhat interested to hear the history of the unsharp mask tool in graphics applications such as photoshop, and also to hear whether film photographers ever made dramatically unsharp masks to boost contrast at medium scale. I imagine that the high-end imaging workstations had similar tools before PS did, but I really have no idea. In any case, I'd hesitate to ascribe intent to programmers I've never spoken with.

I fear that not seeing

them separately may lead to yet another flavor of

a long-standing problem that has for so long

plagued the color correction field: While Jacob's

descriptions of his process of correcting images

do indicate that he duly & thoughtfully considers

their significant content subjectmatter,

nevertheless my "Calibrationist radar" went off

while reading his remarks (possibly knee-jerk;

I hope I'm wrong), and I feel like I can already

see it coming: a whole new initiative by the

Pixel Mafia to take over an entire industry with

"appearance modeling" involving automatic

adjustment algorithms that ignore the individuality

of image subjectmatters that only human judgement

can discern.

I do like Jacob's proposed sharpening tool that

uses curves (not Lightroom-type sliders!) to

adjust various parameters. I can see how it could

simplify our present cumbersome use of blend mode

layers, masks, and such to accomplish our task.

As a general principle, I dislike giving over control of any aspects of the image creation process unless I know precisely what they're doing. For instance, I am willing to enable autofocus and auto exposure metering only when I expect them to behave predictably. When it comes to software tools, I would much rather see software vendors such as Adobe aim to make tools which can be used quickly and effectively *with some experience*, but which are also flexible and put as much control as possible into users' hands. "Push-button" algorithms are nowhere on my agenda. That said, if some automatic algorithms can be developed which in one click dramatically improve most images, I wouldn't be opposed to revealing them: plenty of users just want to shoot and print snapshots without careful aesthetic judgment, and there's no reason not to provide them with best possible automatic output. Why do you consider automatic tools a plague?

masks, and such, there's an interesting thread

over on DPReview about sharpening through a type

of edge mask I've not heard of before:

http://forums.dpreview.com/forums/readflat.asp?forum= 1006&thread=29402337&page=1

This is a bit long and complex, but I'll try to read and digest when I have some time. Looks interesting anyway.

(note: I have been led to understand that Photoshop's USM

does not use Gaussian Blur, but one still gets those halos!),

It seems to me to have a pretty darn Gaussian looking shape: I'll try to see if I get identical results when I sharpen some shapes using Gaussian blur instead; my impression is that I will. If that is inaccurate though, I'd certainly be interested to hear more precise details of what the tool is doing.

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: "Alex Kent"t

Sat Dec 6, 2008 7:51 pm (PST)

jacob,

thanks for this post, it was an interesting read.

i can reply to a few points quickly: unfortunately "sharpen", "sharpen more", "sharpen edges", "blur", or "blur more" all exist for historical reasons. presumably adobe are frightened of breaking people's recorded Actions and bad habits if they remove these.

"shape blur", "radial blur" - useful for creating; a simplistic example, to set a still wheel to a spinning.

the gist of your Filter Menu rant, i completely agree with. most of the Filters menu is antiquated crap, with horrible ui and very limited function. as you pointed out, there are so many filters included that each one gets diminished.

with regard to the wider argument for a more modern approach to sharpening;have you ever seen the 'KPT Equalizer' filter ? the KPT Equalizer, as i understood it, was a 'graphic equalizer' style row of vertical sliders each which controlled a different radii of USM/ blur. sliding each slider changed the amount +100% (sharpen) to -100% (blur) for each radius.

whilst this isn't actually using a continuous curve for the ui (which is an elegant idea i think), this filter had a similar function to that which you're suggesting: it allowed you to sharpen or blur details at many different scales all from one interface in a single pass.

i believe it was included with the KPT-X set, the last set of KPT filters released. i think there is a macosx version (as opposed to OS9) but i don't have it to hand so can't check if it works with a current Photoshop. i experimented with it for a while, but in the end the 'unique' interface of all the late KPT filters became too frustrating to use regularly.

alex kent.

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sat Dec 6, 2008 7:51 pm (PST)

Jacob Rus wrote:

It seems to me to have a pretty darn Gaussian looking shape: I'll try

to see if I get identical results when I sharpen some shapes using

Gaussian blur instead; my impression is that I will.

I just tried this, and verified that unsharp mask does in fact do the precise opposite of Gaussian blur. To see this for yourself, try the following:

* Get a low-contrast image in grayscale mode (something like all values falling between L* =40 and L* = 60; grayscale so we only have one channel to worry about )

* Make four copies of your background on separate layers.

* Apply unsharp mask to the topmost layer, 10px radius, 100%, no threshold. Set its blend mode to difference, and merge down onto the next layer below

* Hide that layer, apply Gaussian blur to the next layer down, 10 px, set *it's* blend mode to difference, and merge it onto the layer below

* Toggle between the two layers, and notice that they are identical

This happens because the difference blend mode is the *absolute value* of the difference.

Interestingly, high-pass filter doesn't seem to precisely have the same effect as Gaussian sharpen (unsharp mask) or Gaussian blur. It's fairly similar though. To use a high-pass filter for blur or sharpen, do these steps:

* Duplicate your background layer

* Run high-pass filter on the duplicate layer with the desired radius

* Set the blend mode to linear light

* Add a solid color adjustment layer, set to exactly middle gray, set its opacity to 50%, right click it and select "make clipping mask". Name this layer "cut 50%".

* Add a curves layer, with the curve inverted, so that it goes from the top left corner to bottom right corner of the box, and set this layer to also be a clipping mask. Name this layer "Invert".

* Toggle the invert layer on and off to swap between sharpen and blur.

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sat Dec 6, 2008 7:51 pm (PST)

Francis Corvin wrote:

The idea that the series of stepped USMs attempts to approximate e^-x

makes a lot of sense in theory (I haven't tried it in practice).

However, using e^(-x) makes an arbitrary balance between sharpening

at all scales that may not suit everyone

Yes, I don't imagine ce^(-kx) [or really any similarly simple formula; the gaussian function is just ce^(-kx^2)] is ever precisely the kind of function we want. I was just tossing it out so people could get a better visual picture of the sort of shape I was discussing. :-)

As I said, in my ideal-world tool, there would be a great deal of control over the various effect at different scales.

–Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sun Dec 7, 2008 6:00 am (PST)

Hi again everyone.

I really should be writing papers about, respectively, Latin American debt crises and the breakdown of the Breton Woods monetary system, but I got sucked into thinking carefully about this "sharpening" problem, and started reading the various links I could find, including that posted by George Machen, and have a (not-so-) quick response.

George Machen wrote:

Digression: Inasmuch as we're at the moment stuck

with the cumbersome use of blend mode layers,

masks, and such, there's an interesting thread

over on DPReview about sharpening through a type

of edge mask I've not heard of before:

http://forums.dpreview.com/forums/readflat.asp?forum= 1006&thread=29402337&page=1

It's a method for microdetail enhancement to

sharp edges without introducing halos. Whereas

Photoshop's USM presumably depends on some

difference of blurs for edge detection (note: I

have been led to understand that Photoshop's USM

does not use Gaussian Blur, but one still gets

those halos!), this guy's procedure uses an edge

detection of a prior edge detection. A "second

order" edge detection, if you will. Or a *rate of

change* of edges.

Actually, that's a somewhat misleading description of what it does (as far as I can tell from reading the text). What you wrote on that page is:

Since a Difference of Gaussians calculation is itself

a method of edge detection, what Joe0Bloggs is

doing would appear to be an edge detection of a prior

edge detection. A "second order" edge detection, if

you will. Or a *rate of change* of edges.

I've not seen that anywhere before, and I think its

use as a sharpening layer mask accounts for its

ability to single-out mid-contrast edges while largely

leaving high-contrast edges alone.

But actually, from what I understand, the intent in the method of mask construction is twofold:

1. Black out non-edge areas ("surface areas"), so that small noise won't be amplified. This is essentially an attempt to make an analogue of the "threshold" value in the unsharp mask tool which is slightly more sophisticated/adjustable.

2. Black out excessively edgy areas ("high contrast edges"), so that ugly halos and over-exaggeration of very rough/contrasty textures (for instance the highlights in hair) can be minimized.

To accomplish those two goals, this method subtracts two different "edge mask" layers from each-other, and uses the result as a new mask. The "edge masks" are constructed using the "find edges" tool, which basically results in a layer which is the absolute value of a high-pass-filtered image (there are various edge-detection algorithms which do variations on this theme; I'm not precisely sure how Photoshop's works), and one of the edge masks has its contrast changed using the levels tool, to force the highlights to clip. When this is subsequently subtracted from the other (notice, "subtracting" is different from "difference", the latter being the absolute value of the former), any areas which are particularly bright white end up getting knocked out of the mask. This is basically sets a particular "upper threshhold" on the mask as well.

Then this mask is used for a layer which is sharpened "conventionally".

What I wrote about the need for a way to deal with these two types of thresholds in my first email in this thread was:

Unsharp mask also simply multiplies the value difference between

adjacent shapes, in a linear way, and then has a single lower-bound

cut-off point (called "threshold"), which prevents noise, and mostly

this means color noise in RGB images, and especially in the R & B

channels (not to suggest that Fats Domino or Ray Charles just makes

noise!!), from being amplified, when overall there is not so much

difference between pixels. First, the operation of this cut-off is

quite arbitrary from the perspective of what the user needs it for,

and indeed is mostly only needed because unsharp mask computation is

performed in a brain-dead color space like RGB or CMYK, where such

disproportionate color noise can "infect" channels which also

contribute to overall value/lightness. But more importantly, this

linear effect is bad because it undesirably excessively accentuates

already strikingly contrasting shapes. That is, because the

multiplication of contrast is linear, two strongly contrasting regions

are spread apart with *even more* distinction than two less

contrasting regions. What we actually want is to increase the

contrast more among similarly valued pixels, so as to drive them

apart. Two regions of extreme contrast only need a slight contrast

boost, if any. In other words, we get really ugly excessive halos and

we often over-sharpen fine details with already high contrast, or

already rough textures, when all we really wanted to do was bring out

edges and bring blurry shapes into focus. A truly sophisticated tool

would provide some kind of curve or set of sliders for determining the

shape of the fall-off at both low and high ends, so that a more

accurate low-bound "threshold" could be determined, and also so that

the effect could be reduced or eliminated (or perhaps sometimes even

reversed) for very high contrast shapes.

But really, the solution offered in the link is a decidedly wrong way to handle this, because a) it is extremely inflexible, and b) there's no reason to believe the mask so constructed looks anything like ideal, no matter what the image.

In general, I think the idea of making "edge masks" for use with the unsharp mask tool is quite laughable, and is basically jumping through lots of unnecessary hoops by users who haven't really considered the straight-forward implications of the tools, and are instead blindly stumbling about using seriously ugly (and undoubtedly inaccurate) kludges based on these imprecise understandings. They're using one type of spatial frequency filter (the mask) to reduce the effect of another spatial frequency filter (the unsharp mask layer), when all the data needed for decision about how much to apply the effect at each pixel is *right there* in the effect of the unsharp mask itself.

Because the effect of unsharp mask is linear with respect to the amount of contrast at the edge, all we need to do is one of (a) separating out the effect of the unsharp mask tool or (b) just using high-pass filter directly, and then just apply a curve to these effects, where we damp down small changes ("threshhold" or "edge mask"), leave medium-level changes applying substantially, and then damp down big changes (this new type of "refined edge mask") as well. This looks like a curve in a shape something roughly like a demented sine wave, which is at 0 at the left, middle, and right, and has a flat slope in the middle to boot, and then swoops down to the mid-left and up to the mid-right. After constructing our curve, we put the whole set in linear light mode. We can adjust this curve to our heart's content and finely control the thresholds at both ends after the process is finished.

Note: this explanation is a bit tougher I fear than my first email, as it really needs diagrams and examples to explain. Apologies. Shorter version: all these "edge mask" sharpening techniques are inventing work for themselves, and in the process they make their operation less understandable, and much less flexible, than just doing the straight-forward thing.

Further note: the fact that people are doing this should imply to Adobe that something is broken that could be greatly improved, and they'd in so doing save their users a lot of extra trouble.

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Stephen Marsh

Sun Dec 7, 2008 6:01 am (PST)

Jacob, firstly, thank you for the topic. I can see that it has a lot of potential, please do get back to the list when your other studies are over.

Your post is timely, I have recently been playing with this subject (yet again). I too will let the list know when I have something to report.

As your recent posts cover a lot of ground, feedback may be slow, so please do not feel discouraged if responses are not as you expected. I think that it will be easier for me to "cherry pick" selected comments from your posts, so please forgive any excessive snipping of your original comments, which are appreciated and will be taken into full account (we can all review the original text in context if required).

In reply to - http: //tech.groups.yahoo.com/group/colortheory/message/20703

Jacob Rus wrote:

Interestingly, high-pass filter doesn't seem to precisely have the

same effect as Gaussian sharpen (unsharp mask) or Gaussian blur. It's

fairly similar though.

If I remember the image processing theory correctly, USM and High Pass are doing different things that have similar results, perceptually and when compared with difference tests. Apologies in advance if there are any errors, this is just meant to be a very brief overview in order to perhaps flesh out your quote.

* USM creates the edge difference by subtracting a blurred version of the original image from the original - the resulting difference being between the original and the blurred image. This region of difference decides the areas that will be sharpened.

* High Pass is a [secret sauce] band pass filter, passing through frequencies that are high, while discarding lower frequencies.

* Subtracting a Gaussian blur from the original, normalised via a mid gray offset superficially resembles the Photoshop High Pass filter, however as you note it is not exactly the same, which is why I believe that there is some secret sauce in the Adobe High Pass filter, as it is not as simple as subtracting an inverse Gaussian of the same value (sometimes named Laplacian, although the Laplacian is more commonly detected via a simple convolution kernel and not the more complex Gaussian subtraction). I think it is safe to say that Adobe sometimes add some "secret sauce" to some commands, which may share a "standard" name or definition in other similar software, while at other times they give different names to the same functions as found in other software.

* As the principle of using the difference between the original image and a Gaussian blurred version demonstrates, one can sharpen an image. This principle is extended into finding the difference between two different Guassian blurs, such as found in the "Difference of Gaussians" or "DoG" sharpening approach, which is less sensitive to noise (independently of a threshold function).

* These basic principles can be taken further with say Difference of Median, or Difference of Gaussian to Median. It could be the difference of Smart Blur, Surface Blur, Lens Blur with alpha or many other softening methods.

As one can create the appearance of sharpness by finding differences, one could use the difference that is generated by a softer resampled image to the original. The variations go on and on. As there are many ways to blur or soften an image, one can sharpen with any of these methods.

Although some of these hybrid options are novel and may not have been created or used even today, as noted they are still using the same basic principle that has been around for decades.

So what of the present/future?

Some methods may use deconvolution restoration rather than sharpening to achieve similar results. Technically it is not the same as USM or whatnot, however an average person would probably not be able to tell the difference - both would appear to be sharper than the original. There are of course many problems with this approach, however it does appear to have potential.

Another approach that appears to be gaining some attention is wavelet based sharpening.

My search for information on these last two topics has taken me away from graphic arts and photography and into astronomy and medical imaging - not to mention the odd trip to machine vision and other places that are far beyond my humble background. One problem with this is that the only good info is in published journals, which are mostly at a level beyond my full comprehension (not to mention the cost per article). Finding freely published work is harder, it is mostly from vendors rather than neutral sources.

Sincerely,

Stephen Marsh

___________________________________________________________________________

proper "sharpening" at all scales, ideas (and rants) about the unsha

Posted by: "John Ruttenberg"

Sun Dec 7, 2008 8:45 am (PST)

I've been thinking about implementations which could enable some experimentation with these ideas. For the more advanced ideas of using some different math for the filter, it seems there is some sort of ability to do this in Matlab, which might help a lot. I've never done anything like it, but there is an interface and a Matlab image processing library. I wonder if anyone on the list has ever used this to make a plugin? Or just to experiment with images (perhaps outside of Photoshop.)

I also thought about how to play more flexibly with multiple applications of USM at different radii. What about making 5 or 6 smart filter layers, one for each of the radii? Instead of using a very low amount in the USM dialog, use a very low opacity for the layer itself. Once the layers are created, their opacity can be adjusted independently and even their thresholds can be changed. The actual creation of the smart filter layers can be automated easily into an action, leaving only the "small matter" of adjusting the opacity of the layers. I think this makes a much easier platform for exploring this technique than trying to guess ahead at the relative amounts of the USM applications.

--John Ruttenberg

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sun Dec 7, 2008 9:47 am (PST)

Stephen Marsh wrote:

* USM creates the edge difference by subtracting a blurred version of

the original image from the original - the resulting difference being

between the original and the blurred image. This region of difference

decides the areas that will be sharpened.

* High Pass is a [secret sauce] band pass filter, passing through

frequencies that are high, while discarding lower frequencies.

Okay, but both of these are effectively band-pass methods. The question is what the "secret sauce" is. It's possible that Photoshop's high-pass filter is closer to what we actually want than unsharp mask is.

* Subtracting a Gaussian blur from the original, normalised via a mid

gray offset superficially resembles the Photoshop High Pass filter,

however as you note it is not exactly the same,

You're right, it's not the same as a high-pass filter. It is however exactly the same as subtracting the original from an unsharp masked version.

(sometimes named Laplacian, although the Laplacian is more commonly

detected via a simple convolution kernel and not the more complex

Gaussian subtraction).

A Gaussian blur is just a simple convolution, so these would be the same operations.

* "Difference of

Gaussians" or "DoG" sharpening approach, which is less sensitive to

noise (independently of a threshold function).

This is basically the same idea as the threshold, though the implementation might be slightly different.

* These basic principles can be taken further with say Difference of

Median, or Difference of Gaussian to Median. It could be the

difference of Smart Blur, Surface Blur, Lens Blur with alpha or many

other softening methods.

As far as I know, median works on squares, resulting to some extent in square artifacts. I don't quite know what smart blur or lens blur do, technically, so I can't quite imagine what effect their various settings would have on an attempted sharpen. It would take understanding technically what these do before they could really be put to work.

As there are many

ways to blur or soften an image, one can sharpen with any of these

methods [...] some of these hybrid options are novel [... but they] are

still using the same basic principle that has been around for decades.

Sure. The question then is deciding what we actually want to do, when editing images; my suggestion is that we want to increase contrast using a radially-symmetric kernel which privileges fine detail but then falls off more slowly in its tail than a Gaussian function, and then we want to apply a curve to the resulting differences, so that we can chop out the effect on low- and high-contrast areas.

I think I can actually set something reasonably flexible up as a series of layers with adjustable opacities, and then a curves layer at the top of them, with the whole group set to linear light mode.

Some methods may use deconvolution restoration rather than sharpening

to achieve similar results.

Personally, I don't expect anything so fancy is needed. What we want is to give users control over contrast at different spatial frequencies, not necessarily to precisely counteract blur caused by a camera lens, though here I'm talking more about creative uses than salvaging horribly blurry original images; others likely have a different agenda.

Another approach that appears to be gaining some attention is wavelet

based sharpening.

This could be. I haven't ever seriously studied signal processing. I don't expect though that my first-order goals are un-meetable using Fourier-transformed data.

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Sun Dec 7, 2008 12:06 pm (PST)

Okay, here are some example images, so people have some idea what I'm talking about.

First, Dorothea Lange's classic "Migrant Mother":

<http: //www.hcs.harvard.edu/~jrus/colortheory/sharpening/mm-orig.jpg>

Then, the same, with unsharp mask applied about 6 or 8 times at different radii (I don't remember the precise settings), to improve local contrast:

<http: //www.hcs.harvard.edu/~jrus/colortheory/sharpening/mm-multiscale-sharpen.jpg>

This version has some unfortunate nasties, because the already high-contrast areas have been "over-sharpened", and also noise has been increased dramatically. So, we construct a layer which defines the difference between sharpened and unsharpened versions, and then apply a curve to it which damps small effects (to reduce noise), and also damps large effects (to avoid over-sharpening).

Interface screenshot:

http: //www.hcs.harvard.edu/~jrus/colortheory/sharpening/mm-curve-screenshot.png

Final image:

http: //www.hcs.harvard.edu/~jrus/colortheory/sharpening/mm-multiscale-curved.jpg

* * *

It's quite possible that my settings here are not ideal. But hopefully this helps others figure out the kinds of things I'm thinking about.

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Stephen Marsh

Thu Dec 11, 2008 5:36 am (PST)

In reply to:

http: //tech.groups.yahoo.com/group/colortheory/message/20710

Stephen Marsh wrote:

* USM creates the edge difference by subtracting a blurred version

of the original image from the original - the resulting difference

being between the original and the blurred image. This region of

difference decides the areas that will be sharpened.

* High Pass is a [secret sauce] band pass filter, passing through

frequencies that are high, while discarding lower frequencies. < <

Jacob Rus replied:

Okay, but both of these are effectively band-pass methods.

The difference between a sharp and soft version of the image is slightly less sensitive to noise than High Pass which contains the original high frequencies, which are then amplified and added to the original image via a contrast enhancing blending mode. Just as the difference between two Gaussian blurs is less sensitive to noise than is the difference between the original and a blurred copy. Just attempting to note the main theoretical differences, which may or may not mean much in final application.

Stephen Marsh wrote:

(sometimes named Laplacian, although the Laplacian is more

commonly detected via a simple convolution kernel and not the more

complex Gaussian subtraction).

Jacob Rus replied:

A Gaussian blur is just a simple convolution, so these would be the

same operations.

Ah yes, although the laplacian sharpen custom convolution kernel is much simpler (faster) than the Gaussian kernel, although as you say both are simple convolution operations.

The point I was making was that most would use a simple Laplacian kernel to find the Laplacian. However, there is an area in image processing known as Gaussian and Laplacian Pyramid processing. In this context, the Gaussian is used to create the Laplacian, instead of the simpler Laplacian kernel.

I don't have time to go into the rest of the message Jacob (it is much as above), I don't think that we are missing others main points although some of the small ones may be lost, however if a comment needs clarificaton, please ask.

Regards,

Stephen Marsh

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: "John Ruttenberg"

Thu Dec 11, 2008 6:56 am (PST)

A copule of years ago I audited a Computational Photography course at MIT. The course notes (in the form of PowerPoint slides) are still online and quite a resource:

http: //groups.csail.mit.edu/graphics/classes/CompPhoto06/syllabus.shtml

They cover some relevant ground, particularly the image pyramid. I'm pretty sure the high pass filter is covered somewhere but haven't dug though the slides to find it.

I've already point Jacob at these slides, but Stephen's not makde me realize that someone else might be interested.

--John Ruttenberg

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Stephen Marsh

Thu Dec 11, 2008 4:22 pm (PST)

Thanks John, that reference will be handy for discussions on topics other than sharpening too.

Please keep such links coming (it beats paying for scientific journal downloads that are beyond my full understanding or ability to use).

Stephen Marsh

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Jacob Rus

Thu Dec 11, 2008 4:47 pm (PST)

I may have pasted this link in before, but I definitely recommend Dick Lyon's lectures to Google engineers about photographic technology, here: <http: //www.dicklyon.com/phototech/>

Cheers,

Jacob Rus

___________________________________________________________________________

Re: proper "sharpening" at all scales, ideas (and rants) about the u

Posted by: Dan Margulis

Fri Dec 12, 2008 7:54 am (PST)

Jacob writes,

I thought I'd send along some thoughts about "sharpening", because I

hope that others either know where to look for improved tools or how

to bend Photoshop into a more useful form, or will perhaps have ideas

of their own in response to my rambling. Hopefully the length and

assumed experience here aren't too off-putting: improving local/edge

contrast of images is a large subject, which could probably fill a

book, so it takes a bit of explanation to get my thoughts across.

it's indeed a long and thought-provoking post. I agree with a lot of it. More practical alternatives exist for other parts. I encourage continued investigation, though. because sharpening is a stalking horse for other types of operations that depend on blurring, such as the overlay and multiply moves that pervade the picture-postcard workflow. Many of the problems are common to all three.

Figuring out a generalized sharpening method faces several obstacles.

*Unlike other areas of color correction, there isn't a general agreement on what looks good. By-the-numbers color correction is easy to analyze because certain moves appeal to everyone. A sharpen that's appealing to one person, however, may offend another.

*Also, unlike other areas, it's very difficult for us to evaluate what we've done. All sharpening depends on inserting halos or other artifacts and hoping that the viewer doesn't sense the artificiality. That we know for a fact that the halos are there, but our target audience does not, makes us overly sensitive to them, so we are biased observers.

*Sharpening is prone to more exceptions IMHO than any other area in color correction. One size does not fit all; everything depends on what's in the image and how far away the viewer who sees it will be. Some of these specific-image related issues are far from obvious--and sometimes the alternative method of sharpening is not obvious either. It's easy to sharpen an image pleasingly but hard to envision how it could have been done better.

*For all these reasons, testing ideas is extremely difficult. I've often been fooled by a series of misleading results, where as many as four or five trials suggest that a certain way is right or wrong, but the fifth or sixth try suggests the opposite. Also, a large number of alternatives need to be tested.

*The point of diminishing returns is easily reached. There are many credible sharpening methods out there. Any variation in quality between them is dwarfed by the impact of other corrections. In my class competitions sharpening only is decisive when a person omits conventional sharpening altogether or else uses *really* offensive settings.

*Better ways of sharpening often take more time than they're worth, and better algorithms eat computing time rapidly.

Unfortunately, the Gaussian distribution is *never actually* what we

want to use in "sharpening" an image. It can easily create garish

halos, because we want strong effects at tight edges, but also want to

take into account the contribution of medium-scale shapes. (The

unsharp mask tool has other problems, discussed below, but I'll focus

on this one for now.) So what happens is, either we split the

difference, picking a radius & amount which negotiates that trade/off

between sufficient edge contrast and unattractive side-effects,

Any form of sharpening is an artificial attempt to add focus. There is no such thing as too much focus but there is such a thing as looking artificial. Every sharpening method depends on introducing some form of artifact that hornswoggles the viewer into perceiving more crispness. Every method can be overdone to make the image look artificial, however the problem that each method produces would be different. To say that because Gblur-based sharpening can produce nasty diffuse halos, another way has to be better, is to say that because Clinton couldn't keep his pants on in the Oval Office, Bush is the better president.

Dan oddly names our

increase of local contrast at different scales by two different

names, respectively "hiraloam sharpening" & "conventional

sharpening", which is unfortunate for three reasons: (a) neither name

gives much indication of what is being attempted;

Technical discussion of how things work is generally useful and appreciated. Quibbling about what terms should be used to describe what, particularly when the terms are well established and understood, is generally a waste of time and also tends to descend into holier-than-thou fairly quickly.

(b) the distinction

between the two is completely artificial, as they simply operate at

different scales,

The two operations have major differences and should not be considered as two sides of the same coin.

*One depends on its artifacts (halos) being visible, but mistaken by the viewer as being something else; the other depends on not having the halos visible.

*Noise avoidance is a major concern in one and not the other.

*One fails when the Radius is too large, making the halos clunky; the other fails when the Amount is too large, making them visible.

*Color shifts are undesirable in one and usually desirable in the other.

That these should be considered as two separate forms is confirmed by the inability to split the difference between them. Conventional sharpening is obviously useful, otherwise it would never have become conventional. Hiraloam, where the Radius is around 25 times as high as in conventional, has also clearly demonstrated its use. Plus, at various times I've shown effective sharpening with an *ultra*-high Radius--75 or 100 times wider than conventional. But medium-Radius sharpening (Radius of maybe 5 times higher than conventional) has no beneficial effect AFAIK.

that the additive combination of two Gaussian distributions is the

proper shape for enhancing local contrast,

It implies nothing of the sort.

Okay, so when we apply unsharp mask to an image, what we're trying to

do is bring out (i.e. make visually distinct and interesting)

features: shape details, textures, and so forth. These features have

different sizes, and contrast to different degrees. Humans generally

find images with more distinct features at every perceivable scale to

be the most interesting.

Not in faces, not in skies, not in flowers, not in butterflies. It's a slippery slope that the last sentence can start you down.

If we use a Gaussian function for weighting our sharpen, we can

increase the contrast in one such distance range, but instead of

making razor-sharp-looking edges, we actually make somewhat wide

edges, with definite halos with their own distinct shapes. This is

because the Gaussian function has horizontal slope at 0, and then a

"bell top" region of significant effect between 0 and 1 standard

deviations from 0, and then very quickly falls off towards zero from

there, and by 3 standard deviations has almost no effect whatsoever.

Okay, so what happens is our pixel moves away in value from other

pixels which lie at particular distances--mostly between 0 and 2

standard deviations. This has the effect of especially heightening

any details which are overall 1 or 2 standard deviations in size.

Anything bigger will have its edge contrast increased with a halo

extending about 1 or 2 standard deviations away from the edge, and

anything smaller will be effectively un-enhanced.

Right. That's why the choice of Radius is so important.

Because we want to highlight delicate details as well, and avoid

halos, we want to instead use a weighting function which immediately

starts decreasing at 0 (unlike the horizontal slope at 0 of the

Gaussian function), and which does not go to zero quite so fast as we

get away from 0. Perhaps some function which looks like `y = e^(-x)`.

To construct such a function out of the unsharp mask tool involves

its repeated application. We need to carefully eyeball the image

after each step, and try to make sure that we overdo our sharpen by

about the same relative amount each time, so that when we then cut

global opacity on our newly-sharpened copy of our image, we'll end up

with various distance ranges sharpened appropriately. This takes

practice to learn, and time and finesse to perform; it's tedious and

easy to screw up. Overall, we'd really prefer to just use a better

tool. Still though, it's usually worth running unsharp mask at least

four or five times, if not more, because the improvements to images

are quite dramatic, and the result is significantly better than only

one or two applications of the tool can accomplish.

What do we want instead?

Now you've reached the point of no return. You've identified the problem. There are two fundamental ways of proceeding. The elegant way is with a new algorithm that is designed to evade the problem (but is likely to introduce others in its stead.) The blunderbuss method is to apply damage control: if you find the halos too obtrusive, figure out a sneaky way to defeat them.

My suggestion is that the second way, while perhaps less intellectual appealing, is more practical. An example of inelegant, yet effective, planning: add to the current dialog a slider that says "Exclude Blues/Cyans". Why? Because serious problems with hiraloam sharpening, while they can occur anywhere, *most often* occur in skies, which don't have much noise to disguise a large halo. And in conventional sharpening, bringing out noise in skies is itself a serious problem. It's asymmetrical and illogical, but probably this single slider would add more quality to more people's work than a new sharpening algorithm.

The biggest need by far in any improvement to the sharpening routine is independent computation of the lightening and darkening artifacts, because usually the lightening is much more offensive than the darkening. I teach a workaround that is fairly quick (sharpen on a duplicate layer, then apply bottom layer, Darken mode, opacity 50%) but this only lessens the relative Amount of the lightening, whereas one would want to reduce the Radius as well. Doing this at present is ridiculously time-consuming.

In 2004, I wrote a column called "Life on the Edge" in which I showed how to sharpen an image using Gaussian blur only, plus some blend modes. My method took 23 steps and was intended as a joke. Nevertheless, a surprising number of people took it seriously and some actually started sharpening images that way. The suspicion has to be that, in addition to having a lot of time on their hands, these people saw the merits of computing light and dark halos independently, which the 23-step monstrosity did.

I think it is also now clear that we need a way to limit sharpening in saturated areas, although the blue/cyan problem is so severe that it warrants its own set of sliders.

But really, Gaussian sharpening as it exists in the unsharp mask tool is

designed for 80s hardware & unsophisticated users, and I am astounded

that it has not been improved in 20 years of Photoshop ("smart

sharpen" is some kind of tragic ironic joke).

True. 1980s scanners had independent control of light and dark sharpening, and to that extent were better than Photoshop. They did not, however, have the Threshold, which AFAIK was a Photoshop innovation, and a good one. When I started using Photoshop in the early 1990s, unsharp mask required SERIOUS computing time--on my office Macintoshes, which were state-of-the-art at the time, running USM on a decent-sized image could take 10-15 minutes. Under those circumstances, the Photoshop implementation has to be seen as a good one.

What was adequate in 1992, however, was obsolete by 1996, and it hasn't been improved since, For a tool of such fundamental importance, this is indeed stunning. Everybody comes up with multi- step workarounds to make up for the inadequacy of the filter itself. That Smart Sharpen was seriously offered as a substitute, plus the incorporation of somebody else's hiraloam routine as Clarity in ACR, suggests that whatever you come up with will have to work independently, as the Photoshop engineers will not be able to help you.

What would I prefer? Well, first and most simply, I don't understand

why Gaussian blur and unsharp mask need to be separate tools,

USM contains a Threshold that is inappropriate for Gblur.

or why "sharpen", "sharpen more", "sharpen edges", "blur", or "blur more"

even exist (they're utterly pointless).

Because some people are used to them, and as the Photoshop engineering team has no vendetta against those who employ them, it does not threaten to delete them.

Gaussian blur is exactly the

unsharp mask tool, where "amount" has been set to -100%, and threshold

has been set to 0. I also can't really imagine uses for "box blur",

"shape blur", "radial blur", but I can see that someone might imagine

such a use; all three are nonetheless extremely simplistic, and radial

blur in particular could be fairly easily beefed up for sophisticated

users without sacrificing comprehensibility for novices.

Give the matter some more thought.

As you noted, while a blur of some kind is probably necessary, there's no requirement that it be of the Gaussian persuasion. Any of the blurs above will also serve to sharpen the image but the artifacting structure would be different. Smart Sharpen has different blurs preloaded if one wants to experiment.

Suppose we sharpen an image hiraloam as usual with Gblur methodology, but also sharpen a second image, trying to mimic the radius effect, using a different blur. To make sure we don't lose any opportunity, these are good, stiff sharpens, more than we would like. If we then blend the two 50-50, place the result on a layer on top of the original, and reduce opacity to taste, the result will be better than by using either parent. It happens because while both methods add certain positive effects, their negative effects largely cancel one another. The unpleasant soft Gblur halos that you complain about are reduced.

Of course, if we make the split not between two different blurs but four, the effect gets better. Plus, we can add various kinds of interference, or operate at different geometry, prior to a Gblur to make the result less predictable.

Do I know which types of alternate blur work best, or what kind of ratio should be? No, although I've had good results from mixing Motion blurs at different angles. Do I have any plans to comprehensively investigate the matter? No. But I do know that the concept works. Obvious halos can be sharply reduced while retaining perceived sharpening. It will, however, take somebody a whole lot of testing to find the optimal default method because not only would one need to find a way to make Radii correspond and pick proper ratios, but some types of blurring should be done in the equlvalent of Darken or Lighten mode.

What I'd like to see in an ideal world is a tool which provides

something like a curve of some sort, with radius (probably in

logarithmic scale) along the horizontal axis, and amount (maybe also

logarithmically) along the vertical axis. I'd be able to adjust the

amount of effect at every scale by drawing a curve in a box, and in a

single interface with a real-time preview perform what now takes

infinite (or to quite roughly approximate, at least 5 or 6)

invocations of the current unsharp mask tool. This would have several

benefits: it would let us sharpen different scales using a single

tool, and would take a lot of the guess-work out of the process.

This mistakes sophistication for damage control. First, the above presumably applies only to hiraloam sharpening, as in conventional the halos are too small for a difference to be noticeable. In both conventional and hiraloam, there's an optimal range for each image--a cascade of different Radii is often counterproductive. For example, in good hiraloam sharpening of a face, we add shape and depth. If the Radius is too low, the subject gets bags under the eyes. And if it's too high, the whole face lightens or darkens as a unit. Doing multiple sharpens masks objectionable halos, for sure. But there's a price: it also masks some of the positive effects of choosing the right Radius. Better IMHO to do multiple types of blur at approximately the same Radius and merge them, as discussed above.

Here's where I'm hoping for some advice from the list. I still need

to experiment, but I've been having trouble constructing actions to

make manifest my desires for a sharpening tool. If I want to counter

effects at different lightness levels, in different difference ranges,

and so forth, I need to get really creative, with, for instance,

putting curves on specially-constructed high-pass filter layers (I am

not precisely sure what Photoshop's high-pass filter does--I should do

some more experiments, but if anyone knows the technical details I'd

be obliged),

Gernot Hoffmann has an excellent paper on Gaussian functionality,

including a good discussion of high and low pass, at

http://www.fho-emden.de/~hoffmann/gauss25092001.pdf