The heat equation in relation to smoothing algorithm

Recently I learned about a technique in image processing, which has its roots in something called the 'heat equation' from physics. The original creators of this technique were inspired by the physics of how heat diffuses through an object.

The objective of course is to 'smooth out' the image, for general noise removal. This was done by taking a Gaussian kernel, and convolving it with the image. However, the professor says that the heat-equation is actually a generalization of this process, and in fact, we can get much better techniques using the heat-equation framework.

Essentially, if the original image we have is $I$, then the heat equation framework says that:

$$I(t) = \nabla \cdot (\ D(x,y) \ \nabla I)$$

where the $\nabla$ means spatial derivative, (I think). The $I(t)$ indicates the image new image at some point in time as it evolves - as heat flows - as this algorithm is run. Finally, the $D(x,y)$ is the "diffusion co-efficient", and if $D=1$, (or any constant), then the above simply collapses to a Gaussian kernel convolving the image $I$.

Now, what I am hoping for is the following: I am hoping someone here can add some intuitive insight into how/why this is working, vis-a-vis a physical analogy to 'heat flow' in the image.

The way I currently understand it, is that we have an image. The bigger the amplitude of certain pixels, the 'hotter' those pixels are. In fact every image pixel is as hot as its amplitude. Now, we also know from physics and entropy, that the heat will try to dissipate, so that eventually, the "temperature" across the image becomes equal. (This I take it, is what is happening when a gaussian convolves the image - this is the 'smearing' we are seeking for removal of noise...).

Now, with the diffusion co-efficient being a constant or 1, this 'heat flow' occurs everywhere. However, if the diffusion co-efficient is, say, a binary function of the spatial image, the the co-ordinates of where $D(x,y)$ equal 0, are where no heat can flow through, and so those are pixels that are spared from the heat flow...

Is my understanding of this physically inspired algorithmic technique correct? Are we really doing nothing but 'simulating' heat flow, except in the frame work of an image? Thank you so much.

• @Dan, I am not sure that changing the $I(t)$ to $\frac{\delta I}{\delta t}$ is accurate. This is because - at least as far as I understand this - the left hand side of the equation I put up, is meant to say "This is the result of your image at time $t$, in its evolution", not "This is the derivative in time of your image". This is how I understand it... – The Grape Beyond Jul 3 '13 at 4:25
• I may have misunderstood it. You had a subscript, like this: $I_t$ and that's usually the notation for a partial derivative. Making it a partial derivative also makes that equation the heat equation. If you think I overstepped my bounds, feel free to revert my edit. – Dan Jul 3 '13 at 4:30
• Though $I_t$ can be used either as a derivative or as a time-indexed quantity, I suspect @Dan had it right. The quantity $\nabla \cdot (D \nabla f)$ generally only gives the rate of change of $f$ (see Wikipedia for confirmation). As added proof, you would otherwise not have time dependence on the right-hand side. – user10851 Jul 3 '13 at 5:29
• @ChrisWhite I see, it would appear my understanding of it is not complete then. I thought it was "Image at time t", whereas it seems that the left hand side is saying "Change of image in time"... – The Grape Beyond Jul 3 '13 at 15:07

Yes, you are literally simulating the flow of heat through a material. The diffusion coefficient is basically just a local measure of thermal conductivity and heat capacity. Rather than convolving your image with a Gaussian kernel, you could use something like the Crank-Nicholson method and make some number of timesteps. You could also adjust $D$ to make it greater in parts of the image with greater noise.

$\nabla$ is the gradient operator. It's a vector operator that looks like this in two dimensional cartesian coordinates: $$\nabla= \hat{\mathbf{x}} \frac{\partial}{\partial x} + \hat{\mathbf{y}} \frac{\partial}{\partial y}$$ where $\hat{\mathbf{x}}$ and $\hat{\mathbf{y}}$ are the unit vectors pointing in the $x$ and $y$ directions.

In short, your intuitive understanding is correct.

• "The diffusion coefficient is basically just a local measure of thermal conductivity." Beautiful. That is simply amazing. Dan, thanks for your answer, can you also expand as to why the equation is what it is? I am just looking for some intuition behind it, (for example, why the double gradients, etc), do nothing too fancy. Thanks so much! – The Grape Beyond Jul 3 '13 at 4:40
• (PS I tried to upvote you but apparently I need more than 15 rep to do so). – The Grape Beyond Jul 3 '13 at 4:41
• @TheGrapeBeyond: You can "accept" (that's the checkmark), but it might be a good idea to hold off on that for a while. People are more likely to answer a question that doesn't have an accepted answer yet. – Dan Jul 3 '13 at 4:51
• @TheGrapeBeyond: "Why does the heat equation have the form it does" is probably best put as a separate question. – Dan Jul 3 '13 at 4:55
• @TheGrapeBeyond: I'll make it another answer anyway, though. – Dan Jul 3 '13 at 5:06

The heat equation comes from two very intuitive ideas: the rate of heat flow is proportional to the temperature difference, and the conservation of energy.

First, from Newton's law of cooling or Fourier's law we get that the flow of heat is proportional to the gradient of the temperature:

$$\mathbf{j}_{\text{heat}}=-k \nabla T$$ where $k$ is the thermal conductivity, and the minus sign represents that heat flows from the hot areas to the cold areas.

For energy to be conserved, the amount of heat leaving a given point has to be equal to the rate of change of heat contained at that point. This is represented by the continuity equation:

$$\nabla \cdot \mathbf{j}_{\text{heat}} = -\frac{\partial q}{\partial t}$$

This equation basically says "the rate of heat decrease at a point is equal to the rate at which it flows out". The operator $\nabla \cdot$ is called the "divergence" and represents the tendency of a vector quantity to flow away from where it's being calculated.

Putting these two together:

$$\frac{\partial q}{\partial t}=-\nabla \cdot \mathbf{j}_{\text{heat}}=-\nabla \cdot (-k) \nabla T$$

We then multiply both sides by the heat capacity $c_p$ and cancel out the minus signs:

$$\frac{\partial T}{\partial t}=c_p\nabla \cdot (k \nabla T)$$

and finally combine $c_p$ and $k$ into the diffusion coefficient $D$.

$$\frac{\partial T}{\partial t}=\nabla \cdot (D \nabla T)$$

• Thanks again Mr Dan. To confirm some things: The $\nabla T$ is basically the spatial slope of the image. (Temperature slope, or Temperature gradient). Heat likes to flow down this slope. I am not clear though about $\nabla J$, or even just $J$. $J$ is the "flow of heat" - is this a rate?... I am very confused by the $J$... – The Grape Beyond Jul 3 '13 at 16:10
• $j_{\text{heat}}$ is heat current density. I used $j$ by analogy with electromagnetism, where $j$ is current density (current per unit volume). – Dan Jul 3 '13 at 18:03
• @TheGrapeBeyond: $j_{\text{heat}}$ is the flow of energy. It has units $\frac{\text{Watts}}{\text{cm}^2}$. – Dan Jul 4 '13 at 2:32