I am trying to refocus images from a microlens array light field using Ren Ng's Fourier Slice photograph theorem found in his thesis chapter 5, equation 5.7, which is available at https://stanford.edu/class/ee367/reading/Ren%20Ng-thesis%20Lytro.pdf

Fourier Slice Photograph theorem

$$ P_a=F^{-2}\circ B_a \circ F^4$$

Where P is the refocused photograph, F is either inverse or forward Fourier transform, and B is the Fourier photographic imaging operator, which he gives as $$B_a [G](k_x,k_y)= \frac{1}{F} G(\alpha k_x, \alpha k_y, (1-\alpha)k_x, (1-\alpha)k_y) $$

which is given in equation 5.6. In words, the Fourier Slice photograph theorem means a photograph is the inverse 2D Fourier transform of a dilated 2D slice in the 4D Fourier transform of the light field.

My confusion is partly notation and applying it in python to an acquired light field image. To start, I understand what the Fourier transform is but when Ng says 4D Fourier transform of the light field, I am not sure how to interpret that. Currently, I have my light field as a NumPy array such that my u=i, v=j values index to the different sub-aperture images where each pixel in the sub-aperture image has an x and y coordinate, hence giving me a 4D light field array.

        lf_img = cv2.imread(lf_img, -cv2.IMREAD_ANYDEPTH)
        row_lens = lf_img.shape[0]/14 #14 is pixels under a microlens in this direction
        col_lens = lf_img.shape[1]/14 # 14 is pixels under a microlens in this direction
        vp_imgs = np.zeros([14,14,int(row_lens),int(col_lens),3])
        for i in range(14):
            for j in range(14):
                vp_imgs[i, j, :, :, :] = lf_img[i::14, j::14, 0:3] ##U,V,X,Y,(RGB)

Then I see images like this http://graphics.stanford.edu/papers/fourierphoto/ in Ng's thesis where to me it looks like he just took the 2D Fourier transform of each lenslet and arranged them in a grid. So I am not entirely sure how to interpret this 4D Fourier transform and how the slicing is done. Thanks!

Sample of what sort of light field image I am working with: light_field_data


The trick is avoiding the computation of $4D$ Fourier transform (high computational complexity) by using Fourier slice theorem. Its implementation is pretty tricky AFAIK. He never computes $4D$ transform. $4D$ transform is not much different from $1D$ to $2D$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.