2
$\begingroup$

I am trying to refocus images from a microlens array light field using Ren Ng's Fourier Slice photograph theorem found in his thesis chapter 5, equation 5.7, which is available at https://stanford.edu/class/ee367/reading/Ren%20Ng-thesis%20Lytro.pdf

Fourier Slice Photograph theorem

$$ P_a=F^{-2}\circ B_a \circ F^4$$

Where P is the refocused photograph, F is either inverse or forward Fourier transform, and B is the Fourier photographic imaging operator, which he gives as $$B_a [G](k_x,k_y)= \frac{1}{F} G(\alpha k_x, \alpha k_y, (1-\alpha)k_x, (1-\alpha)k_y) $$

which is given in equation 5.6. In words, the Fourier Slice photograph theorem means a photograph is the inverse 2D Fourier transform of a dilated 2D slice in the 4D Fourier transform of the light field.

My confusion is partly notation and applying it in python to an acquired light field image. To start, I understand what the Fourier transform is but when Ng says 4D Fourier transform of the light field, I am not sure how to interpret that. Currently, I have my light field as a NumPy array such that my u=i, v=j values index to the different sub-aperture images where each pixel in the sub-aperture image has an x and y coordinate, hence giving me a 4D light field array.

        lf_img = cv2.imread(lf_img, -cv2.IMREAD_ANYDEPTH)
        row_lens = lf_img.shape[0]/14 #14 is pixels under a microlens in this direction
        col_lens = lf_img.shape[1]/14 # 14 is pixels under a microlens in this direction
        vp_imgs = np.zeros([14,14,int(row_lens),int(col_lens),3])
        for i in range(14):
            for j in range(14):
                vp_imgs[i, j, :, :, :] = lf_img[i::14, j::14, 0:3] ##U,V,X,Y,(RGB)

Then I see images like this http://graphics.stanford.edu/papers/fourierphoto/ in Ng's thesis where to me it looks like he just took the 2D Fourier transform of each lenslet and arranged them in a grid. So I am not entirely sure how to interpret this 4D Fourier transform and how the slicing is done. Thanks!

Sample of what sort of light field image I am working with: light_field_data

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.