Hi Manali,
I understand that you want to filter an image in the frequency domain using the defined filter 
where n and theta represent number of orientations and the orientation of the polar indices in the fourier domain. You can follow the following steps to achieve the same:
where n and theta represent number of orientations and the orientation of the polar indices in the fourier domain. You can follow the following steps to achieve the same:1) Convert the image from the spatial domain to the frequency domain and shift the zero-frequency component to center of spectrum using the "fft2" and "fftshift" functions respectively.
2) Calculate the orientation of each frequency component
3) Create a filter for each orientation ranging from 0 to 7 and apply the filter to the image by doing the element-wise multiplication using ".*" operation.
4) Shift the zero-frequency component back to the corner and perform the inverse fourier transform to obtain the image in the spatial domain using the "ifftshift", "ifft2" functions respectively.
You can refer to the following MathWorks documentations to understand more about the "fft2", "fftshift", "ifftshift" and "ifft2" functions respectively.
- https://www.mathworks.com/help/matlab/ref/fft2.html
- https://www.mathworks.com/help/matlab/ref/fftshift.html
- https://www.mathworks.com/help/matlab/ref/ifftshift.html
- https://www.mathworks.com/help/matlab/ref/ifft2.html
I hope this helps.
