Woods Hole Oceanographic Institution Improves Underwater Imaging
"MathWorks tools enabled us to push the limits on the next piece of the technology that we were developing."
Challenge
Solution
Results
- Accurate underwater photomosaics
- High-performance, compact MATLAB scripts and functions
- Functionality that’s easy to share
Photomosaic of an 80–60 B.C. Roman trading ship.
Underwater optical imaging enables scientists to study everything from biological ecosystems to ancient shipwrecks, but the underwater environment limits the scale and quality of the images obtained. Lack of ambient light makes it impossible to image wide areas or large objects, and images are often distorted when the camera perspective interacts with irregularities on the sea floor. Photomosaicking, which involves collecting overlapping images and then stretching and warping them until they fit together, helps address these problems, but existing photomosaicking methods were developed for land-based projects and are difficult to adapt to underwater conditions.
Researchers in the Deep Submergence Laboratory at Woods Hole Oceanographic Institution (WHOI) developed photomosaicking software specifically for underwater imaging. WHOI used several MathWorks products throughout the project.
"Our work would have been impossible without MATLAB and Image Processing Toolbox," says Hanumant Singh, associate scientist in the Applied Ocean Physics and Engineering Department at WHOI. "Trying to do the same thing in C would have taken a lot more time and effort, and would have been much more complex."
Challenge
Underwater photomosaicking involves surveying extended areas of the ocean floor, collecting overlapping images, identifying their common features, and then merging the images to form larger mosaics.
“Taking pictures underwater is like driving in fog,” says Singh. “There are far more variables involved. For example, since electromagnetic radiation doesn’t travel very far under water, you must use your own light sources. And since there are no straight lines or walls underwater, trying to identify matching features in several images is difficult. In addition, the object that you are photographing has shadows that change size as you move away from it.”
Singh and his team wanted to develop photomosaicking methods to address the unique challenges of underwater imaging.
Solution
Traveling a few meters above the ocean floor with a submersible vehicle, the researchers took several photographs of an underwater site. MathWorks tools enabled them to solve many of the imaging problems that they encountered. Lighting underwater was irregular, which made it critical to capture digital images with a dynamic range exceeding 8 bits per pixel. “We could do this because MATLAB can manipulate 16-bit images,” says Singh.
The next step was to preprocess the images to remove lighting artifacts, such as shadows, by means of histogram equalization. They wrote MATLAB routines that work well for underwater imagery. The spatial image transformation (image warping) capabilities of Image Processing Toolbox enabled them to preprocess the images using “in-situ” camera calibrations to correct for radial lens distortion. As a final preprocessing step, Singh’s team constructed multiresolution pyramids from each corrected image, using Image Processing Toolbox for Laplacian filtering.
They then extracted features to match between overlapping images. As Singh explains, “High-quality feature extraction requires convolution with large image kernels. We accomplished this very efficiently using the two-dimensional fast Fourier transforms in MATLAB.” Then, using Optimization Toolbox, they fitted successively more advanced sets of parameters to achieve pair-wise image registrations.
Moving toward global registration, they represented the image-to-image topology of the mosaic with a network matrix having a row and column for every image collected. These were sparse matrices containing zeroes, except where a row-column pair corresponded to a pair of overlapping images. “The ability to represent and manipulate sparse matrices in MATLAB made this approach practical and efficient,” says Singh.
Optimization Toolbox enabled the researchers to perform topology refinement via a nonlinear Levenberg-Marquardt algorithm. “MathWorks tools have amazing flexibility,” says Singh. “For example, when we were stretching and warping our images, we could pose this as an optimization problem, pick up a function from Optimization Toolbox, and quickly run it through.”
The WHOI researchers have applied their MATLAB routines to solve a host of underwater imaging problems and to image shipwrecks, plane crashes, and underwater archeological sites.
Results
- Accurate underwater photomosaics. The WHOI researchers can now view an entire underwater site in a single mosaic image. Features are correctly positioned, unobscured, and well-illuminated. In a recent survey of the 350 square meter site of a Roman shipwreck, they generated images accurate enough to enable them to count individual amphoras on the vessel.
- High-performance, compact MATLAB scripts and functions. “There’s no need for a C implementation—MATLAB is fast, works well, and keeps the code size small,” says Singh. “It’s a couple of orders of magnitude smaller code than with C or C++.” When the team started using Image Processing Toolbox, they further reduced the volume of code that they needed to write, test, and maintain.
- Functionality that’s easy to share. “We can send out a small, nicely commented M-file, and other groups can use it for their own underwater exploration,” says Singh. For example, the University of Singapore is using the software to detect and classify mines along the ocean floor.
