Ethan Tam

Overview

Prokudin-Gorskii was a Russian photographer and scientist in the early 1900s who worked towards color photography. By creating red, blue, and green filters of the same image, he envisioned a time when these images could be stacked together to create a singular colored image.

Single-scale Search

I first implemented a single-scale search that iterated over a window of possible displacements, anywhere between [-20, 20]. This brute force algorithm used the Sum of Squared Differences (SSD) to calculate the best (lowest) score for a given displacement of a target image on a reference image. I specifically aligned the red and blue channels on the green channel. This was effective in aligning smaller .jpg images. I noticed that better alignment occurred when processing channels without their borders, so I cropped off the 10% of the top, bottom, left, and right of the images prior to the single-scale search.

cathedral.jpg | blue: (-2, -5) red: (1, 7)

monastery.jpg | blue: (-2, 3) red: (1, 6)

tobolsk.jpg | blue: (-3, -3) red: (0, 4)

Multiscale Pyramid Search

The single-scale search is too slow on larger .tif files. To optimize runtime, I employed the image pyramid technique, in which both the target and reference images are recursively rescaled and processed to find the best displacement vector. I followed the steps below for my implementation:

1. Check if the image height is less than or equal to 300. If so, run the brute force search with a [-20,20] window.
2. Otherwise, make copies of and rescale the target and reference images to half their resolution.
3. Recursively call the image pyramid function with the rescaled images to find the displacement vector of those rescaled images.
4. Adjust the target image (not the rescaled one!) with the found displacement vector multiplied by 2 to account for the rescaling.
5. Run the brute force search on the shifted target image and the reference image with a [-20,20] window to find the displacement vector of those images.
6. Return a displacement vector containing the output values of Step 5 and the values of Step 3 multiplied by 2.

church.tif with no alignment

church.tif aligned | blue: (-4, -25) red: (-8, 33)

Bells & Whistles

Edge Feature