When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

Mahmoud Afifi¹, Brian Price², Scott Cohen², and Michael S. Brown<sup>1

1</sup>York University    ²Adobe Research

Reference code for the paper When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images. Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown, CVPR 2019. If you use this code or our dataset, please cite our paper:

@inproceedings{afifi2019color,
  title={When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images},
  author={Afifi, Mahmoud and Price, Brian and Cohen, Scott and Brown, Michael S},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={1535--1544},
  year={2019}
}

The original source code of our paper was written in Matlab. We also provide a Python version of our code. We tried to make both versions identical. However, there is no guarantee that the Python version will give exactly the same results. The differences should be due to rounding errors when we converted our model to Python or differences between Matlab and OpenCV in reading compressed images.

Quick start

1. Matlab:

1. Run install_.m
2. Run demo.m to process a single image or demo_images.m to process all images in a directory.
3. Check evaluation_examples.m for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

2. Python:

1. Requirements: numpy, opencv-python, and skimage (skimage is required for evaluation code only).
2. Run demo.py to process a single image or demo_images.py to process all images in a directory.
3. Check evaluation_examples.py for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

Graphical user interface

We provide a Matlab GUI to help tuning our parameters in an interactive way. Please, check demo_GPU.m.

Code/GUI parameters and options

1. K: Number of nearest neighbors in the KNN search (Sec. 3.4 in the paper) -- change its value to enhance the results.
2. sigma: The fall-off factor for KNN blending (Eq. 8 in the paper) -- change its value to enhance the results.
3. device: GPU or CPU (provided for Matlab version only).
4. gamut_mapping: Mapping pixels in-gamut either using scaling (gamut_mapping= 1) or clipping (gamut_mapping= 2). In the paper, we used the clipping options to report our results, but the scaling option gives compelling results in some cases (esp., with high-saturated/vivid images).
5. upgraded_model and upgraded: To load our upgraded model, use upgraded_model=1 in Matlab or upgraded=1 in Python. The upgraded model has new training examples. In our paper results, we did not use this model.

Dataset

In the paper, we mentioned that our dataset contains over 65,000 images. We further added two additional sets of rendered images, for a total of 105,638 rendered images. You can download our dataset from here. You can also download the dataset from the following links:

Input images: Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Input images [a single ZIP file]: Download (PNG lossless compression) | Download (JPEG) | Google Drive Mirror (JPEG)

Input images (without color chart pixels): Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Input images (without color chart pixels) [a single ZIP file]: Download (PNG lossless compression) | Download (JPEG) | Google Drive Mirror (JPEG)

Augmented images (without color chart pixels): Download (rendered with additional/rare color temperatures)

Ground-truth images: Download

Ground-truth images (without color chart pixels): Download

Metadata files: Input images | Ground-truth images

Folds: Download

Online demo

Try the interactive demo by uploading your photo or paste a URL for a photo from the web.

Working with videos

You can use the provided code to process video frames separately (some flickering may occur as it does not consider temporal coherence in processing).

Project page

For more information, please visit our project page

Commercial Use

This software and the dataset are provided for research purposes only. A license must be obtained for any commercial application.

Related Research Projects

• White-Balance Augmenter: An augmentation technique based on camera WB errors (ICCV 2019).
• Deep White-Balance Editing: A multi-task deep learning model for post-capture white-balance correction and editing (CVPR 2020).
• Interactive White Balancing:A simple method to link the nonlinear white-balance correction to the user's selected colors to allow interactive white-balance manipulation (CIC 2020).