Deep learning is one of many machine learning techniques that have brought the field of computer vision on in leaps and bounds. The technology has made it possible to combine different types of input data to significantly enhance an image, a technique known as sensor fusion. We can use it to do things like add elements from the near-infrared to RGB images or blend outputs from close range radar with those from thermal sensors.
Here are some examples of what we’ve previously achieved for our customers:
Sensor fusion for RGB + near IR
What did Digica want to achieve
Images taken in a low-light environment have problems with:
- high noise level
- low contrast
- poor visibility
While it’s possible to improve image quality, the drawbacks are huge. Upping the exposure time introduces motion blur, which you can only get around by stabilising the camera but this only works for stationary objects. We could use flash to illuminate the subject but this isn’t always appropriate for every situation.
We wanted to find a way around these limitations, so we conducted a research project to explore methods of brightening up low-light mobile images without extending the exposure time or resorting to traditional flash.
How Digica met this challenge
Modern smartphones have very advanced cameras with sophisticated sensors that are sensitive to near-infrared. It’s a waveband that humans can’t see, but it holds a lot of valuable information that can help us see better in dark conditions. Pictures become much brighter after we merge colour information from the visible spectrum with shapes from the near-infrared. We did use a flash in the end, but not a normal one. It lights up the subject in the near-infrared only, so while your eyes don’t notice it, the final image certainly does.
What we achieved
Compared to traditional methods PSNR (peak signal-to-noise ratio) grew from 8.12 to 25.51 and the SSIM (structural similarity index) grew from 0.044 to 0.833.
Sony IMX219 sensor, Near-Infrared LED flash 850nm, TensorFlow