Thanks to AI, color night vision becomes (almost) a reality

Thanks to AI, color night vision becomes (almost) a reality

Sam Fisher’s iconic night vision goggles could well become obsolete within a few years.

Among all the military technologies that have been widely used in fiction, we must necessarily mention night vision. This essential of the perfect spy kit has found its way into various and varied works. But as fans of Splinter Cell know only too well, these have long suffered from a major flaw: the user must be satisfied with a monochrome image, traditionally green.

But this limit is jumping thanks to deep learning, a subcategory of artificial intelligence. In a research paper spotted by Interesting Engineering, researchers at the University of California-Irvine presented an algorithm capable of reconstructing a full-color image of a nighttime scene based on infrared radiation.

Even if we tend to forget it since we use them constantly, our eyes are true technological marvels. On any scale, it is one of the most complex structures in our body; it is made up of many substructures that all play a very specific role in capturing visual information.

A ghost story

Each light wave is defined by a wavelength, which directly determines the color captured by our eye. At the end of its journey in the eye, the light strikes a layer of very special cells which lines the bottom of the cavity: the retina. This is where a very important phenomenon takes place, phototransduction.

Very briefly, it is this phototransduction which makes it possible to convert the wavelength into an electrical signal. This signal can then exit through the optic nerve and then travel to the brain; it is then translated again to determine the color of light that has struck the retina.

The problem is that our eye may be extremely sophisticated, but it is not infallible. He can not capture only a certain range of wavelengthswhich therefore corresponds to the spectrum of the so-called “visible” light. Physically, this concerns all electromagnetic radiation whose wavelength is between 400 nanometers (violet) and about 700 nanometers (red).

© UC Davis ChemWiki, CC-BY-NC-SA 3.0

Color the invisible

As soon as one ventures outside this domain, these radiations (we then speak of infrared and ultraviolet) become invisible to the human eye. But that doesn’t mean they’re going away! Many scientific instruments and even the eyes of certain animals, especially insects, are even specialized in capturing this radiation.

The whole point is therefore to convert these invisible rays into visible images and understandable by a human. This is something that engineers already know how to do very well. For example, it is this concept that allows thermal imaging cameras to work. We begin by measuring the intensity of infrared radiation, which is closely linked to temperature. A false color is then arbitrarily assigned to each value, traditionally red for hot surfaces and blue for colder ones.

This approach is also applicable to night vision. Technically, we already have color night vision devices. But since these are false colors that do not correspond to reality, it can be difficult to interpret them in real time.

An algorithm as a visual translator

The approach of American researchers is different. Here, their goal was indeed to achieve “real” night vision in real colors. They started from the same measurements of infrared radiation; but instead of processing them with a rudimentary coloring algorithm, they fed them to a system based on deep learning, a subcategory of artificial intelligence.

By compiling visible and infrared imagery, they succeeded in producing an algorithm capable of constructing a true color image from simple IR radiation. And the result is quite stunning, as can be seen in the images above.

The system is still far from perfect. At present, the resolution remains very limited and the colorimetry still leaves something to be desired. But it is in any case a very impressive proof of concept which could have concrete applications quite quickly.

The first of them will certainly appear in the military field; we can therefore expect that these systems will not be accessible to the general public for at least several years. But in the long term, it is typically the kind of technology that will one day be able to equip the smartphone of the average user.

The text of the study is available here.

Leave a Reply

Your email address will not be published.