Based on NVIDIA’s 2D to 3D technology, scientists have developed an artificial intelligence that transfers the reflection in the eye to the virtual world in 3D.
The “mirror” function in the phrase “eyes are the mirror of the soul” is invariably reflected, at least in the real world. Our pupils absorb the light reflected on it and therefore the person’s objects in front of can reflect.
While scientists at the University of Maryland were thinking about “how to use this reflection”, they developed a tool suitable for the artificial intelligence trend. Objects seen in reflections in the eye, with the help of computer can be transferred to the three-dimensional medium.
Your eye can now create small worlds:
Scientists used NVIDIA’s previous development to convert reflections in the eye into three-dimensional objects. from NeRF technology took advantage. This technology was able to create an environment in three dimensions with just a few photos.
However, NeRF was insufficient to directly capture the reflections in the eye and transfer them to the 3D environment. Because in our pupils complex texture of the iris areacould make objects difficult to detect.
To prevent this, they developed a new method. In this method, “eye reflections”radian field” is trained by rays ejected from the camera and reflected from the eye-like structure.
In order for the complex texture of the iris not to be reflected in the 3D area, a 2D texture map, or simply a mask, is used, which simultaneously learns the iris texture.
Of course, the resulting image is still not clear. For this “eye pose optimizationA method called ‘ is used. Thanks to this optimization, the image improves as follows:
The developed method still does not give results in all kinds of eye photos. for now only in suitable ambient and lighting conditions correct results can be obtained.