Image processing enables advanced eye tracking technology

Eye tracking technology is influencing applications from autonomous vehicles to medical monitoring. How does it work and how can it be used?

How it works
Most common methods of eye tracking involve shining infrared (IR) light into the eye and recording the reflections from the cornea and the pupil with a camera; an image processor then analyzes the reflections to determine where the eye is looking – known as the AOI (area of interest). IR light is used because it provides enough contrast to differentiate between the separate parts of the eye. Fixation refers to a steady gaze focusing on a single location, and saccades are the rapid movement of the eyes between different locations. Understanding these processes, along with the actions of blinking and pupil dilation, have allowed researchers to develop extremely accurate methods of tracking what we are looking at, for how long, and even predicting what we’ll focus on next. One eye can be tracked, or both, depending on whether monocular or binocular technology is used.

In the past, eye tracking required a method to keep the head still, such as a chin rest, or apparatus fixed in a static position relative to the eye, such as a headset. Today, however, new technologies have enabled accurate gaze tracking without such interventions, and remote eye tracking systems can be fitted to devices such as computer monitors or car dashboards. SensoMotoric Instruments (SMI) is one such provider; their RED250 mobile eye tracker can be attached below a monitor to record the viewer’s scan path across the screen. Nevertheless, there is still a large market for eye tracking glasses and headsets, and these are becoming smaller, lighter and faster all the time. Tobii, a Swedish based company, operates in three main areas using a combination of these products. They specialize in eye tracking for communication (for example, supporting disabled patients with limited speech or mobility), solutions for understanding human behavior (for use in advertising and market research) and advanced AR and VR technology (aimed at the high-volume consumer gaming market).

A high-resolution IR camera and image processor can measure the eye position to better than 0.1° accuracy and the latest technology can operate at sampling rates of up to 1000 Hz, although 30-60 Hz may be sufficient for many applications. Reductions in size and cost are delivering more portable and flexible eye tracking systems as well as the integration of eye tracking technology in standard devices such as display monitors. Low-cost eye tracking systems may not use IR light and are often based on low cost, low resolution cameras. For example, the Tobii Eye Tracker 4C is designed with gaming in mind; it has a sample rate of 90 Hz and uses Near Infrared (NIR) illuminators.

Reducing the processing burden in VR systems
The quality of any VR experience is directly linked to the resolution of the display, but with higher resolutions comes the requirement for more processing power. However, the user is only able to see high levels of image detail within the AOI, allowing the implementation of a process known as foveated rendering.  By combining eye trackers and VR headsets, the system is able to limit high detail rendering to the image region inside the AOI. Peripheral regions, outside the AOI, are rendered to a lower level of detail, vastly reducing the amount of pixel processing required. This saves GPU processing performance and power, making image processing faster and cheaper without losing the ability for users to view in 4K if required.

Use in the growing autonomous vehicle market
Eye tracking has been used for some time to monitor driver fatigue and prompt an alert when a driver appears drowsy. The next natural step in autonomous technology may also be assisted by eye tracking. Passing control of the vehicle between driver and computer relies on the human element being prepared and situationally aware when the car decides to hand over control. Nuance are developing a gaze-detection solution to trigger multimodal alerts including vibration, visual cues and auditory stimulus to ensure a smooth and safe handover.

Focus on communication
Gaze keyboards and control panels have been available for some time to assist with communication for patients with cerebral palsy and spinal injury, providing a vital channel for people with limited speech or movement. Earlier this month, Microsoft announced the release of Windows 10 Insider Preview Build 18932 (20H1) to some users. This includes cited improvements in the Eye Control feature, including a drag-and-drop with your eyes function. Along with the recent announcement of their “Eyes First” games – four apps which can be played using solely eye movement – Microsoft are putting accessibility at the forefront of their development. Solutions like these are opening up the world of computing to those with mobility disabilities.

Medical diagnostics
According to medical studies, certain neurological conditions such as Autism, ADHD and Parkinson’s disease can be identified using eye tracking technology. It is also being researched as a tool to improve the delivery of care, especially in predicting the needs of patients when monitoring their vital signs.

Eye tracking is an expanding and exciting application for image processing which is set to have a huge impact on our industrial and domestic lives. Active Silicon hardware is compatible with some of the world’s leading computer vision technologies and our experts are on hand to discuss your ideas and requirements. Contact us to see how we can make your vision a reality.