A Remote Free-head Pupillometry Based on Deep Learning and Binocular System

摘要： Objective. Pupillometer plays a key role in a variety of research areas, including disease diagnosis, human-machine interaction, and education. Here we set out to leverage the deep learning theory to develop a remote binocular vision system for pupil diameter estimation. Approach. The system consists of three parts: eye detection, eye tracking, and pupil diameter estimation. We ?rst train a convolutional neural network based on YOLO V2 to perform eye detection, leading to high accuracy and robustness under ambient light interference. By exploring the similarity of binocular camera images, we then propose a master-slave structure for eye tracking, surpassing the traditional parallel structure in tracking speed while keeping considerable accuracy. Furthermore, we develop a pupil diameter estimation algorithm based on binocular vision, avoiding the personal calibration procedure and reducing the measurement distortion error. Main results. Experimental results on real datasets reveal that our system exhibits state-of-the-art performance with high eye detection accuracy (90.6%), fast eye tracking speed (< 11 ms per frame), low pupil diameter estimation error ((0.022 ± 0.017) mm mean absolute error and (0.6 ± 0.7)% percentage of the mean absolute error) and excellent ?exibility. Signi?cance. In contrast with previous pupillometers, which lead to pupil diameter measurement distortion error through a 2D projection image on a single camera, our system measures pupil diameter in 3D space without distortion in?uence, thus improving its robustness to head angle variation and making it more practical for real applications.