An Effective Method for Face Orientation Estimation Using Depth–Gyro Sensor

The proposed method for estimation of face direction overcomes the challenges encountered by traditional two-dimensional image-based methods

 

Conventional techniques for estimating face orientations rely upon the movements of eyes, nose, and mouth. This limits accurate detection if the person is wearing a mask, or is facing sideways. To address this problem, researchers recently collected point cloud data from various face orientations by a depth–gyro sensor and used them to train a deep learning-based classification model. The proposed method demonstrates high accuracy for estimating face orientation, even with a small dataset.

 

In recent years, artificial intelligence has demonstrated tremendous potential for the development and advancement of a wide variety of technologies. A good example is facial direction estimation, which finds applications in driver assistance systems that prevent distracted driving, methods to prevent cheating in examinations, and software for creating three-dimensional (3D) virtual avatars.

Traditional facial orientation estimation techniques recognize the characteristic parts of the face, including the nose, eyes, and mouth, and detect their movements. However, such two-dimensional (2D) image-based methods raise privacy concerns and fail when features of the face are hidden due to a mask, or if the face is turned sideways. The solution may lie in optimizing facial detection using point cloud data (data obtained from a discrete set of data points) and a depth sensor. In fact, some previous studies have employed an estimation model based on the deep learning of 3D point cloud data in five face directions: frontal, diagonal frontal, right, left, and horizontal. However, considering the level of accuracy required for driver assistance systems that crucially verify the driver’s status, this five-class (k = 5) classification is insufficient for satisfactorily detecting the face direction.

To address this limitation, scientists from Shibaura Institute of Technology, led by Professor Chinthaka Premachandra of the Graduate School of Engineering and Science, have developed a 老挝磨丁赌场_老挝赌场-中国竞彩网官网推荐 precise, horizontal wide-range angle detection approach (with k > 5). They accurately measured the horizontal angle of the face during the acquisition of the training data using gyroscopic sensors. Their paper was made available online in the IEEE Sensors Journal on 21 July, 2023.

In this study, the scientists gathered point cloud data from various orientations using a depth sensor, which was integrated with a gyro sensor during data collection. This data was employed to train a deep learning-based classification model, which was utilized for face orientation estimation. The scientists changed the horizontal angle of the face relative to the camera from +90 degrees to -90 degrees, using step sizes of 30, 22.5, 18, and 15 degrees between them. As a result, the classification of face direction was represented by 老挝磨丁赌场_老挝赌场-中国竞彩网官网推荐 than seven classes (k = 7, 9, 11, 13).

“Precise training data for each orientation was obtained from the integration of the depth and gyro sensors, which reduce the number of point cloud samples required for constructing the classification model. Further老挝磨丁赌场_老挝赌场-中国竞彩网官网推荐, applying a weight reduction process to reduce the weight of point cloud data enhanced training efficiency and resulted in fast face orientation estimation,” explains Prof. Premachandra.

The proposed classification method, designed for 老挝磨丁赌场_老挝赌场-中国竞彩网官网推荐 than seven classes, achieves remarkable performance in face direction detection through deep learning. For example,
Overall, this study opens new doors to a wide range of practical applications where accurate face orientation detection is required. As Prof. Premachandra explains,
Let us hope that this novel face direction detection technology enables accurate face orientation detection soon! 

 

 

 

 

 

 

Reference

Title of original paper:	Depth–Gyro Sensor-Based Extended Face Orientation Estimation Using Deep Learning
Journal	IEEE Sensors Journal
Article link:	10.1109/JSEN.2023.3296531

Funding Information

This study was partially supported by the Branding Research Fund of the Shibaura Institute of Technology.