Proposal For An Algorithm To Automatically Determine Strabismic Eyes Using Deep Learning!

3 main points
✔️ Strabismic eye - strabismus - is increasingly being acquired due to the widespread use of smartphones, and has been linked to serious eye diseases such as cataracts and cranial nerve damage. Early detection of strabismus is important to realize prognosis improvement, etc., especially because of the high incidence in childhood.
✔️ In this study, we construct and validate a deep learning DL model to screen strabismic eyes based on a photo and deep learning algorithm of eye gaze.
✔️ As a result of the evaluation, the area under the ROC curve- AUC- was about 0.99: 94.0% sensitivity, and 99.3% specificity were confirmed to have been achieved.

Detection of Referable Horizontal Strabismus in Children's Primary Gaze Photographs Using Deep Learning
written by Ce Zheng; Qian Yao; Jiewei Lu; Xiaolin Xie; Shibin Lin; Zilei Wang; Siyin Wang; Zhun Fan; Tong Qiao
(Submitted on January 2021)
Comments: Translational Vision Science & Technology

The images used in this article are from the paper, the introductory slides, or were created based on them.

background

Can deep learning fill in more experience than a medical specialist?

In this research, we aim to develop an automatic classification algorithm for strabismic eyes, which is rapidly increasing with the spread of smartphones.

Strabismus - Strabismus is a misalignment of the two eyes - the right eye and left eye looking in different directions - and is considered an ophthalmic condition that particularly affects children: in children, it includes both internal and external strabismus Horizontal strabismus is the most common form of strabismus and the most common cause of childhood amblyopia. Children with obvious strabismus are also more likely to have psychosocial sequelae - i.e., low self-confidence and self-esteem anxiety - due to visual problems; on the other hand, the diagnosis of strabismus requires a certain level of skill and expertise, and therefore, specialists who can diagnose these conditions - especially those who are specialized in the treatment of strabismus - are needed. On the other hand, the diagnosis of strabismus requires a certain level of skill and expertise, which limits the number of specialists, especially pediatric ophthalmologists, who can diagnose these disorders.

In this study, we construct and validate a deep learning model for screening horizontal strabismus from primary eye photographs taken during clinical evaluation, mainly for children. The model is based on a convolutional neural network (CNN), which is frequently used in image analysis and is validated using an external dataset.

What is a strabismic eye?

First, I will describe the strabismic eye - strabismus - which is the subject of the analysis in this study.

Strabismus is a condition in which the right and left eyes are looking in different directions - both eyes are not looking at the correct target: outwardly, one eye is looking in the correct direction while the other is looking inward, outward, up, or down. The right eye and the left eye are oriented in different directions - the two eyes are not looking at the correct target. This misalignment of the eyes makes it difficult to see correctly with both eyes: specifically, it is difficult to perceive three-dimensionality - binocular dysfunction - and visual development on one side is disturbed - strabismus amblyopia - a condition in which one eye's visual development is disturbed. In particular, the development of visual acuity on one side of the eye may be hampered by strabismus or amblyopia. These symptoms can also lead to problems such as weakened vision development in early childhood - amblyopia, double vision, cosmetic problems, and fatigue.

In recent years, acquired entropion - smartphone entropion - caused by the use of digital devices has become a problem due to the widespread use of smartphones. Normally, the human eye turns inward and focuses when looking at objects up close, while the inner yawning loosens when moving the gaze from near to far. This condition is caused by the excessive use of smartphones, which makes it difficult to control this inward shift. This worsening of strabismus due to digital device use is more common in young people - especially those younger than 12 years old - and the number of patients has been increasing since 2018.

technique

data-set

In this study, a dataset consisting of a total of 7530 primary gaze photographs - 3330 strabismic and 4200 normal gaze photographs - from 2013 to 2019 - the SCH dataset - was The data set was used for the study. Each image was examined by three specialist ophthalmologists and was removed from the dataset if any ophthalmologist judged the image to be non-gradient - visually undetectable for strabismus. As a result, 7026 images were obtained - 3829 positive images of 3021 patients with correct vision and 3197 images of 2772 patients with strabismus. The patients included were children with primary horizontal strabismus who underwent surgery and children with normal vision who underwent regular refractive examinations at SCH. Strabismus was also defined as steady infantile internal strabismus (≥ 40 PD - interpupillary distance -), residual accommodative hyperopia with complete hyperopia correction (> 10 PD), intermittent or external strabismus occurring > 50% of the awake time after complete or infantile exotropia (> 15 PD). We also excluded subjects with restrictive strabismus, sensory strabismus, paralytic strabismus, myasthenia gravis, nystagmus, Duane syndrome, and All photographs were taken with a pen torch attached to a commercially available camera (D800; Nikon Corporation, Tokyo, Japan) at a distance of 1 m from the subject, and only main gaze photographs were collected.

Assessment Environment

In the evaluation, the system was divided into training - 80%- and validation - 20%-: the whole system was divided into 5 groups, 4 groups were used for training and 1 group for validation. The training process was repeated 1000 times. The model performance was also used for accuracy, sensitivity, specificity, and area under the examinee operating curve-area under the ROC curve: AUC-. In addition, the evaluation was performed based on the primary eye photographs of 277 patients (133 strabismic and 144 normal) - the JSIEC dataset - as an external dataset.

proposed model

The proposed model - see the figure below - utilizes two stages of deep learning: first, the primary gaze is detected and identified, and then classified into oblique and normal gaze. In the first stage, we used Faster R-CNN to detect the region of interest (ROI), which is a type of object detection algorithm that defines a bounding box consisting of position and size information. which is a kind of object detection algorithm. The extracted images were manually checked by a medical specialist and corrected if necessary - adjusting both eyes to the horizontal position; in the second stage, the images were extracted using ImageNet26 - a 3-architecture pre-trained on more than 1 million images. In the second stage, we performed transition learning based on three architectures-VGG16, Inception-V3, and Xception-pre-trained on ImageNet26-over one million images. The image pixels are rescaled to values between 0 and 1 and interpolated to fill 299 × 299 matrices. For training, we used the Adam optimizer - with a learning rate of 0.0001 - and a mini-batch gradient descent method - size 32 - with 10 epochs of Early-stopping. To better visualize the learning procedure of the algorithm, we used a class activation map - class activation map: CAM - to clarify the image regions related to the discrimination level of the DL algorithm.

We also used an external validation dataset for human expert judgment and instructed three resident ophthalmologists with at least three years of clinical experience in pediatrics and strabismus to determine each test image independently - in identifying strabismic eyes, the first Purkinje image from the center of the pupil -reflection pattern due to corneal light reflection - utilizing the displacement of the

result

This section describes the evaluation of this study.

In the evaluation, we used Faster R-CNN to extract only the ROI on the image. For the evaluation, we trained with five-fold cross validation using the training and validation dataset (see the figure below) and verified the model performance using the external dataset (see the table below). See table below.

 

The results showed that the average AUCs of the 5-fold cross validation for the models using deep learning was as follows: InceptionV3: 0.993; VGG16: 0.993; Xception: 0.991. -JSIEC-, the classification performance was 0.94 for sensitivity and 0.99 for specificity - see table below and figure below. The deep learning model was found to be higher than the ophthalmologist's diagnostic accuracy in both sensitivity and specificity.

 

The misclassified images - see the figure below - showed that the classification failed because of Off center - a condition in which the child's eye cannot be centered due to the tilt of the head - and poor image quality - a weak reflection of light from the cornea. image - was confirmed to be the reason for the classification failure.

consideration

In this study, we constructed and validated a deep learning model that can automatically diagnose strabismus screening with high accuracy based on photographs of eye gaze. The number of patients with strabismus has been rapidly increasing due to the spread of smartphones and other devices, and the cost and expertise of clinical evaluation have been issues in strabismus. In this study, we constructed and validated an automatic screening model using deep learning based on primary photographs (i.e., photographs taken with smartphones and other devices). The model was validated based on an external dataset, and we confirmed that the model achieved higher accuracy than the diagnosis system by medical specialists - AUC: 0.997, Sensitivity: 94%, Specificity: 99.3%. Against this background, this study is expected to serve as a baseline for future DLs for strabismus detection.

In addition, nearly half of the misjudged cases - 49.1% - were due to off-center - the eye was not centered due to the tilt of the head - followed by poor image quality -Therefore, the cause of misjudgment in the present evaluation was not the performance of the proposed model, but rather the methodological issues in photography; the solution to these issues is the pre-processing of the images The solution to these problems is to use the pre-processing of the image -i.g. smoothing filtering- as input for the proposed model.

Second, the majority of the data utilized were for Chinese ethnicity, so it is necessary to consider whether they can be generalized to other ethnic groups; third, the patient was not wearing glasses at the time of imaging - wearing glasses could have altered the angle of strabismus - so additional validation of performance on images with glasses was needed. additional validation of performance in images with spectacles is needed.

• 
• 
• 

Categories related to this article

• Article
• Medical Care
• Medical