Biomarker Discovery blog

Transfer Learning Between Multiple Therapeutic Areas

01 Jul 2020

Many pharmaceutical companies develop their AI capabilities to improve operational efficiencies, scientific research, and the clinical trial process. As for medical image analysis, the complexity and heavy investment of time and effort lead them to turn to specialized partners who possess this expertise. Keen Eye holds a prominent position in this field with significant knowledge inAI, deep learning, and evaluation of large sets of histopathological, diagnostic and molecular images.

Our experience in multiple therapeutic areas comes from successfully transferring knowledge – both AI and human – to other areas. Analyzing images for one disease could help train the AI on the diagnosis of another. 

A Case Study in Ophthalmology

Sjögren’s syndrome (SS) is a chronic autoimmune condition affecting moisture-producing glands, in some cases leading to diseases, including cancer. In ophthalmology, SS causes dry eyes. Early diagnosis improves treatment efficacy but is difficult to obtain.  In vivo confocal microscopy (IVCM) is an emerging imaging and diagnostic tool. For SS, IVCM takes around 200 images of the eye. The images are observed by the clinician who then performs the diagnosis based on the information gleaned from the IVCM images.  Current research is evaluating IVCM to determine if it can be a viable diagnostic alternative. While positive, preliminary results reveal an overly tedious and subjective process, an increased efficiency is needed and Keen Eye’s participation in the project seeks to determine whether AI can help. In this case, AI has the potential to deliver more systematic results than human analysis.



Automate a Larger-Scale Analysis

Data scientists at Keen Eye leveraged its knowledge of AI and deep learning to further automate the analysis of IVCM images. With a larger-scale solution, Keen Eye reduced the reliance on manual processes while considering all patients’ medical images instead of only a selection. To develop the model, 80 patients – 17 healthy and 63 with SS – and approximately 200 images per patient were considered. The classic approach consists of training the neural networks by associating a label for each image. Since some images are inconclusive or don’t show SS for an affected patient, the training model yielded a random and unusable prediction.  An alternative approach was Multiple Instance Learning (MIL), which considers all images belonging to a patient attributing weight to those that make relevant contributions to the final diagnosis. The AI model was trained by receiving all 200 images for each patient, but only one label: the patient’s final diagnosis. While this can work, we faced another hurdle: the small sample size.



Leverage Transfer Learning

Small datasets pose a problem that can be fixed by using transfer learning, as demonstrated by literature. Keen Eye previously experimented with this technique using public data to train its AI on classifying different lung cancer cells. The model developed was then used to pre-train a new AI model for classifying breast cancer cells. This technique proved successful and could be reproduced to increase the efficiency in the IVCM analysis.  Our next hurdle was to find similar images for pre-training. The novelty of IVCM as an image modality, however, precludes this option. Using the existing data appeared to be our only option. After careful examination, we noticed certain patterns among patients with SS: tortuous and more reflective nerves in the cornea, morphological anomalies, and an increase in inflammatory cells. This raised the idea that if we could focus the attention of the AI algorithm on segmenting these features, perhaps the weights generated from the trained AI model could be used to pre-train the final classification model. Therefore, we created a hybrid transfer learning method that transfers weights from a segmentation model to a classification model for pre-training. 


We selected the 10 best images from each patient to train the model to segment objects on the ICVM images and used the weights of this secondary model to initialize those of the MIL classification model. The accuracy rate jumped to 81.1%, a tremendous improvement from the initial unusable model. Additional research and work on a larger dataset will increase the validity of the model and might allow us to extend it to diagnose other features of SS or related diseases such as meibomian gland dysfunction. 




Watch the video below to learn more about the study, Deep Learning based diagnosis of Sjögren Syndrome using In Vivo Confocal Microscopy.


Deep Learning based diagnosis of Sjögren Syndrome using In Vivo Confocal Microscopy


Although the popular transfer learning approach is typically a viable solution for a low dataset for training an AI model, Keen Eye faced another challenge with the lack of comparable images for pre training. We overcame the obstacle by creatively designing a hybrid transfer learning model that uses the project’s own dataset for pre-training. We successfully transferred knowledge from studies on histopathology slides to training AI on ophthalmology images.  The novel solution we devised for pre-training the IVCM images can potentially be applied to solve similar problems in other therapeutic areas.