| Abstract|| |
Machine learning and artificial intelligence (AI) have become a part of our daily routine. There are very few of us who are not influenced by this technology. There are a lot of misconceptions about the scope, utility, and fallacies of AI. Digital neuropathology is an evolving area of research. The importance of digital image processing stems from the rapid gains in computer vision and image processing that have happened in the past decade thanks to advancements in deep learning (DL). The article attempts to present to the audience a simple presentation of the technology and attempts to provide a context-based understanding of the DL process for image processing. Also highlighted are current challenges and the roadblocks in adopting the technology in routine neuropathology.
Keywords: Artificial intelligence, convolutional neural networks, deep learning, digital neuropathology, image processing
|How to cite this article:|
Vazhayil V. Artificial intelligence in neuropathology: Current status and future perspectives. Indian J Pathol Microbiol 2022;65, Suppl S1:226-9
| Introduction|| |
Artificial intelligence (AI) is a relatively newer branch of technology. It refers to a class of computer programs that mimic human intelligence. The aim of developing technology is to harness the power of human intelligence on a computer platform. AI has wide acceptance and has found applications in our daily routine. For example, when a search is performed on the internet, a photo is taken on a mobile device, an order is placed for an item of clothing, an advertisement is viewed on a mobile, AI is at work. AI programs and packages are distinguishable in solving real-world problems in real-time.
| Programming, Machine Learning, and Neural Networks|| |
A program is a series of codes that conveys instructions to the computer to do something. Simple programs perform sequential instructions that can solve numerical problems or logical problems. Programs are written in computer languages. A well-designed computer program can handle significant problems that are difficult to handle by groups of humans. The problem solving involves coding a dedicated technique. Each such method or computerized process is called an algorithm. Often the class of problems written on a computer would not account for the complete set of the issues encountered in the real world. Thus, conventional programming was inadequate to solve many real-world problems. One option was to develop multiple algorithms to deal with every possible scenario. In such cases, the programs would thus become large, unmanageable, and challenging to implement. A method developed to overcome these issues was to use statistical techniques. Programs could be written to use advanced statistical methods to solve these “real-world” problems. These were bracketed under the general heading of machine learning (ML). The term reflected these techniques' statistical origins and differed from conventional programs by the “learning” involved. They could learn on a trial data set and be shown to solve problems hitherto not exposed to the program, which was a unique property of these programs. The idea of learning thus could be implemented. This method, though better, did not result in ideal solutions.
The idea of harnessing neural computing to solve problems is not new. The earliest method was known as a perceptron. The perceptron was a method of coding to emulate a single neuron's function. At the heart of the development was one important theme. The perceptron's output was not a yes or no (one or zero). Instead, an activation function ensured that a number between zero and one could be obtained as a result. This was different from the yes-no options available with other types of programming.
A breakthrough in neural network research occurred with the idea of learning in the network. However, unlike biological networks, learning in artificial neural networks works on a different principle. The following example explains some of the core concepts involved in the process [Figure 1]. Consider a group of students being trained to recognize mitosis in a slide. Let the students be of varying backgrounds, such as students learning pathology, students learning undergraduate medicine, and some novice students from a non-medical background. The students learning pathology will learn the fastest, whereas it would be challenging to train the novice students. Consider a slide passed on to the different batches of students sequentially across the groups, from the novice to the pathology students, to recognize regions containing mitosis. Even without any training, some mitosis will be correctly identified. Suppose there is an expert teacher at the end of the reading who checks the correctness of the task and sends the slides back with the corrections back in the reverse order, with the pathology students receiving the corrections first. In turn, the pathology students correct the medical students, who finally explain to the novice students. If the slide is repeatedly passed in the fashion described, the students' ability to recognize mitosis in the given slide will become the same as the teacher's.
|Figure 1: Diagramatic representation of the example of neural network computing|
Click here to view
This method of passing back the changes in a systematic fashion is called backpropagation. Each group of students named in the example would constitute a neural network layer. Each student participates in the learning process. The comments, which are the differences from the ideal, form the learning mechanism. Each layer interacts with only the layer in front and behind it. The expertise at any given point for a given student or node can be referred to as an “activation function.” Each cycle of passing the slide to and fro constitutes a learning cycle. A neural network is subjected to numerous learning cycles. The set of slides where the answers are known, that is, where the teacher is present, is called a “trial set.” When the teacher is not in the class, the slides passed would be called a “test set.” Generally, any training set of slides or data is split into 2/3 and 1/3. The more extensive set (2/3) would form the training set, and the smaller set would form the test set. The test set quantifies the accuracy of the network, and the “learning” of the network is assessed.
There are usually several layers of neurons in an artificial neural network (ANN). Each neuron can be thought of as an equation. When values are entered into the equation, the result is transferred to a neuron in the next layer. For backpropagation, differential calculus methods calculate values transmitted back as “comments.” These changes are small changes that change how the results are obtained in the next “pass.” These back and forth changes have all to be stored for serial computation. The ability of a network to compute problems depends on several factors. Some of the crucial factors include the number of layers, total number of neurons, the type of activation function used, the rate at which the network is trained, the numbers passed back in backpropagation, etc. Problems such as image processing require large networks to get good results. These more extensive networks generate many numbers that need to be stored and calculated continuously. Deep learning (DL) refers to large neural networks with several layers containing several thousand “neurons.” Internally, the computations are done using matrices. The simultaneous storage of large numbers and matrix and tensor-based calculations has necessitated dedicated graphical processing units (GPUs) for neural network computation. An important technique in DL is the use of convolutional neural networks (CNNs).
CNNs evolved from primary neural networks. CNNs attempt to duplicate the hierarchical organization within the human brain with specific reference to human vision. Thus, CNNs have channels that are conceptually similar to rods and cones in the retina. They have alternating layers, as seen in the various parts of the human visual pathway, namely the lateral geniculate body and the visual cortex. Each piece of a CNN thus would train on specific aspects of the input. CNNs became popular because of their ability to classify and identify objects in photographs. Extensive databases of labeled photos could be solved, that is, tagged accurately with CNN. Based on their success, several other varieties of large neural networks have been proposed with crucial changes in architecture. CNN dramatically improved the efficiency of the conventional neural network. However, as the size of the network increased, the costs in terms of time taken for computation and processor power, especially GPU power, also increased.
| Scope in Neuropathology|| |
AI has several applications in medical practice. Several of the problems in patient management are difficult to solve with conventional programming techniques. Applications in pathology include image recognition for histopathological evaluation,[1-9] associating pathological diagnosis to patient prognosis, and education. The various approaches are indicative of the existing utilities of AI in neuropathology. Several other applications can be potentially utilized. Deng et al. listed the methodologies used by various authors who have used DL techniques for histopathological evaluation The highlight of the paper is the list of technical methods used to recognize various features of a histopathological slide.
| Digital Neuropathology|| |
The term “digital neuropathology” here refers to ML/DL techniques used for histopathological diagnosis. A short brief of the process is as below. Computer vision has been one of the major success stories of DL. Object recognition in photographs has been a long-term objective in computing. ML techniques had some breakthroughs in object recognition in photos. However, significant issues persisted. A classic example quoted is that recognition of objects dropped just because of changes in the background light. DL solved many of these issues and has become the standard technique for object recognition. Thus, DL techniques can recognize cars in photographs irrespective of the car's orientation, color, or make.
The histopathological slide as an image on which these techniques can be used is a natural extension of the logic. However, though image processing with DL has been continually evolving for over a decade, similar progress has not happened in digital histopathology. An overview of the issues is as follows. A neuropathologist views a slide under the microscope. Generally, there is some prior history of the patient, which indicates a class of diseases that can be expected to be seen. The history and findings are essential in narrowing down possibilities. On the contrary, a straightforward image processing approach “sees” the slide as an independent entity.
Whole slide imaging dramatically changed digital histopathology. Image processing before whole slide imaging required that sample images be acquired under different magnifications from a single slide. The ability to generate gigapixel-sized photographs of the entire slide at high resolution with uniform lighting and color scales solved several issues related to image preprocessing. Preprocessing refers to standardization techniques that need to be adopted before an image processing software analyzes the image. A whole slide image is of the size of 100,000 × 100,000 pixels. Standard image inputs into neural networks are of the size of 256 × 256 pixels. Thus, sampling needs to be done so that the DL tool can process the image.
Furthermore, features in the histopathological examination are made out at different magnifications. For example, proper evaluation of mitosis requires high magnification, whereas features such as endothelial proliferation and necrosis are visible at low magnification. Thus, it is challenging to integrate these findings into a comprehensive diagnosis by using conventional image processing techniques.
Two broad approaches can be used to diagnose based on slide analysis. The first is by a method called multiple object segmentation. Using this, numerous target objects in a given slide can be identified. For example, features such as rosettes and endothelial proliferation, which are multicellular structures, can be identified as individual features. The diagnosis is then derived from these findings, which are subsequently compiled. The second approach can be termed as a “black box” approach. Many slides of a particular diagnosis are trained on a neural network. The network would then recognize and output a diagnosis when a similar slide is encountered. Each technique has its benefits and disadvantages. In the first approach, there is a logical approach to the diagnosis. However, individual networks would need to be trained for each feature, such as rosette identification or mitosis. For feature-rich diseases or diseases in which there is a similarity in features, this would result in problems in differentiation. The second approach has the advantage that training the network is a more straightforward task: the larger the training dataset, the more robust the model and performance on an actual sample. However, there is no control over individual parameters, and the diagnosis will have to be taken on faith. Due to this issue, a fear of uncertainty is always present with the black-box approach.
| Discussion|| |
The central crux in digital histopathological diagnosis is the ability of relatively simple networks to perform complex image classifications. However, despite widespread success in several other domains, medical image processing has not reached a level of maturity to reach the pathologist's desktop. The challenges listed in the paper are multiple, some of which are non-technological—for example, regulatory concerns and ownership of responsibility. When a software makes a mistake, is it the pathologist who becomes responsible, or is the software designer/developer at fault? These kinds of questions need a regulatory and legal framework to be built. There are no ideal solutions; however, a consensus framework must provide a starting point from which incremental advances can be made. There are several other challenges. A robust neuropathological software package would need training sets constituting several lakhs of slides. The logistics of building such a database, ethical concerns, and financial requirements are enormous. A consortium of academic institutions and industry is essential for concrete steps to be made in the field.
| Conclusion|| |
Digital neuropathology using DL is the way ahead. Several questions are yet to be answered. However, the questions are when and how, not the if. As a technological tool, challenges exist. Resistance is expected because of several “issues” that need to be addressed. However, as an advisor, mentor, and helper, AI platforms will change how neuropathology will be practiced in the coming years. The benefits far outweigh the risks. The scope of discovery, customization of diagnosis is yet unexplored areas where digital neuropathology aided by AI techniques will play a significant role.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Griffin J, Treanor D. Digital pathology in clinical use: Where are we now and what is holding us back? Histopathology 2017;70:134-45.
Deng S, Zhang X, Yan W, Chang EIC, Fan Y, Lai M, et al
. Deep learning in digital pathology image analysis: A survey. Front Med 2020;14:470-87.
van Es SL. Digital pathology: Semper ad meliora. Pathology 2019;51:1-10. doi: 10.1016/j.pathol.2018.10.011.
Tizhoosh HR, Pantanowitz L. Artificial intelligence and digital pathology: Challenges and opportunities. J Pathol Inform 2018;9. doi: 10.4103/jpi.jpi_53_18.
Abels E, Pantanowitz L, Aeffner F, Zarella MD, van der Laak J, Bui MM, et al
. Computational pathology definitions, best practices, and recommendations for regulatory guidance: A white paper from the digital pathology association. J Pathol 2019;249:286-94.
Pantanowitz L, Sharma A, Carter AB, Kurc T, Sussman A, Saltz J. Twenty years of digital pathology: An overview of the road travelled, what is on the horizon, and the emergence of vendor-neutral archives. J Pathol Inform 2018;9. doi: 10.4103/jpi.jpi_69_18.
Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol. 2019;20:e253-e261. doi: 10.1016/S1470-2045(19)30154-8.
van Es SL. Digital pathology: Semper ad meliora. Pathology 2019;51:1-10. doi: 10.1016/j.pathol.2018.10.011.
Thombre BD, Nandeesh BN, Vazhayil V, Prabhuraj AR. Deep learning use for differentiation of low-grade vs high-grade glioma in intraoperative squash smears. Research Square 2021. doi: 10.21203/rs.3.rs-329319/v1.
Department of Neurosurgery, NIMHANS, Bengaluru - 560 029, Karnataka
Source of Support: None, Conflict of Interest: None