\n<\/p>\n
Paul AI\u2019s so-called black box refers Some It\u2019s Given Of These resources require clinicians to take the initiative and Deep Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform, and John Halamka, M.D., president, Mayo Clinic Platform, wrote this article. AI\u2019s so-called black box refers to the fact that much of the underlying technology behind machine learning-enhanced algorithms is probability\/statistics without a human readable explanation. Oftentimes that\u2019s the case because the advanced math […]<\/p>\n","protected":false},"author":1,"featured_media":7080151,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[393,1067,1224],"dealstore":[],"offerexpiration":[],"class_list":["post-7080150","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-black","tag-box","tag-open"],"yoast_head":"\n
\nCerrato, senior research analyst and communications specialist, Mayo Clinic
\nPlatform, and John Halamka, M.D., president, Mayo Clinic Platform, wrote this
\narticle.<\/span><\/i><\/p>\n
\nto the fact that much of the underlying technology behind machine learning-enhanced
\nalgorithms is <\/span>probability\/statistics
\nwithout a human readable explanation. <\/span>Oftentimes that\u2019s the case because the advanced math or the data
\nscience behind the algorithms is too complex for the average user to understand
\nwithout additional training. Several stakeholders in digital health maintain, however,
\nthat this lack of understanding isn\u2019t that important. They argue that as long
\nas an algorithm generates actionable insights, most clinicians don\u2019t really
\ncare about what\u2019s \u201cunder the hood.\u201d Is that reasoning sound?<\/span><\/p>\n
\nthought leaders point to the fact that there are many advanced,
\ncomputer-enhanced diagnostic and therapeutic tools currently in use that
\nphysicians don\u2019t fully understand, but nonetheless accept. \u00a0<\/span>The CHA2DSA-VASc score, for instance, is used
\nto estimate the likelihood of a patient with non-valvular atrial fibrillation
\nhaving a stroke. Few clinicians are familiar with the original research or detailed
\nreasoning upon which the calculator is based, but they nonetheless use the
\ntool. \u00a0<\/span>Similarly, many physicians use the
\nFRAX score to estimate a patient\u2019s 10-year risk of developing a bone fracture,
\ndespite the fact that they have not investigated the underlying math.\u00a0 <\/span><\/span><\/p>\n
\nimportant to point out, however, that the stroke risk tool and the FRAX tool
\nboth have major endorsements from organizations that physicians respect. The <\/span>American Heart Association<\/a> and the American College of
\nCardiology both recommend the CHA2DSA-VASc score while the National
\nOsteoporosis Foundation supports the use of FRAX score. That gives physicians
\nconfidence in these tools even if they don\u2019t grasp the underlying details. To
\ndate, there are no major professional associations recommending specific AI-enabled
\nalgorithms to supplement the diagnosis or treatment of disease. The <\/span>American
\nDiabetes Association<\/a> did
\ninclude a passing mention of an AI-based screening tool in its 2020 Standards
\nof Medical Care in Diabetes, stating: \u201c<\/span>Artificial intelligence systems that
\ndetect more than mild diabetic retinopathy and diabetic macular edema
\nauthorized for use by the FDA represent an alternative to traditional screening
\napproaches. However, the benefits and optimal utilization of this type of
\nscreening have yet to be fully determined.\u201d That can hardly be considered a
\nrecommendation.<\/p>\n
\nthis scenario, most physicians have reason to be skeptical, and surveys bear
\nout that skepticism.\u00a0\u00a0 A <\/span>survey of 91 primary care physicians<\/a> found that understandability
\n<\/i>of AI is one of the important attributes they want before trusting its
\nrecommendations during breast cancer screening. Similarly, a survey of senior
\nspecialists in UK found that understandability was one of their primary
\nconcerns about AI.\u00a0 <\/span>Among
\nNew Zealand physicians<\/a>,
\n88% <\/span>were
\nmore likely to trust an AI algorithm that produced an understandable
\nexplanation of its decisions.<\/p>\n
\ncourse, it may not be possible to fully explain the advanced mathematics used
\nto create machine learning based algorithms. But there are other ways to describe
\nthe logic behind these tools that would satisfy clinicians.\u00a0 As we have mentioned in previous publications
\nand oral presentations, there are tutorials available to simplify machine
\nlearning-related systems like neural networks, random forest modeling,
\nclustering, and gradient boosting. Our <\/span>most recent book<\/a> contains an entire chapter on this
\ndigital toolbox. Similarly, JAMA <\/i>\u00a0has created <\/span>clinician friendly video tutorials<\/a> designed to graphically
\nillustrate how deep learning is used in medical image analysis and how such
\nalgorithms can be used to help detect lymph node metastases in breast cancer
\npatients.\u00a0<\/span><\/p>\n
\nlearn a few basic AI concepts, but developers and vendors also have an
\nobligation to make their products more transparent.\u00a0 <\/span>One way to accomplish that goal is through <\/span>saliency maps and generative adversarial
\nnetworks<\/span><\/a>. Using
\nsuch techniques, it\u2019s possible to highlight the specific pixel grouping that a neural
\nnetwork has identified as a trouble spot, which the clinician can then view on
\na radiograph, for example.\u00a0 <\/span>Alex DeGrave,
\nwith the University of Washington, and his colleagues, used this approach to
\nhelp explain why an algorithm designed to detect COVID-19-related changes in
\nchest X-rays made its recommendations. <\/span>Amirata Ghrobani and associates<\/span><\/a> from Stanford University have
\ntaken a similar approach to help clinicians comprehend the echocardiography
\nrecommendations coming from a deep learning system. <\/span>The
\nresearchers trained a convolutional neural network (CNN) on over 2.6 million echocardiogram
\nimages from more than 2,800 patients and demonstrated it was capable of
\nidentifying enlarged left atria, left ventricular hypertrophy, and several
\nother abnormalities. To open up the black box, Ghorbani et al presented readers
\nwith \u201cbiologically plausible regions of interest\u201d in the echocardiograms they
\nanalyzed so they could see for themselves the reason for the interpretation
\nthat the model has arrived at. For instance, if the CNN said it had identified
\na structure such as a pacemaker lead, it highlighted the pixels it identifies
\nas the lead. Similar clinician-friendly images are presented for a severely
\ndilated left atrium and for left ventricular hypertrophy. <\/span><\/p>\n
\nlearning systems are slowly ushering in a new way to manage diagnosis and treatment,
\nbut to bring skeptical clinicians on board, we need to pull the curtain back. In
\naddition to providing evidence that these tools are equitable and clinically
\neffectively, practitioners want reasonable explanations to demonstrate that they
\nwill do what they claim to do.<\/span><\/p>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"