31. Artificial Intelligence in CT

Detection and classification models answer a deceptively simple question — is a particular finding present, and if localized, where — yet the machinery required to answer it on a CT volume is anything but simple, and understanding that machinery is the prerequisite for judging when its output can be trusted. The dominant architecture remains the convolutional neural network, whose defining operation is the discrete convolution of a small learned kernel $w$ across the image. For a two-dimensional slice, a feature map value is $y_{ij} = \sigma\!\left(b + \sum_{m}\sum_{n} w_{mn}\, x_{i+m,\,j+n}\right)$, where $\sigma$ is a nonlinearity such as the rectified linear unit $\mathrm{ReLU}(z)=\max(0,z)$. Three properties of this operation are doing the clinical work. It is translation-equivariant, so a hyperdense hemorrhage is recognized wherever it appears; it shares weights, so the number of learned parameters is small relative to the number of pixels and the model generalizes rather than memorizing pixel coordinates; and stacked convolutions with intervening pooling expand the receptive field, the region of the input that influences one deep activation, so that early layers encode edges and Hounsfield gradients while deep layers encode whole structures such as a lacuna of low attenuation or a rounded nodule. CT adds the third spatial dimension, and clinically serious models operate on the volume — either as true 3D convolutions or as 2.5D stacks of adjacent slices — because a pulmonary nodule and a vessel-on-end are separable only through-plane.

The network terminates in a probability, not a verdict. For binary detection the final logit $z$ is squashed by the logistic function $p = 1/(1+e^{-z})$; for mutually exclusive categories the softmax $p_k = e^{z_k}/\sum_j e^{z_j}$ produces a normalized distribution over classes. Training minimizes the cross-entropy between this predicted distribution and the labels, $\mathcal{L} = -\sum_k y_k \log p_k$, by gradient descent with backpropagation. The continuous, calibratable nature of $p$ is the conceptual heart of the matter: a detection model does not find a bleed, it estimates the posterior probability of a bleed given the pixels, and the binary alert seen by the clinician is the result of thresholding that probability at an operating point chosen to trade sensitivity against specificity. Moving the threshold trades the two error types continuously and is the lever by which a triage tool is tuned for near-total sensitivity at the cost of false alarms.

CT detection is also a hard class-imbalance problem: positive voxels — an early infarct, a subsegmental embolus, a few millimetres of subdural blood — are vanishingly rare against the background of normal anatomy, so a naive model can score high accuracy by predicting normal and detecting nothing. This is why detection systems are trained with imbalance-aware objectives (focal loss, hard-negative mining) and why accuracy is a forbidden metric in this domain. The clinical validity of these systems is genuine but bounded: Titano and colleagues showed that a 3D CNN could triage head CT for critical findings and shorten time to read in a simulated workflow, and Chilamkurthy and colleagues demonstrated detection of multiple critical head-CT findings at radiologist-comparable accuracy on large retrospective cohorts. The recurring lesson, developed throughout this chapter, is that these are probabilistic estimators whose reported accuracy is conditional on the spectrum of disease and the acquisition characteristics of the data on which they were built.

Segmentation models answer a different and more demanding question than detection: rather than emitting one label per image, they emit a label for every voxel, partitioning the volume into anatomically or pathologically meaningful regions — delineating an infarct, contouring the liver and spleen for volumetry, or outlining a tumor for response assessment. Architecturally the field is dominated by the encoder–decoder with skip connections introduced by the U-Net and, in three dimensions, by the self-configuring nnU-Net of Isensee and colleagues, which automatically adapts patch size, resolution, and normalization to the dataset and remains a benchmark-topping default. The encoder progressively contracts the volume into a compact semantic representation through convolution and downsampling; the decoder symmetrically upsamples it back to full resolution; and the skip connections splice high-resolution early-layer features into the decoder so that the network recovers both what a structure is and where its boundary lies to the voxel. TotalSegmentator (Wasserthal and colleagues) exemplified the maturation of this paradigm, producing robust automated segmentation of more than one hundred anatomic structures across heterogeneous clinical CT, which is precisely the substrate that turns the quantitative imaging of the preceding chapter — organ volumetry, body-composition analysis, opportunistic screening — into a one-click reality.

The defining feature of segmentation, and the source of its rigor, is that correctness is measured by spatial overlap rather than by a single right-or-wrong label. The dominant metric is the Dice similarity coefficient, $\mathrm{Dice} = \dfrac{2|A\cap B|}{|A|+|B|}$, where $A$ is the predicted region and $B$ the reference; it ranges from $0$ (no overlap) to $1$ (perfect). The closely related Jaccard index, or intersection-over-union, is $\mathrm{IoU} = \dfrac{|A\cap B|}{|A\cup B|}$, and the two are deterministically related by $\mathrm{Dice} = \dfrac{2\,\mathrm{IoU}}{1+\mathrm{IoU}}$, so Dice always meets or exceeds IoU and the choice between them is largely conventional. The deeper point is why overlap metrics, and a Dice-based loss $\mathcal{L}_{\mathrm{Dice}} = 1 - \dfrac{2\sum_i p_i g_i}{\sum_i p_i + \sum_i g_i}$, are preferred over voxelwise cross-entropy for small targets: a 2 mL lesion in a 4000 mL field is so outnumbered that a cross-entropy-trained network is rewarded for ignoring it, whereas the Dice objective is scale-invariant to the background and forces the network to capture the small foreground. This is the segmentation analogue of the detection class-imbalance problem and explains the field's metric conventions.

Two cautions complete the picture. First, Dice is dominated by region interior and is comparatively insensitive to boundary error; for tasks where the margin is clinically decisive — a tumor abutting a vessel, the surgical edge — boundary-aware metrics such as the Hausdorff distance, the maximum of the closest-point distances between surfaces, are reported alongside Dice. Second, a segmentation is only as good as the human contours used to train and test it, and inter-observer variability in manual segmentation sets a hard ceiling: when expert radiologists disagree on a boundary, an algorithm that matches one of them perfectly will appear to err against the other, so the achievable Dice is bounded by the reproducibility of the reference standard itself.

Foundation models represent a paradigm shift from the task-specific networks of the preceding sections to large, generally pre-trained representations that are subsequently adapted to many downstream tasks, and understanding them requires understanding two ideas: the attention mechanism and self-supervised pre-training. The architectural substrate is the transformer, whose central operation, self-attention, computes for each element a weighted combination of all other elements, $\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\left(\dfrac{QK^{\top}}{\sqrt{d_k}}\right)V$, where queries $Q$, keys $K$, and values $V$ are learned projections of the input and $\sqrt{d_k}$ stabilizes the scale. Applied to imaging by the vision transformer, which tokenizes an image into patches, attention lets every patch directly attend to every other patch, capturing long-range spatial dependencies — the relationship between a mediastinal mass and a distant pleural effusion — that a convolution, with its local receptive field, reaches only after many layers. This global context is the mechanistic appeal of transformers for whole-volume CT interpretation.

The second idea is what makes these models foundational. Rather than learning from scarce, expensively annotated labels, they are pre-trained by self-supervision on enormous unlabeled corpora using pretext objectives — masking image patches or text tokens and predicting the missing content, or contrastive learning that pulls matched image–report pairs together in an embedding space and pushes mismatched pairs apart. The result is a representation that has internalized the statistical structure of medical images and language, which can then be adapted to a specific task with comparatively little labeled data through fine-tuning or even zero- and few-shot prompting. Moor and colleagues articulated the vision of generalist medical AI: a single multimodal model ingesting images, reports, and structured data and performing tasks it was not explicitly trained for, queried in natural language. In CT this promises report generation, visual question answering, cross-modal retrieval, and unified segmentation–detection–description from one backbone.

The rigor demanded of the physician reader is skepticism proportionate to the hype. A foundation model inherits, and can amplify, every bias in its pre-training corpus; if that corpus underrepresents a scanner vendor, a body habitus, or a demographic group, the model's confident fluency will mask its incompetence on those inputs. The fluency itself is a hazard: large generative models confabulate, producing plausible, well-formatted findings that are simply false, and unlike a probability score from a detection CNN, a generated radiology sentence carries no native calibrated uncertainty. Evaluation methodology for these systems is immature — the field has not agreed on how to measure factual correctness of a generated report or on how to externally validate a model with effectively unbounded scope — and the regulatory frameworks designed for narrow, fixed-function devices fit poorly to a model whose behavior changes with the prompt. The promise is real and the trajectory steep, but in 2026 these systems are decision-support research tools, not autonomous interpreters, and the burden of verification that the following sections formalize applies to them with redoubled force.

Validation is where machine learning meets clinical epistemology, and the central, non-negotiable lesson is that a model's reported accuracy is a statement about a distribution, not about a patient, and is meaningful only insofar as the validation distribution matches the deployment population. The atomic quantities are sensitivity, $\mathrm{Se}=\mathrm{TP}/(\mathrm{TP}+\mathrm{FN})$, and specificity, $\mathrm{Sp}=\mathrm{TN}/(\mathrm{TN}+\mathrm{FP})$, which are properties of the test conditional on true disease status and are, to first approximation, prevalence-independent. What the clinician actually needs, however, are the predictive values, and these are not prevalence-independent. Bayes' theorem makes the dependence explicit: $\mathrm{PPV} = \dfrac{\mathrm{Se}\cdot p}{\mathrm{Se}\cdot p + (1-\mathrm{Sp})(1-p)}$, where $p$ is the pre-test prevalence. A vivid worked case: a hemorrhage-detection model with $\mathrm{Se}=0.95$ and $\mathrm{Sp}=0.95$ applied in an emergency cohort with $20\%$ prevalence yields $\mathrm{PPV} = (0.95)(0.20)/[(0.95)(0.20)+(0.05)(0.80)] \approx 0.83$; deploy the identical model in an outpatient screening stream with $1\%$ prevalence and the PPV collapses to $(0.95)(0.01)/[(0.95)(0.01)+(0.05)(0.99)] \approx 0.16$ — more than four of every five positive alerts are false, from a model whose sensitivity and specificity never changed. The odds form is the most transportable expression of this logic: $\mathrm{post\text{-}test\ odds} = \mathrm{pre\text{-}test\ odds} \times \mathrm{LR}$, with the positive likelihood ratio $\mathrm{LR}^{+} = \mathrm{Se}/(1-\mathrm{Sp})$ and the negative $\mathrm{LR}^{-} = (1-\mathrm{Se})/\mathrm{Sp}$; for the example above $\mathrm{LR}^{+}=19$ and $\mathrm{LR}^{-}\approx 0.053$, fixed properties that the reader then multiplies against whatever local prevalence obtains.

Because a model emits a continuous probability, its discrimination across all thresholds is summarized by the receiver operating characteristic curve, a plot of sensitivity against the false-positive rate $(1-\mathrm{Sp})$ as the threshold sweeps from $1$ to $0$. The area under this curve (AUC) has a precise probabilistic meaning — it equals the probability that the model assigns a higher score to a randomly chosen diseased case than to a randomly chosen non-diseased one — and it is threshold-independent, which is both its strength and its trap. Equivalently, under a Gaussian signal-detection model the separation is captured by the detectability index $d' = (\mu_1 - \mu_0)/\sigma$, the standardized distance between the diseased and non-diseased score distributions. A high AUC certifies that some threshold separates the classes well; it says nothing about whether the chosen operating threshold is safe, nothing about calibration (whether a predicted probability of $0.7$ corresponds to a true frequency of $0.7$), and nothing about performance in subgroups, because it averages over the whole spectrum. The AUC of a model evaluated on an enriched case–control set — many florid positives, many clean negatives — is inflated by spectrum bias relative to its performance on the ambiguous, comorbid, technically imperfect cases that dominate real practice.

The methodological apparatus that disciplines these inferences is by now codified. Park and Han laid out the design requirements for evaluating clinical AI performance — chiefly the demand for external, and ideally prospective, validation on data from sites and scanners the model never saw. The CLAIM (Mongan and colleagues, with a 2023 update) provides an item-by-item reporting standard for AI imaging studies; STARD-AI extends diagnostic-accuracy reporting; and CONSORT-AI and its protocol companion SPIRIT-AI govern the rare but essential randomized trials. Nagendran and colleagues, surveying studies comparing AI to clinicians, found pervasive deficiencies — few prospective designs, frequent absence of external validation, and overstated claims — a sobering reminder that the published AUC is the beginning, not the end, of evidence appraisal.

If validation science establishes how a model should be proven, failure analysis establishes the mechanisms by which a proven model nonetheless fails in deployment, and a physician supervising these systems must hold a mechanistic taxonomy of those failures. The first and most consequential is dataset shift: the deployment distribution differs from the training distribution. It decomposes into covariate shift, in which the input distribution changes while the input-to-label mapping is stable — a new scanner vendor, a different reconstruction kernel or iterative-reconstruction strength, a thicker slice, a contrast-timing protocol, a different patient demographic — and label shift, in which disease prevalence changes, with the predictive-value consequences derived in the prior section. Zech and colleagues furnished the canonical demonstration: a pneumonia model trained at one hospital generalized poorly to others, and probing revealed it had partly learned to recognize the hospital from image artefacts and then exploited the differing baseline pneumonia rates between sites, a confound that vanished into a respectable internal AUC and surfaced only on external data.

That finding exemplifies the second mechanism, shortcut learning: a model optimizing a loss will seize whatever feature most cheaply reduces it, and that feature is frequently a spurious correlate rather than the pathology. CT-specific shortcuts are legion — a chest tube or support line that co-occurs with the disease, a body-part-specific scanner setting, a laterality marker, the very presence of a follow-up scan implying known disease. The model's internal logic is opaque, so the shortcut is invisible until the correlation breaks. Closely allied is miscalibration: modern deep networks are systematically overconfident, their softmax outputs poorly matching empirical frequencies, so a reported probability of $0.9$ may correspond to a true positive rate well below that. Calibration is assessed by reliability diagrams and the expected calibration error, and a model can be well-discriminating (high AUC) yet badly calibrated, which corrupts any downstream Bayesian use of its probability and any risk-stratified workflow built upon it.

The remaining mechanisms are subtler but equally real. Neural networks are adversarially fragile and, more relevantly for clinical safety, fragile to genuine out-of-distribution inputs: an artefact-laden scan, an unusual anatomic variant, a pathology absent from training, or a corrupted volume can elicit a confident, wholly wrong output with no flag, because a standard classifier has no native ‘‘I have never seen this’’ response and will force every input into a known class. This is why out-of-distribution detection and uncertainty quantification are active safety requirements, not embellishments. Layered atop all the technical failures is the human one: automation bias, the tendency of clinicians to defer to algorithmic output — to discount a true finding the AI missed (omission error) or to accept a false finding the AI flagged (commission error). Automation bias is most dangerous precisely when the model is usually right, because trust is calibrated to the average and then misapplied to the rare failure. Wu and colleagues, analyzing FDA-cleared AI devices, documented how often clearance rested on limited, retrospective, single-site evaluation with sparse prospective or multi-site data, meaning the very failure modes catalogued here are frequently untested at the point a device reaches clinical hands. The synthesis is unforgiving: each failure mode converts an impressive in-silico metric into silent patient-level harm, and each is detectable only by deliberate external validation, calibration assessment, out-of-distribution monitoring, and disciplined human oversight.

Clinical integration is the discipline of deploying an imperfect estimator into a sociotechnical system of radiologists, ordering physicians, workflows, and liability such that net patient benefit is realized and the failure modes of the preceding section are contained. The first and most clarifying decision is the model's role, which determines the evidentiary burden it must meet. In a triage or worklist-prioritization role the AI reorders the reading queue — pushing a probable hemorrhage or aortic dissection to the top — without altering the report; here the human reads every study regardless, so the relevant metric is time-to-diagnosis and the dominant risk is a falsely reassuring de-prioritization, mitigated by tuning the threshold for very high sensitivity. In a concurrent or second-reader aid the AI annotation is presented during interpretation, and the central question becomes the performance of the human–AI team, which is emphatically not the performance of either alone: a model that is individually accurate can degrade team performance if it induces automation bias, and a model of modest standalone accuracy can improve outcomes if it surfaces the cases humans systematically miss. In an autonomous role the AI issues findings without human review, the configuration with the highest benefit ceiling and the highest harm floor, justified in 2026 only for narrowly scoped, exhaustively validated tasks under explicit governance.

The regulatory and life-cycle dimension is inseparable from the clinical one. Clearance pathways evaluate a model at a frozen point in time, yet the deployment environment is non-stationary — scanners are upgraded, protocols drift, case mix shifts seasonally and demographically — so a device that met its specification at clearance can silently decay, the phenomenon of performance drift. Responsible integration therefore mandates prospective post-deployment monitoring: tracking the alert rate, the positive predictive value against confirmed outcomes, calibration over time, and subgroup performance, with pre-specified triggers for recalibration, retraining, or withdrawal. This is the imaging instantiation of the broader recognition, expressed in regulatory thinking about a total-product-life-cycle approach and the special challenge of continuously learning systems, that an AI medical device is a process to be governed rather than an artefact to be approved once.

Underlying all of this is an epistemic reframing that this chapter has built toward. Topol's articulation of high-performance medicine as the convergence of human and machine intelligence is apt precisely because it is not a hand-off but a synthesis: the model supplies tireless, reproducible pattern detection at scale, and the physician supplies causal reasoning, integration of the clinical context the pixels never contained, calibration of confidence, and — crucially — accountability for the decision. The radiologist's role is not abolished by a capable model; it is elevated to that of an accountable supervisor who understands the estimator's architecture well enough to anticipate where it will fail, who interrogates its output rather than ratifying it, who recognizes the out-of-distribution case the model cannot, and who retains responsibility for the patient. That supervisory competence — knowing what the model computes, on what data, with what biases, at what operating point, and with what calibrated uncertainty — is the literacy this chapter exists to instill, and it is the connective tissue between the quantitative-imaging, evidence, reasoning, and error-science chapters that surround it.