Your Clinical AI Works in the U.S. Will It Work in São Paulo?

Key takeaways:

• A recent peer-reviewed study published in JHEP Reports tested whether an AI model for hepatocellular carcinoma (HCC) risk, built on U.S. RWD, could generalize to populations in Latin America and Asia-Pacific. The U.S. model achieved AUC 0.93. Applied directly to non-U.S. data, performance dropped to AUC 0.84. Retraining on local data restored performance to AUC 0.92. 
• The drop and the recovery, not the headline U.S. number, are the most instructive part of the result. They illustrate what clinical AI generalizability actually requires: not an algorithm that works everywhere, but a data foundation that makes local validation and retraining possible wherever deployment is planned. 
• The study was conducted within the TriNetX federated network, which enabled multi-site training, internal-external validation, and region-specific retraining across 93 healthcare organizations in the U.S., Latin America, and Asia-Pacific without transferring patient data across institutional or national boundaries. 

The fourth post in this blog series, 66.9%, 85%, 18 Months: What Clinical AI Actually Delivers When the Data Is Right, laid out a set of numbers from organizations that have paired artificial intelligence (AI) with high-quality real-world data (RWD). One was a model that predicts HCC risk 6 to 36 months before diagnosis. That result was presented briefly, as one of several the eBook, The Real-World Data Advantage: Why Clinical Operations Teams Are Rethinking AI Strategy, develops in full. 

This post goes deeper on that one result, because the details matter. And because they reveal something about the relationship between data and AI generalizability that doesn’t show up in a single headline stat. 

The study, conducted by researchers at Beth Israel Deaconess Medical Center, MIT, and TriNetX, developed and validated a set of models called LIRIC (LIver cancer RIsk Computation) for predicting three-year HCC risk using routinely collected electronic health record (EHR) data. It’s the kind of result that, if it holds up in prospective deployment, could meaningfully change who gets screened for one of the most aggressive cancers in the world. 

But the real story is in how it was validated. 

The Setup

HCC screening today has a specific structural problem. Current guidelines target patients with cirrhosis, but the framework misses two groups. An estimated 13% of patients who develop HCC lack a documented cirrhosis diagnosis, making them ineligible for screening under current protocols. And even among patients who are eligible for surveillance, only a minority, about 20%, actually undergo screening. The screening pathway catches some of the people at risk. It misses others, with consequences that track demographic and geographic disparities. 

The LIRIC study set out to test whether routinely collected EHR data could be used to identify high-risk patients in both populations: risk-stratified within cirrhosis and flagged within the general (non-cirrhotic) population. Adults over 40 without prior HCC were included, with cases identified through ICD codes 6 to 36 months before diagnosis. 

The U.S. dataset was substantial. De-identified longitudinal EHR data from 64 U.S. healthcare organizations in the TriNetX federated network, each contributing an average of 13 years of data. The general-population cohort comprised 46,679 HCC cases and 1,128,202 controls. Features were drawn from structured EHR fields (demographics, encounters, diagnoses, procedures, labs, and medications), standardized across institutions using the TriNetX data model and mapped to standard vocabularies including ICD-10-CM, LOINC, and RxNorm. 

The U.S. Result

On the U.S. data, the neural network variant of LIRIC (LIRIC-NN) achieved an AUC of 0.93 (95% CI: 0.9218 to 0.9289) for three-year HCC risk prediction, using 46 clinical features. The model held up across internal-external validation, with average AUCs of 0.93 maintained across sites and across race and ethnicity strata, and with good calibration. 

At an operating point corresponding to a standardized incidence ratio of approximately 31 (the threshold at which biannual surveillance is considered cost-effective for non-cirrhotic hepatitis B patients), sensitivity was 48.2%, specificity was 96.8%, positive predictive value was 1.44%, and number needed to screen was 70. 

That’s a strong result. It’s also the part of the story that most AI-in-clinical-development coverage would stop at. 

The Generalizability Test

The harder and more instructive question was whether a model trained on U.S. data would work elsewhere. The study tested this directly, using data from 29 healthcare organizations across Brazil, Colombia, Georgia, India, and Taiwan, also in the TriNetX network. The non-U.S. cohort comprised 5,992 HCC cases and 1,081,171 controls. 

Applied directly, without retraining, the U.S. model’s AUC on non-U.S. data dropped to 0.84. 

That number is worth sitting with for a moment. It’s still better than many clinical prediction models achieve on their primary validation set. But it’s a meaningful drop from 0.93, and it illustrates a principle that’s often glossed over in AI-in-healthcare discussions: clinical AI does not generalize across populations for free. The distribution of underlying liver disease etiologies differs across regions. Risk factor prevalence differs. Documentation practices differ. Care pathways and screening norms differ. A model that learned the signal from one population will see a different version of that signal in another. 

Then came the recovery. After retraining on non-U.S. data, the LIRIC-NN AUC climbed to 0.92. Models trained entirely on non-U.S. data (rather than retrained from the U.S. version) reached AUC 0.94 on local test sets, confirming that the predictive signal is strong in these populations when models are built for them. 

What the Recovery Actually Required

The retraining result is where the data foundation argument this blog series has been building becomes concrete in a new way. 

Retraining a model for a new region sounds simple in the abstract. In practice, it requires a specific kind of underlying infrastructure. 

It requires comprehensiveness extended globally. You can’t retrain on Latin American and Asia-Pacific populations if your data network doesn’t reach those populations. The 29 non-U.S. healthcare organizations that made the retraining possible, spanning five countries across two continents, represent a form of coverage most RWD networks don’t have. 

It requires standardization across heterogeneous sources. Healthcare organizations in different countries operate with different EHR vendors, different coding systems, different documentation standards, and different care pathway norms. For retraining to produce a comparable model, the underlying data has to be harmonized to a consistent data model. The LIRIC study used the TriNetX standard data model with curated terminologies to achieve this across 93 HCOs spanning the U.S. and five other countries. 

It requires federation. Clinical data often cannot leave the institution where it originated, especially across international borders where data residency rules differ. A federated architecture, where de-identified data stays at the contributing institution and analyses run against it without requiring data transfer, is what makes multi-region retraining operationally possible in the first place. The LIRIC paper explicitly names this as the property of the TriNetX platform that “lays the groundwork for scalable, multi-institutional implementation.” 

And it requires transparency. The LIRIC study followed the TRIPOD guidelines for reporting prediction model development and validation. It’s published, peer-reviewed, and specifies its methods, including acknowledging that U.S.-trained performance declined in non-U.S. cohorts before retraining. That’s what allows other researchers, regulators, and potential deployers to evaluate it. A proprietary model validated only on internal benchmarks, no matter how impressive the numbers, can’t support the same kind of scrutiny. 

The Generalizable Principle

The LIRIC result is, in one sense, a story about one disease. But the pattern it illustrates applies well beyond HCC. 

Any clinical AI model trained on data from one population will need to be validated, and in many cases retrained, before it can be trusted in another. That’s not a weakness of the AI. It’s a reflection of how heterogeneous healthcare actually is across populations, geographies, and care systems. 

The question is whether the organizations and vendors behind those models have the data foundation to do the revalidation and retraining when it’s needed. If the network doesn’t extend to the populations where deployment is planned, the answer is no, regardless of how good the U.S. numbers look. If the data across geographies aren’t standardized, the retraining produces incomparable results. If the architecture isn’t federated, the data can’t be accessed at all. If the methods aren’t transparent, the retrained model can’t be defended to regulators. 

This is why generalizability in clinical AI isn’t primarily an algorithmic property. It’s a property of the data foundation the algorithm sits on. The comprehensiveness, quality, recency, and transparency pillars from the framework post, Why Two Clinical Teams Can Run the Same AI and Get Completely Different Results, are what make global validity achievable. Without them, the question of whether a clinical AI model works in São Paulo or Seoul or Taipei can’t be answered. It can only be guessed at. 

What This Means for Programs Running Globally

For clinical development organizations running or planning trials across multiple regions, the implications are practical. 

A feasibility model that works in the U.S. may not perform equivalently in Europe, Latin America, or Asia-Pacific without revalidation. A recruitment algorithm calibrated on U.S. data may under-identify or over-identify candidates elsewhere. A site selection model trained on one health system may not transfer to another. 

The organizations that can address this without pausing a program are the ones whose underlying data and AI infrastructure supports multi-region validation as a standard capability, not a bespoke project. The LIRIC study is one demonstration of what that capability looks like in practice. It isn’t the only one. The eBook develops additional examples across protocol design, patient identification, and trial operations, along with the evaluation framework for assessing whether a vendor’s infrastructure supports this kind of global rigor. 

The full picture is in The Real-World Data Advantage: Why Clinical Operations Teams Are Rethinking AI Strategy. 

About Steve Kundrot 

Steve is a technology and business leader with over 20 years of experience in clinical research, health analytics, consulting, and software development. As Chief Operating Officer, he oversees TriNetX’s core operational functions and leads the development of a unified product roadmap designed to revolutionize clinical research and accelerate drug development by optimizing clinical trial design, enhancing post-market safety, and delivering research-grade data and evidence that enable and expedite regulatory approvals.