EVIDEX® Signal Detection

Fresh, clean, and comprehensive real-world data. 

Aggregated and harmonized, global clinical data from the TriNetX federated network, as well as claims data, integrated with public data that is processed and prepared to allow you to go broad and deep with greater speed and accuracy. 

Data Sources for Signal Detection

Signal detection in pharmacovigilance is the process of actively searching for and identifying safety signals from a wide variety of data sources. Signal Detection in GVP Module IX is one of the core stages of GVP Module IX for Signal Management. This guide will explain the sources of data and information used in signal detection, the statistical aspects of signal detection, documenting the signal detection process, and what next generation signal detection software looks like.

The most common source for signals comes from spontaneous reporting systems that are required by regulators to be kept by biopharma companies. These data are stored in a database that is either hosted and maintained by the company itself, or outsourced to a technology or full services contract research organization (CRO). Signal detection on this database is required as a part of periodic monitoring. Additionally, data from national databases like the FDA Adverse Event Reporting System (FAERS), EudraVigilance, and VigiBase are important sources of potential signals.

While spontaneous reporting systems remain the gold standard for signal detection, other sources including, but not limited to, scientific literature that discuss adverse events and active monitoring systems such as the FDA Sentinel system should be considered.

GVP IX Signal Detection

Once a drug is on the market, it will quickly amass individual case safety reports (ICSRs). It is not feasible to review the contents of each report for the purposes of signal detection. Thus, appropriate prioritization thresholds must be put in place to focus attention on groups of ICSRs. There are both descriptive as well as quantitative ways of achieving prioritization.

Adverse events should be categorized using the MedDRA hierarchy to be able to identify signals using different levels of granularity. Preferred Term (PT) level of analysis has been shown to provide the best combination of sensitivity and positive predictive value for signal detection. [Hill R, Hopstadius J, Lerch M, Noren GN An attempt to expedite signal detection by grouping related adverse reaction terms. Drug Saf. 2012; 35:1194-1195.]

Also, to focus efforts on serious and important events, the Important Medical Event (IME) list should be used to filter results that need further medical review. The IME list can be even further refined by mapping a list of Designated Medical Events (DME) that should always be prioritized. Anyone conducting signal detection should have access to a company’s core common data sheet (CCDS) file of labeled events. This ensures that further work on validating and assessing a signal is not done for information that is already known.

Quantitative signal detection is most commonly done through disproportionality statistics: the ratio of the proportion of spontaneous ICSRs of a specific drug-event combination to the proportion that would be expected if no association existed between the product and the event. There are various different ways to calculate disproportionality, with the most common using frequentist methods such as the reporting odds ratio (ROR) or the proportional reporting ratio (PRR). Bayesian methods such as the Empirical Bayes Geometric Mean (EBGM) and the Information Component (IC) offer additional clarity, especially when dealing with smaller numbers of events.

ROR calculation

The ROR for a given Adverse Event Y and Drug X depends on four values:

a = the number of primary case reports involving Drug X that do report Adverse Event Y

b = the number of primary case reports involving Drug X that do NOT report Adverse Event Y

c = the number of primary case reports that do NOT involve Drug X but do report Adverse Event Y

d = the number of primary case reports that do NOT involve Drug X and do NOT report Adverse Event Y

It can be helpful to visualize these values in a 2×2 contingency table:

The formula for ROR is (a/b) / (c/d).

An equivalent formula is  (a*d)/(c*b).

An ROR is an inference about a general population from sample data. As such, it comes with some uncertainty, which can be quantified with a confidence interval.

ROR 95% CI, Lower = e^{ln(ROR)-(1.96*SQRT(1/a + 1/b + 1/c + 1/d))}

ROR 95% CI, Upper = e^{ln(ROR)+(1.96*SQRT(1/a + 1/b + 1/c + 1/d))}

Example calculation

ROR = (a/b)/(c/d) = (4/50)/(100/1000) = .08/.10 = .80

ROR 95% CI, Lower = .283

ROR 95% CI, Upper = 2.261

Any value of disproportionality greater than 1 indicates that the drug-event combination is being reported more than expected. It is common to use the lower bound of a 95% confidence interval greater than one or the absolute mean value greater than two as a prioritization threshold. [Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Saf. 2015; 38: 577–587]

Trends can also be evaluated on a drug-event combination, for both frequency and disproportionality to better understand changes over time. Large, unexpected jumps in periodic reporting could, for example, highlight a manufacturing quality issue that a point-in-time analysis would not show.

Furthermore, applying signal detection specifically for individual populations of patients should be considered. Pediatric and geriatric populations, for example, have special characteristics that should be differentiated.

Documenting the Signal Detection Process

Good Pharmacovigilance Practices (GVP) are a set of measures put into practice in 2012 to facilitate the performance of pharmacovigilance in the European Union (EU). GVP is broken out into several modules that govern different aspects of pharmacovigilance processes. GVP Module IX – Signal Management (GVP IX) provides general guidance and requirements on scientific and quality aspects of signal management.

The guidelines apply to Marketing Authorization Holders (MAH) for medicines authorized by the European Medicines Agency (EMA) and EU Member States. However, in the absence of formal regulation on the process of signal management by other health authorities, such as FDA, these guidelines have become the de facto global standard.

There are no specifications on how these requirements are met, but in the age of software-as-a-service (SaaS), technology is playing an ever bigger role.

In addition to signal detection in GVP IX, there is the concept of signal validation, prioritization, and assessment. A full overview of the GVP IX signal management process can be found in GVP Module IX for Signal Management: The Complete Guide. The overarching theme throughout all stages of GVP IX, is that the processes used should be adequately documented and the steps taken in the management of a specific signal should be tracked and auditable. In short, the system needs to provide for transparency into the who, what, why, and when of the process.

Next Generation Signal Detection Software

Next generation pharmacovigilance signal detection is built with a view toward how data and analytics software can better interact not only with traditional sources like ICSR databases, FAERS, VigiBase, and clinical trial data, but with emerging disparate sources such as social media, claims, EHR and other unstructured data. Bringing these pools of information together creates an opportunity to enhance signaling algorithms, make validations and assessment more efficient, and ultimately get answers to drug safety questions faster.

Linking all of the data requires ontology mapping. Drug name and active ingredient represented as NDC, RxCUI or ICSR drugs are resolved to one record. Adverse event coding in MedDRA is mapped back to verbatim labeling and ICD-10 codes. Drugs are characterized by ATC classifications, NDF-RT, label status, and more.  These ontologies and mappings need to be provided off-the-shelf to drive immediate, actionable insight.

How software interacts with the data is important. How end users engage with software is even more important. Complicated, slow, and unintuitive software leads to a poor user experience. Legacy software and platforms that were built during web 1.0 are no longer acceptable. And datamining in 2022 and beyond should not require a data scientist. Today’s signal detection software needs to reinvent how end users feel about the tools they use for their day-to-day job. Safety reviewers must want to engage with the software. When this happens, an organization’s pharmacovigilance software shifts from a system of record to a system of intelligence, and even competitive advantage.

Evidex, a cloud-based, software-as-a-service (SaaS) drug safety data, analytics and signal detection platform shifts the paradigm of pharmacovigilance software and enables the science of pharmacovigilance to advance at a rapid pace. Further innovation will come when end users are empowered to take advantage of disparate data sources through a modern user experience.