Use of endpoint adjudication to improve the quality and validity of endpoint assessment for medical device development and post marketing evaluation: Rationale and best practices. A report from the cardiac safety research consortium

Journal American Heart Journal
Authors Jonathan H Seltzer, MD, MA, MBA; Ted Heise, Phd, RAC; Peter Carson, MD; Daniel Canos, PhD, MPH; Jo Carol Hiatt, MD, MBA; Pascal Vranckx, MD, PhD; Thomas Christen, MD, PhD; Donald E Cutlip, MD
Year Published 2017
Link to article


This white paper provides a summary of presentations, discussions and conclusions of a Thinktank entitled “The Role of Endpoint Adjudication in Medical Device Clinical Trials”. The think tank was cosponsored by the Cardiac Safety Research Committee, MDEpiNet and the US Food and Drug Administration (FDA) and was convened at the FDA’s White Oak headquarters on March 11, 2016. Attention was focused on tailoring best practices for evaluation of endpoints in medical device clinical trials, practical issues in endpoint adjudication of therapeutic, diagnostic, biomarker and drug-device combinations, and the role of adjudication in regulatory and reimbursement issues throughout the device lifecycle. Attendees included representatives from medical device companies, the FDA, Centers for Medicare and Medicaid Services (CMS), endpoint adjudication specialist groups, clinical research organizations, and active, academically based adjudicators.

The manuscript presents recommendations from the think tank regarding

(1) rationale for when adjudication is appropriate,

(2) best practices establishment and operation of a medical device adjudication committee and

(3) the role of endpoint adjudication for post market evaluation in the emerging era of real world evidence.

Constructing the informatics and information technology foundations of a medical device evaluation system: a report from the FDA unique device identifier demonstration

Journal Journal of the American Medical Informatics Association
Authors Drozda, Joseph P.; Roach, James; Forsyth, Thomas; Helmering, Paul; Dummitt, Benjamin; Tcheng, James E.
Year Published 2017
Link to article


OBJECTIVE: The US Food and Drug Administration (FDA) has recognized the need to improve the tracking of medical device safety and performance, with implementation of Unique Device Identifiers (UDIs) in electronic health information as a key strategy. The FDA funded a demonstration by Mercy Health wherein prototype UDIs were incorporated into its electronic information systems. This report describes the demonstration’s informatics architecture.

METHODS: Prototype UDIs for coronary stents were created and implemented across a series of information systems, resulting in UDI-associated data flow from manufacture through point of use to long-term follow-up, with barcode scanning linking clinical data with UDI-associated device attributes. A reference database containing device attributes and the UDI Research and Surveillance Database (UDIR) containing the linked clinical and device information were created, enabling longitudinal assessment of device performance. The demonstration included many stakeholders: multiple Mercy departments, manufacturers, health system partners, the FDA, professional societies, the National Cardiovascular Data Registry, and information system vendors.

RESULTS: The resulting system of systems is described in detail, including entities, functions, linkage between the UDIR and proprietary systems using UDIs as the index key, data flow, roles and responsibilities of actors, and the UDIR data model.

CONCLUSION: The demonstration provided proof of concept that UDIs can be incorporated into provider and enterprise electronic information systems and used as the index key to combine device and clinical data in a database useful for device evaluation. Keys to success and challenges to achieving this goal were identified. Fundamental informatics principles were central to accomplishing the system of systems model.

Handling incomplete correlated continuous and binary outcomes in meta-analysis of individual participant data

Journal Statistics in Medicine
Authors Manuel Gomes, Laura Hatfield, Sharon-Lise Normand
Year Published 2016
Link to publication


Meta-analysis of individual participant data (IPD) is increasingly utilised to improve the estimation of treatment effects, particularly among different participant subgroups. An important concern in IPD meta-analysis relates to partially or completely missing outcomes for some studies, a problem exacerbated when interest is on multiple discrete and continuous outcomes. When leveraging information from incomplete correlated outcomes across studies, the fully observed outcomes may provide important information about the incompleteness of the other outcomes. In this paper, we compare two models for handling incomplete continuous and binary outcomes in IPD meta-analysis: a joint hierarchical model and a sequence of full conditional mixed models.We illustrate how these approaches incorporate the correlation across the multiple outcomes and the between-study heterogeneity when addressing the missing data. Simulations characterise the performance of the methods across a range of scenarios which differ according to the proportion and type of missingness, strength of correlation between outcomes and the number of studies. The joint model provided confidence interval coverage consistently closer to nominal levels and lower mean squared error compared with the fully conditional approach across the scenarios considered. Methods are illustrated in a meta-analysis of randomised controlled trials comparing the effectiveness of implantable cardioverter-defibrillator devices alone to implantable cardioverter-defibrillator combined with cardiac resynchronisation therapy for treating patients with chronic heart failure.


Adverse Event Triggered Event Reporting for Devices: Report of a Food and Drug Administration–Supported Feasibility Pilot of Automated Adverse Event Reporting

Journal Journal of Clinical Engineering
Authors Reed, Terrie L. MS; Levy, Daniel MS; Steen, Leslie Tompkins PhD; Roach, James BCS; Taylor, Thomas BA; Call, Karen PT, MBA; Marion, Jill MS, MBA; Drozda, Joseph P. Jr MD
Year Published 2016
Link to article


Despite US Food and Drug Administration (FDA) requirements for reporting medical device adverse events (AEs), only an estimated 10% of events are actually reported, and many of those lack important data. As part of its plan to strengthen postmarket surveillance of medical devices, the FDA sponsored a pilot project of the Adverse Event Triggered Event Reporting for Devices (ASTER-D) system developed by Outcome Sciences for automated AE reporting.

The objective of this study is to test the feasibility of using ASTER-D to report medical device AEs. This was a cooperative effort of Outcome Sciences, FDA, and Mercy Health. The ASTER-D system enables a new functionality within the electronic health record for initiating AE reports from triggering events with prepopulated patient- and incident-specific data and minimal disruption to clinician workflow. The ASTER-D system employs health information exchange principles to automate medical device safety reporting and create a “safety information exchange” that allows safety information collected by various organizations to be available to others to facilitate postmarket surveillance. Mercy implemented ASTER-D for AEs related to coronary stents occurring in its cardiac catheterization laboratories in the context of Mercy’s automated incident reporting software system and incorporated prototype unique device identifiers to link reports to key device attributes. Mercy succeeded in submitting an AE using ASTER-D to the FDA’s Medical Product Safety Network database. This pilot provides proof of concept of ASTER-D’s functionality. Further testing with greater numbers of AEs, with different devices, and in other healthcare settings, is required.

Meta-analysis of survival curve data using distributed health data networks: application to hip arthroplasty studies of the International Consortium of Orthopaedic Registries

Journal Research Synthesis Methods
Authors Cafri, Guy; Banerjee, Samprit; Sedrakyan, Art; Paxton, Elizabeth; Furnes, Ove; Graves, Stephen; Marinac-Dabic, Danica
Year Published 2015
Link to article


The motivating example for this paper comes from a distributed health data network, the International Consortium of Orthopaedic Registries (ICOR), which aims to examine risk factors for orthopedic device failure for registries around the world. Unfortunately, regulatory, privacy, and propriety concerns made sharing of raw data impossible, even if de-identified. Therefore, this article describes an approach to extraction and analysis of aggregate time-to-event data from ICOR. Data extraction is based on obtaining a survival probability and variance estimate for each unique combination of the explanatory variables at each distinct event time for each registry. The extraction procedure allows for a great deal of flexibility; models can be specified after the data have been collected, for example, modeling of interaction effects and selection of subgroups of patients based on their values on the explanatory variables. Our analysis models are adapted from models presented elsewhere – but allowing for censoring in the calculation of the correlation between serial survival probabilities and using the square root of the covariance matrix to transform the data to avoid computational problems in model estimation. Simulations and a real-data example are provided with strengths and limitations of the approach discussed.