Validation of Electronic Systems to Collect Patient-Reported Outcome (PRO) Data - Recommendations for Clinical Trial Teams: Report of the ISPOR ePRO Systems Validation Good Research Practices Task Force Abstract. Outcomes research literature has many examples of high-quality, reliable patient-reported outcome (PRO) data entered directly by electronic means, ePRO, compared to data entered from original results on paper. Clinical trial managers are increasingly using ePRO data collection for PRO-based endpoints. Regulatory review dictates the rules to follow with ePRO data collection for medical label claims. A critical component for regulatory compliance is evidence of the validation of these electronic data collection systems. Validation of electronic systems is a process versus a focused activity that finishes at a single point in time. Eight steps need to be described and undertaken to qualify the validation of the data collection software in its target environment: requirements definition, design, coding, testing, tracing, user acceptance testing, installation and configuration, and decommissioning. These elements are consistent with recent regulatory guidance for systems validation. This report was written to explain how the validation process works for sponsors, trial teams, and other users of electronic data collection devices responsible for verifying the quality of the data entered into relational databases from such devices. It is a guide on the requirements and documentation needed from a data collection systems provider to demonstrate systems validation. It is a practical source of information for study teams to ensure that ePRO providers are using system validation and implementation processes that will ensure the systems and services: operate reliably when in practical use; produce accurate and complete data and data files; support management control and comply with any existing regulations. Furthermore, this short report will increase user understanding of the requirements for a technology review leading to more informed and balanced recommendations or decisions on electronic data collection methods.

Principles of Good Practice for Decision Analytic Modeling in Health-Care Evaluation: Report of the ISPOR Task Force on Good Research Practices - Modeling Studies Abstract. Objectives: Mathematical modeling is used widely in economic evaluations of pharmaceuticals and other healthcare technologies. Users of models in government and the private sector need to be able to evaluate the quality of models according to scientific criteria of good practice. This report describes the consensus of a task force convened to provide modelers with guidelines for conducting and reporting modeling studies. Methods: The task force was appointed with the advice and consent of the Board of Directors of ISPOR. Members were experienced developers or users of models, worked in academia and industry, and came from several countries in North America and Europe. The task force met on three occasions, conducted frequent correspondence and exchanges of drafts by electronic mail, and solicited comments on three drafts from a core group of external reviewers and more broadly from the membership of ISPOR. Results: Criteria for assessing the quality of models fell into three areas: model structure, data used as inputs to models, and model validation. Several major themes cut across these areas. Models and their results should be represented as aids to decision making, not as statements of scientific fact; therefore, it is inappropriate to demand that models be validated prospectively before use. However, model assumptions regarding causal structure and parameter estimates should be continually assessed against data, and models should be revised accordingly. Structural assumptions and parameter estimates should be reported clearly and explicitly, and opportunities for users to appreciate the conditional relationship between inputs and outputs should be provided through sensitivity analyses. Conclusions: Model-based evaluations are a valuable resource for health-care decision makers. It is the responsibility of model developers to conduct modeling studies according to the best practicable standards of quality and to communicate results with adequate disclosure of assumptions and with the caveat that conclusions are conditional upon the assumptions and data on which the model is built.

Multinational Trials - Recommendations on the Translations Required, Approaches to Using the Same Language in Different Countries, and the Approaches to Support Pooling the Data: The ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Good Research Practices Task Force Report Abstract. Objectives: With the internationalization of clinical trial programs, there is an increased need to translate and culturally adapt patient-reported outcome (PRO) measures. Although guidelines for good practices in translation and linguistic validation are available, the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Task Force identified a number of areas where they felt that further discussion around methods and best practices would be beneficial. The areas identified by the team were as follows: 1) the selection of the languages required for multinational trials; 2) the approaches suggested when the same language is required across two or more countries; and 3) the assessment of measurement equivalence to support the aggregation of data from different countries. Methods: The task force addressed these three areas, reviewed the available literature, and had multiple discussions to develop this report. Results: Decision aid tools have also been developed and presented for the selection of languages and the approaches suggested for the use of the same language in different countries. Conclusion: It is hoped that this report and the decision tools proposed will assist those involved with multinational trials to 1) decide on the translations required for each country; 2) choose the approach to use when the same language is spoken in more than one country; and 3) choose methods to gather evidence to support the pooling of data collected using different language versions of the same tool.

Pediatric Patient-Reported Outcome Instruments for Research to Support Medical Product Labeling: Report of the ISPOR Good Research Practices for the Assessment of Patient-Reported Outcomes in Children and Adolescents Task Force Abstract. Patient-reported outcome (PRO) instruments for children and adolescents are often included in clinical trials with the intention of collecting data to support claims in a medical product label. The purpose of the current task force report is to recommend good practices for pediatric PRO research that is conducted to inform regulatory decision making and support claims made in medical product labeling. The recommendations are based on the consensus of an interdisciplinary group of researchers who were assembled for a task force associated with the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). In those areas in which supporting evidence is limited or in which general principles may not apply to every situation, this task force report identifies factors to consider when making decisions about the design and use of pediatric PRO instruments, while highlighting issues that require further research. Five good research practices are discussed: 1) Consider developmental differences and determine age-based criteria for PRO administration: Four age groups are discussed on the basis of previous research (<5 years old, 5-7 years, 8-11 years, and 12-18 years). These age groups are recommended as a starting point when making decisions, but they will not fit all PRO instruments or the developmental stage of every child. Specific age ranges should be determined individually for each population and PRO instrument. 2) Establish content validity of pediatric PRO instruments: This section discusses the advantages of using children as content experts, as well as strategies for concept elicitation and cognitive interviews with children. 3) Determine whether an informant-reported outcome instrument is necessary: The distinction between two types of informant-reported measures (proxy vs. observational) is discussed, and recommendations are provided. 4) Ensure that the instrument is designed and formatted appropriately for the target age group. Factors to consider include health-related vocabulary, reading level, response scales, recall period, length of instrument, pictorial representations, formatting details, administration approaches, and electronic data collection (ePRO). 5) Consider cross-cultural issues. Additional research is needed to provide methodological guidance for future studies, especially for studies involving young children and parents’ observational reports. As PRO data are increasingly used to support pediatric labeling claims, there will be more information regarding the standards by which these instruments will be judged. The use of PRO instruments in clinical trials and regulatory submissions will help ensure that children’s experience of disease and treatment are accurately represented and considered in regulatory decisions.

Background. Budget impact analyses (BIAs) are an essential part of a comprehensive economic assessment of a health care intervention and are increasingly required by reimbursement authorities as part of a listing or reimbursement submission. Objectives: The objective of this report was to present updated guidance on methods for those undertaking such analyses or for those reviewing the results of such analyses. This update was needed, in part, because of developments in BIA methods as well as a growing interest, particularly in emerging markets, in matters related to affordability and population health impacts of health care interventions. Methods. The Task Force was approved by the International Society for Pharmacoeconomics and Outcomes Research Health Sciences Policy Council and appointed by its Board of Directors. Members were experienced developers or users of BIAs; worked in academia and industry and as advisors to governments; and came from several countries in North America and South America, Oceania, Asia, and Europe. The Task Force solicited comments on the drafts from a core group of external reviewers and, more broadly, from the membership of the International Society for Pharmacoeconomics and Outcomes Research. Results. The Task Force recommends that the design of a BIA for a new health care intervention should take into account relevant features of the health care system, possible access restrictions, the anticipated uptake of the new intervention, and the use and effects of the current and new interventions. The key elements of a BIA include estimating the size of the eligible population, the current mix of treatments and the expected mix after the introduction of the new intervention, the cost of the treatment mixes, and any changes expected in condition-related costs. Where possible, the BIA calculations should be performed by using a simple cost calculator approach because of its ease of use for budget holders. In instances, however, in which the changes in eligible population size, disease severity mix, or treatment patterns cannot be credibly captured by using the cost calculator approach, a cohort or patient-level condition-specific model may be used to estimate the budget impact of the new intervention, accounting appropriately for those entering and leaving the eligible population over time. In either case, the BIA should use data that reflect values specific to a particular decision maker’s population. Sensitivity analysis should be of alternative scenarios chosen from the perspective of the decision maker. The validation of the model should include at least face validity with decision makers and verification of the calculations. Data sources for the BIA should include published clinical trial estimates and comparator studies for the efficacy and safety of the current and new interventions as well as the decision maker’s own population for the other parameter estimates, where possible. Other data sources include the use of published data, well-recognized local or national statistical information, and, in special circumstances, expert opinion. Reporting of the BIA should provide detailed information about the input parameter values and calculations at a level of detail that would allow another modeler to replicate the analysis. The outcomes of the BIA should be presented in the format of interest to health care decision makers. In a computer program, options should be provided for different categories of costs to be included or excluded from the analysis. Conclusions. We recommend a framework for the BIA, provide guidance on the acquisition and use of data, and offer a common reporting format that will promote standardization and transparency. Adherence to these good research practice principles would not necessarily supersede jurisdiction-specific BIA guidelines but may support and enhance local recommendations or serve as a starting point for payers wishing to promulgate methodology guidelines.