Contributed by Sherry Reuter
Pretrial Questionnaires may not be Predictors of Site Performance
While it is standard procedure to send pretrial questionnaires to choose potential sites, more than 94% of clinical trials in the United States fail to complete on time because of enrollment difficulties. It is evident that the current system is not working. A relatively low-cost way to address this issue is to select the sites most capable of recruiting patients. Companies use the information from pretrial questionnaires, quantify the data, weigh the most important criteria, and rank the different sites to determine the best sites for a particular trial. Unfortunately, the questionnaires currently used may not provide useful data to accurately predict a site’s performance in a specific trial. This could help explain why sites do not meet enrollment timeline goals. improving the site-selection process I recently conducted a study to understand the effectiveness of the site-selection process and ways it might be improved. Data were collected from pretrial questionnaires from potential sites. Statistical analyses were performed on these data to determine if the information supplied, and the way this information was used to select sites, was effective in identifying the sites most likely to successfully attain required enrollment. The actual enrollment performance of each site was compared with the information the site supplied on the pretrial questionnaire. In this study, 65 sites were evaluated. The enrollment goal of eight patients was not reached by 22 of the 65 (33.8%) sites. This correlates with the “one-third rule,” a common metric in the industry that in almost all clinical trials, one-third of sites will fail to enroll their quota, one-third will meet or exceed their goal, and the other third will show average performance. In this study, the enrollment commitment was met or exceeded by 43 sites (66.2%), which also correlates with the one-third rule. One reason so many sites may fail to meet their enrollment criteria is that many questionnaires do not ask the right questions nor use specific wording to collect accurate, useful data to predict a site’s performance. Because many issues affect whether a site will have the capacity and interest to make a particular trial a priority when the study is actually initiated, pretrial questions should specifically address these critical determining factors. asking the right questions In addition to analyzing the predictability of the information received from the questionnaires based on the final number of subjects each site enrolled, 19 specific questions were evaluated to learn which criteria were the most predictive of site performance. Although none of the 19 questions were statistically significant in predicting which sites would successfully enroll the required number of patients, chi-square analyses revealed that two questions had the closest correlation to whether sites were successful in reaching enrollment goals and thus were the best markers to signal a site’s possible performance. The two questions were: how many patients the sites forecasted they could enroll and the study coordinator’s years of experience. An example of a question often included on questionnaires that showed very low predictive value was whether there were any conflicting trials at the site. This question has little practical meaning, since trials ongoing at the time of site selection have little to do with what might be happening at the site when the sponsor’s specific trial initiates. Sponsors’ timelines often change, and it is difficult for sites to know for certain if they will be conducting other trials at some undetermined, future date. Another of the least predictive questions was interest in participating in the trial with the protocol as designed: 94% of the principle investigators stated they were interested in participating, but 34% of them failed to reach enrollment. Asking questions that are predictive of site success and those that correlate with actual site performance during the trial and eliminating nonpredictive ones are key to designing a useful questionnaire. An effective tool to use in conjunction with a questionnaire is a telephone conversation with potential investigators about their thoughts about the trial and an open, frank review of the answers on the questionnaire. The industry has an opportunity to make a tremendous difference in the drug-development process by reassessing this tool and the process by which sites are selected. Questioning the value of the current process and conducting research to improve it could lead to better site selection, faster enrollment, and the ability to bring new therapies to market sooner and at lower cost. Sherry Reuter, B.S.N., M.S., is a Senior Clinical Project Leader at Alexion Pharmaceuticals Inc., Cheshire, Conn., which is engaged in the discovery and development of therapeutic products aimed at treating a wide array of severe disease states, including cardiovascular and autoimmune disorders, inflammation, and cancer. Ms. Reuter also teaches The Processes of Clinical Research at George Washington University in the School of Medicine and Health Science Department’s Clinical Research Program. Ms. Reuter may be contacted at [email protected]. PharmaVoice welcomes comments on this article. E-mail us at [email protected]. October 2004 PharmaVOICE