• J Pain Symptom Manage · Dec 2023

    Review

    Treatment Effect Estimates from Pilot Trials Are Unreliable.

    • Jesse D Troy, Megan L Neely, Gina-Maria Pomann, Steven C Grambow, and Gregory P Samsa.
    • Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina, USA. Electronic address: Jesse.Troy@Duke.edu.
    • J Pain Symptom Manage. 2023 Dec 1; 66 (6): e672e686e672-e686.

    ContextThe CONSORT guideline defines a pilot trial as a small-scale version of a desired future efficacy trial that is intended to answer the key questions of whether and how a larger study should be done. For example, a pilot trial might evaluate different approaches to data collection or outcome measurement. However, pilot trials are unreliable for assessing treatment efficacy due to the statistical phenomenon called sampling variability.ObjectivesIn this tutorial we use computer simulation to demonstrate the influence of sampling variability on efficacy estimates from pilot trials, illustrating why pilot trial designs should not be used to evaluate whether a treatment is promising or not.MethodsWe simulate a 2-arm parallel group trial (N=20 per group) with a survival outcome as an example. Simulations are done under two scenarios: 1) the treatment is efficacious at the level of a hypothetical minimum clinically important difference (hazard ratio [HR] = 0.75); and 2) the treatment is not efficacious (HR=1).ResultsAs expected, in both simulated scenarios the range of observed results is distributed around the true treatment effect, HR=0.75 or HR=1. Importantly, ∼20% of trials simulated under scenario 1 incorrectly suggest the treatment may be harmful (HR > 1). Under scenario 2, half of the simulated studies incorrectly suggest the treatment is beneficial.ConclusionTreatment effect estimates from pilot trials should not be used to make future development decisions regarding a novel therapy because of the high risk of misleading conclusions.Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.

      Pubmed     Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…

What will the 'Medical Journal of You' look like?

Start your free 21 day trial now.

We guarantee your privacy. Your email address will not be shared.