Issues in External Validation and Translation Methodology

Starting with the proposition that “if we want more evidence-based practice, we need more practice-based evidence, ” this article (a) offers questions and guides that practitioners, program planners, and policy makers can use to determine the applicabili

Startingwiththepropositionthat“ifwewant

moreevidence-basedpractice,weneedmorepractice-basedevidence,”thisarticle(a)offersquestionsandguidesthatpractitioners,programplanners,andpolicymakerscanusetodeterminetheapplicabilityofevidencetosituationsandpopulationsotherthanthoseinwhichtheevidencewasproduced(generalizability),(b)suggestscriteriathatreviewerscanusetoevaluateexternalvalidityandpotentialforgeneralization,and(c)rec-ommendsproceduresthatpractitionersandprogram planners can use to adapt evidence-basedinterventionsandintegratethemwithevidenceonthepopulationandsettingchar-acteristics,theory,andexperienceintolocallyappropriateprograms.Thedevelopmentandapplicationintandemofsuchquestions,guides,criteria,andprocedurescanbeasteptowardincreasingtherelevanceofresearchfordecisionmakingandshouldsupportthecreationandreportingofmorepractice-basedresearch having high external validity.Keywords:evaluation;externalvalidity;

application;relevance;practice-basedresearch;translation;dis-semination; research methods

EVALUATING THE

RELEVANCE,

GENERALIZATION, AND

APPLICABILITY OF

RESEARCH

Issues in External Validationand Translation Methodology

LAWRENCE W. GREEN

University of California at San Francisco

RUSSELL E. GLASGOW

Kaiser Permanente Colorado

AUTHORS’NOTE:Thisworkwasdevel-oped,inpart,withinaresearchcollaborationoncomplexinterventionsfundedbytheCana-dianInstitutesofHealthResearch,andtheComprehensiveCancerCenterattheUniver-sityofCaliforniaatSanFranciscoand,inpart,fromGrant#CA90974-01fromtheNationalCancerInstitute.WeareindebtedalsotoBarbaraMcCrayofKaiserPermanenteofColoradoforherexpertassistancewithfor-matting, editing, and references.

EVALUATION & THE HEALTH PROFESSIONS, Vol. 29 No. 1, March 2006126-153DOI:10.1177/0163278705284445© 2006 Sage Publications

126

Starting with the proposition that “if we want more evidence-based practice, we need more practice-based evidence, ” this article (a) offers questions and guides that practitioners, program planners, and policy makers can use to determine the applicabili

Green, Glasgow / ISSUES IN TRANSLATION METHODOLOGY127

ecentdevelopmentsinevidence-basedmedicineandpublichealthguidelineshavemadethegapbetweenscienceandprac-ticemoresalientandembarrassingtothehealthprofessionsandtheirsponsoringorganizations(InstituteofMedicine&CommitteeonQualityofHealthCareinAmerica,2001;McGlynnetal.,2003).Meta-analysesandstructuredreviewsthathaveproducedtheguide-linesforpracticefromcumulativebodiesofresearchliteraturehavemadeitdifficulttoignorethestrengthofevidence(e.g.,directevi-dencefromrandomizedcontrolledtrials[RCTs])orspecificpracticesononehand,whileunveilingontheotherthelimitationsofthatevi-dence(theweightofevidence)initsrelevanceformanypracticesitua-tions.Weightofevidencereferstoindirectevidenceincludingnon-experimentaldata,practitionerexperiences,andthecumulativewisdomderivedfromsystematicanalysisoftheseandanunderstand-ingofthesituationsandpopulationsinwhichtheywouldbeapplied(e.g.,Pasick,Hiatt,&Paskett,2004).MuchoftheresearchonwhichpracticeguidelineshavebeenbasedinthehealthprofessionshasbeenstrongoninternalvaliditythatprovidesstrengthofevidenceextendedfromType1translationtraditions(Ames&McBride,inpress),thankstotheemphasisthathasbeengiventoexperimentalcontrolintheeval-uationofevidence.Thesestudieshavebeenweak,however,onexter-nalvaliditythatwouldaddtotheweightofevidenceasappliedtoType2translationofsciencetothevariedcircumstancesofpractice.Mostjudicialandregulatoryagenciesmustresttheirdecisionsmoreonweightofevidencebecausenosinglestudyinvolvinghumanbehaviororsocialchangecanunequivocallyestablishcausation(Haack,2005;Krimsky,2005;Rohrbach,Grana,&Valente,inpress;Steinberg&Luce,2005).Thiscommentaryontheevidence-basedpracticeliteratureexaminestherelativeneglectofexternalvalidityanditsconsequencesfortherelevance,generalizability,andapplica-bilityofresearchintypicalandvariedcircumstancesofmedicalandpublichealthpractice.Tobeclear:Well-controlledefficacystudieshaveanimportantplaceindeterminingcausation;theproblemisthatthecurrentevidencebaseandevaluationschemesconsistalmostentirelyofsuchresearchandverylittle“effectiveness”research(Flay,1986)thatattemptstostudyprogramsundertypical,ratherthanoptimal conditions (Glasgow, Lichtenstein, & Marcus, 2003).

Twomajorconclusionsemergefromourobservations.Oneisthatsomeoftheenergyandresourcesoftheevidence-basedpractice

R

Starting with the proposition that “if we want more evidence-based practice, we need more practice-based evidence, ” this article (a) offers questions and guides that practitioners, program planners, and policy makers can use to determine the applicabili

128Evaluation & the Health Professions / March 2006

movementneedstobedirectedtowarddevelopmentandapplicationofcriteriaandmeasuresofexternalvalidity.Thesecondisthatifthehealthprofessionsandtheirsponsorswantmorewidespreadandcon-sistentevidence-basedpractice,theywillneedtofindwaystogener-atemorepractice-basedevidencethatexplicitlyaddressesexternalvalidityandlocalrealities(Green&Ottoson,2004).Practice-basedresearchwouldproduceevidencethatmoreaccuratelyandrepresen-tativelyreflectsthe“program-contextinteractions”(Hawe,Shiell,Riley,&Gold,2004)andcircumstancesinwhichtheresultsoftheresearchareexpectedtobeapplied.Itwoulddoso,ofcourse,withsometrade-offoftheexperimentalcontrolexercisedinacademicallybased research.

THE IMBALANCE IN INTERNALAND EXTERNAL VALIDITY

Atime-honored,well-developed,andwidelyacceptedtraditionofjudgingandratinginternalvalidityhastranscendedthedisciplines.Thehealthprofessionshave …… 此处隐藏：35391字，全部文档内容请下载后查看。喜欢就下载吧 ……