One of the essential problems in computer vision is to recover the distance information of an object from captured images. Its application areas range from industrial inspection and reverse engineering to autonomous robot navigation

OpticalEngineering45 12 ,127201 December2006

Depthfrommotionanddefocusblur

Huei-YungLin,MEMBERSPIEChia-HongChang

NationalChungChengUniversityDepartmentofElectricalEngineering168UniversityRoad

Min-HsiungChia-Yi621,TaiwanE-mail:lin@u.edu.tw

Abstract.Findingthedistanceofanobjectinascenefromintensityimagesisanessentialprobleminmanyapplications.Inthiswork,wepresentanovelmethodfordepthrecoveryfromasinglemotionanddefocusblurredimage.Undertheassumptionofuniformlateralmotionofthecameraduring niteexposuretime,boththepinholemodelandthecamerawitha niteapertureareconsidered.Itisshownthattheimageblurproducedbyuniformlinearmotionofthecameraisinverselyproportionaltothedistanceoftheobject.Furthermore,ifthespeedoftherelativemotionisknown,thedepthoftheobjectcanbeacquiredbyidentifyingtheblurparameters.Animageblurmodelisformulatedbasedongeometricoptics.Theblurextentisestimatedbyintensitypro leanalysisandfocusmeasurementofthedeblurredimages.Theproposedmethodisveri edexperimentallyusingdifferenttypesoftestpatternsinanindoorenvironment.©2006SocietyofPhoto-OpticalInstrumentation

Engineers. DOI:10.1117/1.2403851

Subjectterms:motinblur;defocusblur;depthrecovery.

Paper050643RRreceivedAug.9,2005;revisedmanuscriptreceivedMay27,2006;acceptedforpublicationJun.5,2006;publishedonlineDec.11,2006.

1Introduction

Oneoftheessentialproblemsincomputervisionistore-coverthedistanceinformationofanobjectfromcapturedimages.Itsapplicationareasrangefromindustrialinspec-tionandreverseengineeringtoautonomousrobotnaviga-tionandcomputergraphics.Typically,thevisualcuesob-servedintherecordedimagesareusedfordepthperceptionofthescene.Forexample,the3-Dinformationcanbeen-codedinthetextureorshadinginformationoftheobject,imagedisparityfrommultipleviewpoints,depthof eldoftheoptics,etc.Thisworkaimstoaddresstheproblemofdepthrecoveryusingthevisualcuesprovidedbybothcam-eramotionandopticaldefocus.Speci cally,monlyusedtechniquesfordepthrecoveryincludeshapefromstereoormotion,shapefromshading,shapefromsilhouettes,andphotometricstereo.1Thesemethodsrequireeithermultipleimagescapturedfromdifferentviewpoints,ordifferentilluminationconditionsappliedforthesingleviewpointimageacquisition.Althoughitispos-sibletoachieveexcellent3-Dreconstructionresultsfrommultipleviewpoints,thecomputationalcostisconsiderablyexpensive.Inadditiontogeneraldepthrecoveryalgo-rithms,whichrelyonthechangesoftheenvironmentortheimagingposition,therearealsosomeothertechniquesthatutilizetheactiveadjustmentsoftheinternalcameraparam-eters.Someoftheproposedmethodsincludedepthfromzoominganddepthfromfocus/defocus.Thedepthinforma-tionisextractedbycomparingseveralimagesrecordedbyasinglecamerawithdifferentcameraparametersettings.Amotorizedzoomlensisusuallyrequiredtochangethezoomorfocuspositionsintheseapproaches.

Depthfromdefocusblurisaclassicapproachtorecover

thedistanceofanobjectandsimultaneouslyrestoreafo-cusedimageusingafewout-of-focusimages.2–4Theim-agesareusuallyobtainedfromasingleviewpoint,andthecamerahastoremainstaticduringimageacquisition.Thedepthoftheobjectisthencomputedusingtheamountofdefocusblurassociatedwiththerecordedimage.Recently,defocusimagecuehasalsobeencombinedwithstereotechniquesfordensedepthmaprecovery.Deschênes,Ziou,andFuchs5proposedauni edapproachforacooperativeandsimultaneousestimationofdefocusblurandspatialshiftsfromastereoimagepair.Basedongeneralizedmo-ment,expansion,theyhadformulatedasystemofequationsfordepthcomputation.Rajagopalan,Chadhuri,andMudenagudi6modeledthedepthandthefocusedimageindividuallyasMarkovrandom elds.Theyusedtwode-focusedstereoimagepairstoobtainadensedepthmapaswellasafocusedimageofthescene.

Differentfromopticaldefocus,motionblurisaresultof niteacquisitiontimeofpracticalcamerasandtherelativemotionbetweenthecameraandthescene.Forarapidscenechange,itisnotnegligibleevenwithapinholecam-eramodel.Traditionally,imagedegradationcausedbymo-tionbluristreatedasanundesirableartifactandusuallyhastoberemovedbeforefurtherprocessing.7Recently,motionblurhasbeenusedinvariousapplicationssuchassurveil-lancesystems,8computeranimation,9increasingthespatialresolutionofstillimagesfromvideosequences,10ormea-suringthespeedofamovingvehicle.11Althoughtheresto-rationtechniquesforimagedegradationcausedbymotionblurhavebeeninvestigatedforthepastfewdecades,12–14fairlylittleworkhasbeendonefordepthrecoveryusingmotionblurcues.Someofthepreviousresearchdealingwithcamera orobject motionandopticaldefocuscanbefound,forexample,inRefs.15–17.

ThemostrelevantresearchtothisworkisprobablytheworkgivenbyMylesanddaVitoriaLobo.15Theypre-