2015 IEEE International Conference on Robotics and Automation (ICRA)Washington State Convention CenterSeattle, Washington, May 26-30, 2015

SalientRegionsDetectionforIndoorRobotsusingRGB-DData

LixingJiang,ArturKochandAndreasZell

Abstract—Thegoalofsaliencydetectionistohighlightob-jectsinimagedatathatstandoutrelativetotheirsurrounding.Therefore,saliencydetectionaimstocaptureregionsthatareperceivedasimportant.Themostrecentbottom-upapproachesforsaliencydetectionmeasurecontrastbasedonvisualfeaturesin2Dscenes,ly,we rstsegmentanimageintoregionstoevaluatetheobjectuniquenessandconsistencyusinggraph-basedsegmentation.Then,weutilizetheregioncolor,depth,layoutandboundaryinformationtoproducerobustforegroundandbackgroundsaliencymeasures.Finally,wecombinethetwosaliencymapsbasedonGaussianweights.Asaresult,ourapproachproduceshigh-qualitysaliencymaps,whichmaybeusedforfurtherprocessinglikeobjectdetectionorrecognition.Experimentalresultsontwodatasetscompareourmethodwiththestateoftheartandhighlightitseffectiveness.

I.INTRODUCTION

A.Motivation

Foranintelligentrobot,aswithahuman,salientregionde-tectionplaysavitalroleinidentifyingand lteringinforma-tioninunknownandcomplexenvironments.Visualsaliencymapscancompetentlyguidetheattentionofanagenttopo-tentiallyrelevantcandidatesandlocationsinascene,whichisbene cialformanyapplicationslikeobjectdetectionandrecognition.Currentmethodsestimatethevisualsaliencybasedonglobalorlocalcontrastsofcolorsortexturesinanimage[1]–[3].However,suchmethodshavedif cultiesinhandlingvariationsinlightingandhomogeneouscolordistributionsbetweenforegroundandbackground.Asaconsequence,saliencydetection,albeitconsideredpracticallyuseful,isstillatechnologicallychallengingprobleminthe eldofcomputervision.

Withtheadventoflow-costRGB-DsensorsliketheMicrosoftKinect,thesuitabilityofRGB-D-basedmethodshasbecomemoreuniversal.Byutilizingadditionaldepthinformationandderivedfeatures,theviewofsaliencybe-comesmorepreciseandthusmorefeasibleforpracticalapplications[4],[5].Depthdatamakesitpossibletoseparateobjectswhicharesimilarinappearance.Inspiredbythoseadvances,weincorporatedepthvalueswithvisualfeaturestoestimatesalientregions.Asimplepracticalscenarioisamobileservicerobotasanobjectrecognitionsysteminareal-worldindoorenvironment.

Themajorfocusofthiswork,therefore,isthedevelopmentofsaliencydetectionmeasurestomeettherequirementsof

L.Jiang,A.KocharewiththeChairofCognitiveSystems,headedbyProf.A.Zell,ComputerScienceDepartment,UniversityofTue-bingen,Sand1,D-72076Tuebingen,Germany{lixing.jiang,

Fig.1.Saliencymapexamples.RGBimagesamples(leftcolumn)fromthepublicdatasetin[4],graph-basedRGB-Dsegmentation(middlecolumn)andtheresultingsaliencymap(rightcolumn).

artur.koch,andreas.zell}

@uni-tuebingen.de

indoormobilerobots.Inadditiontoconsideringcontrastofimages,wecombinedepthcueswithmeasuresofdistance,color,spatiallayoutandboundaryconnectivitytocalculateasaliencymap.Fig.1(a)showsthreescenescapturedbyaservicerobotinanindoorenvironmentfrom[4].Multipleforegroundobjectsofpotentialinterestcanbeseeninthesescenes.ThesegmentedcandidateregionsarevisualizedwithdifferentcolorsinFig.1(b).Insteadofattendingtoentiresegmentedregions,weexpectthattherobotcanidentifythemostvisuallynoticeableforegroundobjectsthroughthesaliencymap(Fig.1(c)).

Forthispurpose,we rstintroduceasegmentationmethodwhichappliesagraph-basedalgorithmforcoloranddepth.Thegraph-basedsegmentationisdesignedtoidentifyhomo-geneousregionsbasedoncoloraswellasdepthcues.Thealgorithmclusterspixelsinregardtosimilarpropertiesbutretainstheuniquenessandconsistencyofdifferentobjects.Thisclusteringinthesegmentationstagelargelydecreasesthecomputationalcomplexitysincewidespreadareasmayusuallybe lteredasbeingvisuallyunimportantduetolowvariance.Afterdiscussingdifferentsaliencymethodsinusetoday,wewillproposeanewsaliencyestimationapproachthatintegratescolor,depth,spatiallayout,andboundaryconnectivity.ThroughthefusionofRGBanddepthdata,theproposedmethodprovidesgoodresultsdespitethepres-enceofhomogeneouscolordistributionsbetweenforegroundandbackgroundareas.Tomeasuretheperformanceoftheapproach,weevaluateitontwodatasetsagainstdifferentstate-of-the-artalternatives.Theresultsshowthatcombining