Augmented Reality Scouting for Interactive 3D Reconstruction

AugmentedRealityScoutingforInteractive3DReconstruction

BernhardReitinger 1

1

ChristopherZach2DieterSchmalstieg1

InstituteforComputerGraphicsandVision,GrazUniversityofTechnology,Austria

2VRVisResearchCenter,Austria

ABSTRACT

Thispaperpresentsa rstprototypeofaninteractive3Drecon-structionsystemformodelingurbanscenes.AnAugmentedRe-alityscoutisapersonwhoisequippedwithanultra-mobilePC,anattachedUSBcameraandaGPSreceiver.Thescoutisexplor-ingtheurbanenvironmentanddeliversasequenceof2Dimages.TheseimagesareannotatedwithaccordingGPSdataandusediter-ativelyasinputfora3Dreconstructionenginewhichgeneratesthe3Dmodelson-the- y.Thisturnsmodelingintoaninteractiveandcollaborativetask.

Keywords:scouting,interactive3dreconstruction,urbanplanning1

INTRODUCTIONANDRELATEDWORK

Generating3Dmodelsofoutdoorscenesinurbanenvironmentsisoftenademandingtask,butnecessaryforapplicationssuchasmobileAugmentedReality(AR)[3,12],interactivevisualiza-tion[2,9],ormodel-basedtracking[13].Creatingthesemodelsisusuallydoneinanof ineprocess,usingconventional3Dmodelingtools,andinvolvestedioushoursofmanualdatapreparation.

Incontrast,manyinterestingapplicationsdemandthatmodelsmustbecreatedon-lineandon-site.Forexample,urbanplannersliketospontaneouslyexperimentwithvariationsoftheirarchitec-turaldesignswheninspectingaplannedconstructionsite.Thismeansthatthe3Dmodelgenerationmustbeperformedinterac-tivelytogiveimmediatefeedback.Ingeneral,mostapplicationsthatrequiredigitalreconstructionofarchitecturecanbene tfromimmediatefeedbackthatallowstoverifythereconstructionpro-cess.Providingthisinteractivityistheaimoftheworkpresentedinthispaper.

However,traditionalreconstructiontechniquesareaimedto-wardsahigh-accuracyoff-lineworkstyle,wheredataacquisitionanddataprocessingarestrictlyseparated.Theobjectiveofsuchsystemsistoobtainthebestscalabilityoftheoverallprocessbyfullautomationoftheacquisitionandreconstructionphase.Forexample,Akbarzadehetal.[1]usegeo-registeredvideosequencescapturedbyamulti-camerasetupmountedonavehicle.Thisdataisthenpost-processedinaseparateoff-linestageand nallygener-ates3Dmodelsofthecapturedenvironment.

Anotherapproachonlyusesaerialimagesforreconstructionur-banscenes[8].Area-basedsegmentationisusedtoclusterhomo-geneousphotometricpropertiesandcalculateadensemaptoob-tainthereconstruction.Thismethodcanbeusedtoreconstructlarge-scalearchitecturalscenes.However,highqualityaerialim-agesmustbeavailable.AnotherapproachispresentedbyWangetal.wherethetextureoffacadesisreconstructedbasedonanumberofphotographs[16].

AdifferentapproachcalledPhotoTourismwaspresentedbySnavelyetal.[14].Inthisproject,similarimagesofthesame

e-mail:

reitinger@tugraz.at

IEEE Virtual Reality Conference 2007

March 10 - 14, Charlotte, North Carolina, USA1-4244-0906-3/07/$20.00 ©2007 IEEE

buildingaretakenfromanexistingdatabaseandprocessedinor-dertogenerateasparse3Dpointcloud.Thecamerapositionsandorientationsarereconstructedforeachimageandimage-basedren-deringisprovided.Sincethissystemaimsatlotsofsimilarimages,theprocessingtimefordozensofimagesisbeyondonehour.Oncethe3Dreconstructionis nished,theresultcanbeobservedinaninteractiveviewer.

Alargebodyofworkonreconstructionalgorithmscanbefoundintheroboticscommunitybutwillnotbediscussedhere.Roboticsaswellasallthereconstructionworksmentionedaboveaimatau-tomated3Dreconstruction;noneofthembringsthehumanintothereconstructionloop.WeproposetoemployahumanARscoutwhoisabletospontaneouslyexploreandreconstructenvironmentswhicharenotyetknown.Theresultingmodelscanimmediatelybeinspectedandre nedbythescoutorusedbyabroaderremoteaudiencethroughawirelessconnection.Thistransformstheusu-allyoff-linemodelingtaskintoaninteractivetaskwhereagroupofpeopleandthescoutgeneratemodelson-the- y.

2SYSTEMOVERVIEW

Ourproposedinteractivereconstructionsystemconsistsoftwomainsub-systems,thescoutandthereconstructionserver(seeFig-ure1):

Figure1:Anoverviewoftheinteractive3Dreconstructionsystem.TheARscoutstorescurrentpositionandimagedatainthedatabase.Thereconstructionenginegetsanoti cationandcalculatesthe3Dmodelwhichisagainstoredinthedatabase.Finally,theresultcanbevisualizedforabiggeraudienceonaprojectionscreen.

TheARscoutacquiresgeo-referencedimagedatawithahand-heldARdeviceanddeliversittoaremotereconstructionserver.Theserverisresponsibleforprocessingtheindividualimagesinto

219