Augmented Reality Scouting for Interactive 3D Reconstruction(3)

Figure3:Thereconstructionpipelineconsistsoffourmaincomponents:featureextraction,correspondencesearch,cameraposeestimation,anddensematching.Eachcapturedimageispassedthroughthispipelineinordertogenerateorenhancethe3Dmodel.

byusinganabsoluteposeprocedure[4],againcombinedwithaRANSACapproachtoincreasetherobustness.Theposeofeveryadditionalincomingimageiscalculatedonthebasisof2Dto3Dpointcorrespondences,whichisstrongerthanusingtheepipolarrelationshipbetweentwoviewsalone.

Purelyimagebasedreconstructionsarelocatedinalocalcoordi-natesystem,whichisupgradedtoaworld-referencesystemusingthemeasuredGPSlocationsofthecamerapositions.2Thetransfor-mationfromthelocaltotheglobalsystemisasimilaritytransforminthecaseofcalibratedcameras.

Thecameraposesandthesparsereconstructionconsistingof3Dpointstriangulatedfrompointcorrespondencesarere nedus-ingasimplebutef cientimplementationofsparsebundleadjust-ment[10].Ourimplementationallowsthere nementofthecameraintrinsicparametersandtheintegrationofGPSdatawithestimateduncertaintiesaswell.Theoutputofthisstepareoptimizedcameraorientationsandintrinsicparametersinthe rstplace.Addition-ally,sparse3Dpointscorrespondingtotheimagefeaturesvisibleinseveralviewsarere ned,too.3.4DenseDepthEstimation

Withtheknowledgeofthecameraparametersandtherelativeposesbetweenthesourceviewsdensecorrespondencesforallpixelsofaparticularkeyviewcanbeestimated.Sincetherelativeposebe-tweentheincorporatedviewsisalreadyknown,thisprocedureisbasicallyaone-dimensionalsearchalongthedepthraysforeverypixel.Triangulationofthesecorrespondencesresultsinadense3Dmodel,whichre ectsthetruesurfacegeometryofthecapturedob-jectinidealsettings.

WeutilizeaGPU-acceleratedplane-sweepapproachtogeneratethedepthmapforeachsourceview[17,18].Planesweeptech-niquesincomputervisionaresimpleandelegantapproachestoim-agebasedreconstructionwithmultipleviews,sincearecti cationprocedureasneededinmanytraditionalcomputationalstereometh-odsisnotrequired.The3Dspaceisiterativelytraversedbyparallelplanes,whichareusuallyalignedwithaparticularkeyview(Fig-ure4).Theplaneatacertaindepthfromthekeyviewinduceshomographiesforallotherviews,thusthereferenceimagescanbemappedontothisplaneeasily.

Iftheplaneatacertaindepthpassesexactlythroughthesur-faceoftheobjecttobereconstructed,thecolorvaluesfromthekeyimageandfromthemappedreferencesimagesshouldcoincideatappropriatepositions(assumingconstantbrightnessconditions).

theGPSantennaandtheprojectioncenterofthecameraare

very

close,weignoretheresultingoffsetbetweenthem.

2Since

Figure4:Planesweepingprinciple.Fordifferentdepthsthehomog-raphybetweenthereferenceplaneandthereferenceviewisvarying.Consequently,theprojectedimageofthereferenceviewchangeswiththedepthaccordingtotheepipolargeometry.

Hence,itisreasonabletoassignthebestmatchingdepthvalue(ac-cordingtosomeimagecorrelationmeasure)tothepixelsofthekeyview.Bysweepingtheplanethroughthe3Dspace(byvary-ingtheplanesdepthwrt.thekeyview)a3Dvolumecanbe lledwithimagecorrelationvaluessimilartothedisparityspaceimage(DSI)intraditionalstereo.Thereforethedensedepthmapcanbeextractedusingglobaloptimizationmethods,ifdepthcontinuityoranyotherconstraintonthedepthmapisrequired.Weemployasimplewinner-takes-allstrategytoassignthe naldepthvaluesforperformancereasons.3.5Output

Adepthmapisgeneratedasdescribedaboveforeverytripletofad-jacentviews,andthesingledepthimagesneedtobefusedintoonecommonmodel.Currently,weemployaverysimpletechnique:thedepthmapsareconvertedintocoloredpointclouds(usingthereferenceviewfortexturing),andthesepointsetsareconcatenatedtoobtainthecombinedmodel.Thisapproachallowsaneasyin-crementalupdateofthedisplayedmodelaftergenerationofanewdepthmap.Futureworkwilladdressthecreationof3Dsurfacemeshesfromthedepthmaps(e.g.[18]),whichrequiresmorecom-plexmethodstoassignatexturetotheresultingmodel.4

RESULTS

The rstprototypewastestedwithmultiplebuildingsatourcam-pus.Additionally,weusedittoreconstructanancientbrickwall

221