Abstract The MediaMill TRECVID 2005 Semantic Video Search En(15)

UvA-MediaMill team participated in four tasks. For the detection of camera work (runid: A CAM) we investigate the benefit of using a tessellation of detectors in combination with supervised learning over a standard approach using global image information.

MeanAverage Precision

System Runs

Figure13:OverviewofallsearchrunssubmittedtoTRECVID2005,erswhoexploitedtheproposedparadigmareindicatedwithspecialmarkers.

5.5.3InteractiveSearch

Wesubmittedfourrunsforinteractivesearch.Threeusersfocussedonusingonlyonebrowser.Thefourthusersmixedallbrowsers.ResultsinFig.12indicatethatformostsearchtopics,usersoftheproposedparadigmforinteractivemulti-mediaretrievalscoreaboveaverage.Furthermore,usersofourapproachobtainatop-3averageprecisionresultfor19outof24topics.Bestperformanceisobtainedfor7topics.BestresultsareobtainedwiththeCrossBrowser.

Dependingonthesearchtopic,theproposedGalaxy-Browseraidsusersinsearchingfortherelevantsubsetofthecollection.Asthefeaturesusedarevisualbased,thesystemworkswellincaserelevantimagesofacertaintopicsharevisualsimilarity,e.g.queriesrelatedtotennisorcar.However,whentopicshavelargevarietyinvisualsettings,forinstancepersonxtopics,visualfeatureshardlyyieldad-ditionalinformationtoaidtheuserintheinteractivesearchprocess.Toourknowledge,noexistingfeaturesworkwellinthesecases.

Twosearchstrategieswerediscoveredduringtheinter-activeretrievaltaskusingtheSphereBrowser.Thereweretopicsforwhichmultipleclusterthreadsyieldedgoodresultsforthattopic,suchasTennis(156),Peoplewithbannersorsigns(161),Meeting(163)andTallbuilding(170).Forthesetopicsonlytherelevantpartsofthethreadsneededtobese-lected.AnotherselectionmethodwasfoundinqueriessuchasAirplanetakeo (167)andO cesetting(172).Heretherewereonlyalimitednumberofconsecutivevalidshotsvisibleineachthread,butbecauseofthecombinationofbothtimeandclusterthreadstherewasalwaysanothervalidbutnotyetselectedshotvisible.Forthesequeries,selectionwasdonebyhoppingfromonevalidresulttoanother.AlsoanumberoftopicswerenotanswerablebytheSphereBrowserbecauseoflackofnearbyshots.Theseincludepersonxtop-ics149,151,and153.

Togaininsightintheoverallqualityofourlexicon-drivenretrievalparadigm.Wecomparetheresultsofouruserswithallotherusersthatparticipatedintheretrievaltasksofthe2005TRECVIDbenchmark.WevisualizedtheresultsforallsubmittedsearchrunsinFig.13.Theresultsarestate-of-the-art.

6ExplorationofBBCRushes

TheBBCRushesconsistofrawmaterialusedtoproduceavideo.Sincethereislittletonospeech,thismaterialisverysuitableforvisual-onlyindexing.We rstsegmentedthevideo’susingourshotsegmentationalgorithm[2].Thenweappliedourbestperformingcameramotiondetector(seeSection4)ontheBBCrushesusingthemodelstrainedforthenewsdata.Tofurtherinvestigatetherobustnessofourvisualfeatures,weperformedvisual-onlyconceptde-tectionontheBBCrushesdata,withoutre-trainingthevisualmodels.Thevisualmodelsarethesameasusedinthevisualonlyfeaturetask(Section3.4)andinthemanualsearchtask(Section5.2).ThedetectorsthuslearnedonnewsdataaresubsequentlyevaluatedontheBBCrushesvideos.Obviously,notall101conceptsareuseful,sincetheyaretrainedonbroadcastnews.However,25conceptstranscendthenewsdomainandsomeperformsurprisinglywellontheBBCrushes:aircraft,bird,boat,building,car,charts,cloud,crowd,face,female,food,governmentbuild-ing,grass,meeting,mountain,outdoor,overlayedtext,sky,smoke,tower,tree,urban,vegetation,vehicle,waterbody.WedevelopedaversionoftheMediaMillsemanticvideosearchenginetailoredtotheBBCrushersbasedonthecom-putedindexes.Whilestillprimitiveintermsofutility,thesearchengineallowsuserstoexplorethecollectioninasur-prisingmanner.Theresultsagaincon rmtheimportanceofrobustvisualfeatures.Hence,forthistaskmuchisto