Abstract The MediaMill TRECVID 2005 Semantic Video Search En(14)

UvA-MediaMill team participated in four tasks. For the detection of camera work (runid: A CAM) we investigate the benefit of using a tessellation of detectors in combination with supervised learning over a standard approach using global image information.

TRECVID 2005 Overall Search Results

IyadMahmoudSearch Topic

Figure12:Comparisonofautomatic,manual,andinteractivesearchresultsfor24topics.Resultsfortheusersofthelexicon-driven

retrievalparadigmareindicatedwithspecialmarkers.

of)entirethreadsispossible.Smoothtransitionanimationsexisttoenabletheusertohaveabetterintuitivefeelingofwhereheisbrowsinginthedataset.TheresultingbrowserisshowninFig.10.

weexpectthattheresultsofoursystemwillimproveasweaddmoresemanticconceptindices,usingoursemanticpath nderstrategy.

5.5

5.5.1

Experiments

AutomaticSearch

5.5.2ManualSearch

Wesubmittedtworunsforautomaticsearch,onebaselinerunusingthe naltextsearchstrategyonly,andonefullrunincorporatingtextandsemanticconcepts.AscanbeseeninFig.12thecombinedsemanticandtextrunout-performedthetextrunonnearlyallcounts.Wedidbestforthosetopicsthathadaclearmappingtothesemanticconceptindices,i.e.tennisfortopic156,meetingfortopic163(achievingthebestresultforthistopic)andbasketballfortopic165.Insomecasestheconceptweightingstrategywasnotoptimal,forexamplefortopic158.Inthiscasewedetectedtheaircraftindex,buttheconceptresultsweregivenaweightingof0intheresultfusionbecausetheinfor-mationcontentoftheconcepthelicopterwascalculatedtobemuchhigherthantheinformationcontentoftheconceptaircraft.Ifwehadutilizedtheaircraftdetectorinthiscase,wewouldhaveachievedanaverageprecisionof0.17,whichishigherthanthebestevaluatedaverageprecisionof0.14.Wehavedemonstratedthatautomaticsearchusingonlytextasinputisarealistictask.Weperformbetterthanthemedianforanumberoftopics,andevenachievethebestscoreforonetopic.Postulatingthatallothersys-temsincorporatemultimodalexamplesintheirsearch,thisisasigni cantresult.Theperformanceofoursearchen-gineisbestwhenoneormorerelatedindicesarepresent;

Wesubmittedonerunformanualsearchwhereweonlyusethe101conceptsinthelexicontoanswerthequeries.More-over,werestrictourselvestousingonlyvisualinformation.Forthirteentopicswescoreabovethemedian.Speci cally,fortwoqueries,i.e.vehiclewith ames(160)andtennisplayers(156)weperformthebestofallmanualruns,andfortwootherqueries,i.e.peoplewithbanners(161)andbasketballplayers(165)wearesecondbest.Fortenquerieswescorebelowthemedian,threeofthosearenotcoveredbyourlexicon,andsevenareperson-xtypequeries.Weperformbadlyforperson-xqueriesbecausethefeaturesde-scribevisualscenelayout,consequently,namesandfacesarenotmodeled.Fortheremainingfourteentopicsthereisonlyonei.e.boat(164)paredtoourautomaticsearchtextbaseline,weperformworseoneightqueries.Ofthoseeightqueries,thetextbaselineperformsbetterforallperson-xqueries,andforoneotherquery(164).Consequently,avisual-onlyap-proachoutperformsthetextbaselinein16queries,includingtheout-of-lexiconqueries.

Webelieveourresultssupportthelexicon-drivenretrievalapproachandshowtheimportanceofvisualanalysis.De-spitetheobviousdisadvantagesofusingonlyvisualinfor-mation,weoutperformthetextbaseline,andevenscorethebestofallmanualrunsintwoqueries.