Abstract The MediaMill TRECVID 2005 Semantic Video Search En(12)

UvA-MediaMill team participated in four tasks. For the detection of camera work (runid: A CAM) we investigate the benefit of using a tessellation of detectors in combination with supervised learning over a standard approach using global image information.

Figure10:InterfacesoftheMediaMillsemanticvideosearchengine.OnthelefttheCrossBrowsershowingresultsfortennis.OntoptheSphereBrowser,displayingseveralsemanticthreads.Bottomright:activelearningusingasemanticcluster-basedvisualizationintheGalaxyBrowser.

Mostexistingsystemsbrowsekeyframesinsequence(left-right,top-down)[28].Hence,relationsbetweenframesarenottakenintoaccount.Fore ectiveinteractionthismaybeunappropriateastheusercannotbene tfromtheinherentstructurefoundinvideocollections.Therefore,(2)Inthevisualization,relationsbetweenkeyframesshouldbetakenintoaccounttoallowselectionofseveralframesbyoneuseraction.

Forthesereasons,visualizationofkeyframesincludingsupportforbrowsingandexploringisessentialinaninter-activesearchsystem.Weexploredthreeadvancedvisual-izations.5.4.1

CrossBrowser

5.4.2GalaxyBrowser

Tovisualizequery-by-conceptresultsweproposeaCross-Browser.Thebrowserdisplaystwoorthogonaldimensions.Thehorizontaloneisthetime-thread,usingtheoriginalTRECVIDshotsequence.Theverticaldimensioncontainstherankedlistofqueryresults.TheGUIgivestheuseracrosslayoutofnearbyshotsonthescreen.Itexploitstheobservationthatsemanticallysimilarshotstendtoclusterinthetimedimension.TheresultingbrowserisvisibleinFig.10.

Tospeedupthesearchwithinthetimelimitation,wewanttosupporttheuserwithasystemthattheyareabletose-lectmorethanonekeyframeinonemouseaction.Itcanbeassumedthatthekeyframesrelevanttoasearchtopicsharesimilarfeatures.Hence,theyshouldbeclosetoeachotherinthefeaturespace.Therefore,visualizationbasedonthesimilaritybetweenthemwillmakethesearcheasierassimi-larimagesaregroupedtogetherinaspeci clocationofthesearchspace.Hence,lessnavigationandinteractionactionswillbeneeded.WeproposetheGalaxyBrowser,whichin-tegratesadvancedsimilaritybasedvisualizationwithactivelearning.

Thesimilaritybasedvisualizationof[15]isthebasisforourretrieval.Inbrief,wehavepointedoutthatforanoptimalvisualizationsystem,threerequirementshavetoobeyed:overview,structurepreservationandvisibility.The rstrequirementensuresthatthesetdisplayedwillbeabletorepresentthewholecollection,thesocalledrepresen-tativeset.Foruserinteraction,thecollectionshouldbeprojectedtothedisplayspace.Hence,thesecondrequire-menttriestopreservetherelationsbetweenkeyframesintheoriginalfeaturespace.The nalrequirementkeepsthecontentofdisplayedkeyframesfeasibleforinteraction.