Topic segmentation with an aspect hidden Markov model(4)

We present a novel probabilistic method for partially unsupervised topic segmentation on unstructured text. Previous approaches to this problem utilize the hidden Markov model framework (HMM). The HMM treats a document as mutually independent sets of words

1

1Introduction

Intheclassicalinformationretrieval(IR)problem,ausersearchesinacorpusoftextfordocumentswhichsatisfyherinformationneeds.Thisframeworkassumesanotionofdocumenti.e.thatthecorpusisdividedintocohesivesetsofwordseachexpressingasmallnumberofinformationneeds.

Insomesearch-worthytextcorpora,suchasnewswirefeeds,televisionclosedcap-tions,orautomaticspeechrecognition(ASR)transcriptsofstreamingaudio,thereisnoexplicitrepresentationofadocument.Thereareimplicitdocumentbreaks(e.g.televisionshows,radiosegments)butnocleardemarcationsofwheretheyoccur.Seg-mentationisacriticalsubtaskoftheIRprobleminthesesituations.

Tothisend,weimplementedanovelprobabilisticmethodoftopicsegmentationwhichcombinesasegmentinghiddenMarkovmodel[6]andanaspectmodel[5].Inthispaper,wedescribeourmethodanddemonstrategoodresultswhenappliedtonoisyASRtranscriptsandstreamsofclean(error-free)unsegmentedtext.

Thispaperisdividedintosixsections.Insection2,wesummarizeofprevioustechniquesanddescribehowourmethodrelatestothem.Insection3,wedescribethestandardHMMsegmentationapproach.Insection4,wedescribethetheorybehindtheaspectHMMapproach.Insection5,wereportonexperimentsonbothcleanandASRtext.Insection6,wepresentourconclusionsandsuggestionsforfuturework.2PreviousWork

Thereisaconsiderablebodyofpreviousresearchonwhichthisworkbuilds.Hearst[4]developedtheTextTilingalgorithmwhichusesawordsimilaritymeasurebetweensen-tencesto ndthepointbetweenparagraphsatwhichthetopicchanges.Thisapproachiseffectiveoncleantextwithexplicitsentenceandparagraphstructure.However,itisdif culttoimplementontextproducedbyaspeechrecognitionengine.InadditiontotheunstructurednatureofASRoutput,speechrecognitionenginesonunrestrictedaudiooftenhaveworderrorratesintherangeof20%to50%.SinceHearst’salgorithmcomputescosinesimilaritybetweenrelativelysmallgroupsofwordsoneithersideofasentenceboundary,itisunclearwhetheritwouldberobustenoughinthefaceofmanyerroneouswords.

Beefermanetal.[1]introducedafeature-basedsegmentationmethodwhichdoesnotrequiretextwithparagraphandsentencestructure.Thoughtheirmethodworkswell,manyofthederivedfeaturesarebasedonidentifyingcue-wordswhichindicateanimpendingtopicshift.Inourdomain,higherrorratesoftencloudsuchcuewordsmakingthemdif culttolearnanddetect.

ThemethodwepresentbuildsdirectlyontheHiddenmarkovmodel(HMM)ap-proachofMulbregtetal.[6].Weextendthismodelbyembeddingtheaspectmodel[5]intheHMM.Thisallowsforauni edmodelwithinwhichwe ndbothsegmentclusterstotraintransitionprobabilitiesandlanguagemodelstodetermineobservationemissionprobabilities.