Meta-classifier approach to reliable text classification(2)

A problem with automatic classifiers is that there is no way to know if a particular classification is just a guess or a certain answer. Reliable classification is the task of predicting whether a certain instance is correctly classified or not, i.e., a cl

Abstract

Aproblemwithautomaticclassi ersisthatthereisnowaytoknowifaparticu-

larclassi cationisjustaguessoracertainanswer.Reliableclassi cationisthe

taskofpredictingwhetheracertaininstanceiscorrectlyclassi edornot,i.e.,a

classi cationisclassi edaseitherreliableorunreliable.Whentheclassi cation

isclassi edasunreliable,itislikesaying“Idonotknow”,andtheinstance

doesnotreceiveaclassi cation.

Givenabaseclassi er,themeta-classi erapproachistotrainameta-

classi erthatpredictsthecorrectnessofeachclassi cationofthebaseclassi er.

Theclassi cationruleofthemeta-classi erapproachistoassignaclasspre-

dictedbythebaseclassi ertoaninstanceifthemeta-classi erdecidesthatthe

baseclassi cationisreliable.

Themeta-classi erapproachisappliedontextclassi cationtasksprovided

bytheCBStoanswerthefollowingproblemstatement:

Doesthemeta-classi erapproachprovideapracticalsolutionto

reliabletextclassi cation?

The rstpartoftheresearchstudiestextclassi ers,andprovidesananswer

totheresearchquestion:

1.Whichtextclassi ersachievehighaccuracyandatthesametimehave

smallspaceandtimecomplexity?

ExperimentsontheCBSdatasetsshowthatthenearestneighbourandthe

na¨ veBayesalgorithmincombinationwiththetfidftextrepresentationare

acceptabletextclassi ers.

Thesecondpartoftheresearchstudiesthemeta-classi erapproachtopro-

videananswertothesecondandthirdresearchquestion.Oursecondresearch

questionis:

2.Whattypeofmetadatarepresentationisbestsuitedforreliabletextclas-

si cation?

Themeta-classi eristrainedonseveraltypesofmetadatarepresentations.The

usedmetadatarepresentationsincludetheoriginalinstances,theprobability

distributionofthebaseclassi erandasetofbasicstatisticsabouttheclassi -

cationofthebaseclassi er.Fortaskswithmanyclasses,theoriginalinstances

representationisbest.Fortaskswithasmallnumberofclasses,theoriginal

instancesrepresentationandtheprobabilitydistributionarebothgoodcandi-

dates.

i