赢得netflix推荐系统大奖的算法

IV.MATRIXFACTORIZATIONWITHTEMPORALDYNAMICSMatrixfactorizationwithtemporaldynamicswasalreadydescribedinlastyear’sProgressReport[3],orwithmoredetailinaKDD’09paper[8].ThemajorenhancementforthisyearistheincorporationoftheimprovedbaselinepredictorsdescribedinSec.III.

Thefullmodel,whichisknownastimeSVD++[8]isbasedonthepredictionrule

r ui=bui+qTi

pu(tui)+|N(u)| 1

yj

.(13)

j∈∑

N(u)

Here,theexactde nitionofthetime-dependentbaseline

predictor,bui,follows(10).

AsistypicalforaSVD++model[7],weemploytwosetsofstaticmoviefactors:qi,yi∈Rf.The rstset(theqis)iscommontoallfactormodels.Thesecondset(theyis)facilitatesapproximatingauserfactorthroughthesetofmoviesratedbythesameuser,usingthenormalized

sum|N(u)| 1

∑j∈N(u)yj.Differentnormalizationsoftheform|N(u)| αcouldbeemployed.Ourchoiceofα=1 xingthevarianceofthesum(seealso[7]formoreattemptsatintuitiononthischoice.)

Userfactors,pu(t)∈Rfaretime-dependent.Wemodeledeachofthecomponentsofpu(t)T=(pu1(t),...,puf(t))inthesamewaythatwetreateduserbiases.Inparticularwehavefoundmodelingafter(8)effective,leadingto

puk(t)=puk+αuk·devu(t)+pukt

k=1,...,f.

(14)

Herepukcapturesthestationaryportionofthefactor,αuk·devu(t)approximatesapossibleportionthatchangeslinearlyovertime,andpuktabsorbstheverylocal,day-speci cvari-ability.

Wewereoccasionallyalsousingamorememoryef cientversion,withouttheday-speci cportion:

puk(t)=puk+αuk·devu(t)k=1,...,f

(15)

Thesamemodelwasalsoextendedwiththeaforementionedfrequencies.Sincefrequencyaffectstheperceptionofmovies,wetriedtoinjectfrequencyawarenessintothemoviefactors.Tothisendwecreatedanothercopyofthemoviefactors,foreachpossiblefrequencyvalue.Thisleadstothemodel

r 1

ui=bui+(qTi+qTi,fui)

pu(tui)+|N(u)|

yj (16)

j∈∑

.N(u)

Herethede nitionofbuiisfrequency-awarefollowing(11).Noticethatwhilethetransitiontofrequency-awarebiaseswasmeasurablyeffective,theintroductionoffrequency-dependentmoviefactorswasbarelybene cial.A.What’sintheblend?

Weincludedmultiplevariationsofthematrixfactorizationmodelsintheblend.Allmodelsarelearnedbystochasticgradientdescentapplieddirectlyontherawdata,nopre-orpost-processingareinvolved.Inotherwords,allparameters(biases,user-factorsandmovie-factors)aresimultaneously

5

learnedfromthedata.Constants(learningratesandregu-larizationtobespeci edshortly)aretunedtoreachlowestRMSEafter40iterations.(Practically,onecangiveortakearoundteniterationswithoutameaningfulRMSEimpact).However,forblendingwehavefoundthatover-trainingishelpful.Thatis,weoftenletthealgorithmrunfarmorethan40iterations,therebyover ttingthetraindata,whichhappenstobebene cialwhenblendingwithotherpredictors.

The rstmodelistheoneusingrule(13),togetherwiththemorememoryef cientuser-factors(15).Thesettingscontrol-lingthelearningofbias-relatedparametersareasdescribedinSec.III-D.Asforlearningthefactorsthemselves(qi,puandyj),weareusingalearningrateof0.008andregularizationof0.0015,wherethelearningratedecaysbyamultiplicativefactorof0.9aftereachiteration.Finally,forαukthelearningrateis1e-5andtheregularizationis50.Thesesamesettingsremainthesamethroughoutthissection.Thethreevariantswithinourblendare:

1)f=20,#iterations=40,RMSE=0.89142)f=200,#iterations=40,RMSE=0.88143)f=500,#iterations=50,RMSE=0.8815

Thenextmodelstillemploysrule(13),butwiththemoreaccurateuser-factorrepresentation(14).Thisaddsonetypeofparameter,pukt,whichislearnedwithalearningrateof0.004andregularizationof0.01.Thetwovariantswithintheblendwerebothheavilyover-trainedtoover tthetrainingdata:1)f=200,#iterations=80,RMSE=0.88252)f=500,#iterations=110,RMSE=0.8841

Finallywehaveourmostaccuratefactormodel,whichfollows(16).Whilemainnoveltyofthismodel(overthepreviousone)isinthebiasterm,wealsoaddedthefrequency-speci cmovie-factorsqi,fregularizationui.Theirrespectivelearningrateis2e-5,withof0.02.Theblendincludessixvariants:1)f=200,#iterations=40,RMSE=0.87772)f=200,#iterations=60,RMSE=0.87873)f=500,#iterations=40,RMSE=0.87694)f=500,#iterations=60,RMSE=0.87845)f=1000,#iterations=80,RMSE=0.87926)

f

=2000,#iterations=40,RMSE=0.8762

Later,werefertothemodelwithf=200and#iterations=40

as[PQ2].

V.NEIGHBORHOODMODELSWITHTEMPORALDYNAMICSThemostcommonapproachtoCFisbasedonneigh-borhoodmodels.Whiletypicallylessaccuratethantheirfactorizationcounterparts,neighborhoodmethodsenjoypop-ularitythankstosomeoftheirmerits,suchasexplainingthereasoningbehindcomputedrecommendations,andseamlesslyaccountingfornewenteredratings.ThemethoddescribedinthissectionisbasedonSec.5ofourKDD’09paper[8].Recently,wesuggestedanitem-itemmodelbasedonglobaloptimization[7],whichwillenableusheretocapturetimedynamicsinaprincipledmanner.Thestaticmodel,withouttemporaldynamics,iscenteredonthefollowingprediction