mahout 运行Twenty Newsgroups Classification实例

【mahout 运行Twenty Newsgroups Classification实例】的更多相关文章

mahout 运行Twenty Newsgroups Classification实例

按照mahout官网https://cwiki.apache.org/confluence/display/MAHOUT/Twenty+Newsgroups的说法,我只用运行一条命令就可以完成这个算法的调用了,如下: mahout@ubuntu:~/mahout-d-0.7/examples/bin$ ./classify-20newsgroups.sh 但是,我首先运行就出错了,因为我不是root账户,所以先改下路径,打开classify-20newsgroups.sh,替换/tmp/maho…

Twenty Newsgroups Classification实例任务之TrainNaiveBayesJob(一)

接着上篇blog,继续看log里面的信息如下: + echo 'Training Naive Bayes model' Training Naive Bayes model + ./bin/mahout trainnb -i /home/mahout/mahout-work-mahout/20news-train-vectors -el -o /home/mahout/mahout-work-mahout/model -li /home/mahout/mahout-work-mahout/lab…

Twenty Newsgroups Classification任务之二seq2sparse（5）

接上篇blog,继续分析.接下来要调用代码如下: // Should document frequency features be processed if (shouldPrune || processIdf) { docFrequenciesFeatures = TFIDFConverter.calculateDF(new Path(outputDir, tfDirName), outputDir, conf, chunkSize); } 首先不看shouldPrune变量,单单看proce…

Twenty Newsgroups Classification任务之二seq2sparse（3）

接上篇,如果想对上篇的问题进行测试其实可以简单的编写下面的代码: package mahout.fansy.test.bayes.write; import java.io.IOException; import java.util.List; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import o…

Twenty Newsgroups Classification任务之二seq2sparse

seq2sparse对应于mahout中的org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles,从昨天跑的算法中的任务监控界面可以看到这一步包含了7个Job信息,分别是:(1)DocumentTokenizer(2)WordCount(3)MakePartialVectors(4)MergePartialVectors(5)VectorTfIdf Document Frequency Count(6)MakePartialVect…

Twenty Newsgroups Classification任务之二seq2sparse（2）

接上篇,SequenceFileTokenizerMapper的输出文件在/home/mahout/mahout-work-mahout0/20news-vectors/tokenized-documents/part-m-00000文件即可查看,同时可以编写下面的代码来读取该文件(该代码是根据前面读出聚类中心点文件改编的),如下: package mahout.fansy.test.bayes.read; import java.util.ArrayList; import java.util…