英文词频统计的java实现方法

需求概要

1.读取文件，文件内包可含英文字符，及常见标点，空格级换行符。

2.统计英文单词在本文件的出现次数

3.将统计结果排序

4.显示排序结果

分析

1.读取文件可使用BufferedReader类按行读取

2.针对读入行根据分隔符拆分出单词，使用java.util工具提供的Map记录单词和其出现次数的信息，HashMap和TreeMap均可，如果排序结果按字母序可选用TreeMap，本例选择用HashMap。

3.将Map中存储的键值对，存入List中，List中的元素为二维字符串数组，并将单词出现次数作为比对依据，利用list提供的sort方法排序，需要实现Comparator.compare接口方法。

4.循环遍历List中的String[]，输出结果。

部分功能实现

输出结果

 public void printSortedWordGroupCount(String filename) {
 　　List<String[]> result = getSortedWordGroupCount(filename);
 　　if (result == null) {
 　　　　System.out.println("no result");
 　　return;
 　　}
 　　for (String[] sa : result) {
 　　　　System.out.println(sa[1] + ": " + sa[0]);
 　　}
 }

统计

 public Map<String, Integer> getWordGroupCount(String filename) {
 　　try {
 　　FileReader fr = new FileReader(filename);
 　　BufferedReader br = new BufferedReader(fr);
 　　String content = "";
 　　Map<String, Integer> result = new HashMap<String, Integer>();
 　　while ((content = br.readLine()) != null) {
 　　　　StringTokenizer st = new StringTokenizer(content, "!&(){}+-= ':;<> /\",");
 　　　　while (st.hasMoreTokens()) {
 　　　　String key = st.nextToken();
 　　　　if (result.containsKey(key))
 　　　　　　result.put(key, result.get(key) + 1);
 　　　　else
 　　　　　　result.put(key, 1);
 　　　　}
 　　}
 　　br.close();
 　　fr.close();
 　　return result;
 
 　　} catch (FileNotFoundException e) {
 　　　　System.out.println("failed to open file:" + filename);
 　　　　e.printStackTrace();
 　　} catch (Exception e) {
 　　　　System.out.println("some expection occured");
 　　　　e.printStackTrace();
 　　}
 　　　　return null;
 }

排序

     public List<String[]> getSortedWordGroupCount(String filename) {
         Map<String, Integer> result = getWordGroupCount(filename);
         if (result == null)
             return null;
         List<String[]> list = new LinkedList<String[]>();
         Set<String> keys = result.keySet();
         Iterator<String> iter = keys.iterator();
         while (iter.hasNext()) {
             String key = iter.next();
             String[] item = new String[2];
             item[1] = key;
             item[0] = "" + result.get(key);
             list.add(item);
         }
         list.sort(new Comparator<String[]>() {
             public int compare(String[] s1, String[] s2) {
                 return Integer.parseInt(s2[0])-Integer.parseInt(s1[0]);             }
         });
         return list;
     }

针对本程序的简易测试用例自动生成

使用随机数产生随机单词，空格，并在随机位置插入回车

 public boolean gernerateWordList(String filePosition, long wordNum) {
 　　FileWriter fw = null;
 　　StringBuffer sb = new StringBuffer("");
 　　try {
 　　　　fw = new FileWriter(filePosition);
 
 　　　　for (int j = 0; j < wordNum; j++) {
 　　　　int length = (int) (Math.random() * 10 + 1);
 　　　　for (int i = 0; i < length; i++) {
 　　　　　　char ch = (char) ('a' + (int) (Math.random() * 26));
 　　　　　　sb.append(ch);
 　　　　}
 　　　　fw.write(sb.toString());
 　　　　fw.write(" ");
 　　　　sb = new StringBuffer("");
 　　　　if (wordNum % (int) (Math.random() * 8 + 4) == 0)
 　　　　　　fw.write("\n");
 　　　　}
 　　} catch (IOException e) {
 　　　　e.printStackTrace();
 　　　　return false;
 　　} finally {
 　　　　try {
 　　　　　　fw.close();
 　　　　} catch (IOException e) {
 　　　　　　e.printStackTrace();
 　　　　　　System.out.println("failed to close file");
 　　　　　　return false;
 　　　　}
 　　　}
 　　　return false;
 　　}
 }

实际用例结果

源自维基百科specification词条节选

"Specification" redirects here. For other uses, see Specification (disambiguation).
There are different types of specifications, which generally are mostly types of documents, forms or orders or relates to information in databases. The word specification is defined as "to state explicitly or in detail" or "to be specific". A specification may refer to a type of technical standard (the main topic of this page).
 
Using a word "specification" without additional information to what kind of specification you refer to is confusing and considered bad practice within systems engineering.
 
A requirement specification is a set of documented requirements to be satisfied by a material, design, product, or service.[1]
 
A functional specification is closely related to the requirement specification and may show functional block diagrams.
 
A design or product specification describes the features of the solutions for the Requirement Specification, referring to the designed solution or final produced solution. Sometimes the term specification is here used in connection with a data sheet (or spec sheet). This may be confusing. A data sheet describes the technical characteristics of an item or product as designed and/or produced. It can be published by a manufacturer to help people choose products or to help use the products. A data sheet is not a technical specification as described in this article.
 
A "in-service" or "maintained as" specification, specifies the conditions of a system or object after years of operation, including the effects of wear and maintenance (configuration changes).
 
Specifications may also refer to technical standards, which may be developed by any of various kinds of organizations, both public and private. Example organization types include a corporation, a consortium (a small group of corporations), a trade association (an industry-wide group of corporations), a national government (including its military, regulatory agencies, and national laboratories and institutes), a professional association (society), a purpose-made standards organization such as ISO, or vendor-neutral developed generic requirements. It is common for one organization to refer to (reference, call out, cite) the standards of another. Voluntary standards may become mandatory if adopted by a government or business contract.

　　

结果截取

a:  16
of:  16
or:  15
to:  14
the:  12
specification:  11
is:  7
A:  7
and:  7
may:  6
in:  5
.:  5
as:  5
be:  5
standards:  4
by:  4
technical:  4
sheet:  4
refer:  4
product:  3
Specification:  3
organization:  3
data:  3
types:  3
an:  2
word:  2
functional:  2
association:  2
It:  2
government:  2
are:  2
national:  2
describes:  2
help:  2
information:  2
developed:  2
group:  2
which:  2
including:  2
this:  2
corporations:  2
for:  2
design:  2
designed:  2
requirement:  2
mostly:  1
practice:  1
bad:  1
products.:  1
considered:  1
type:  1
without:  1
years:  1
professional:  1

　　

可进行的拓展

建立重载函数，添加参数，可以根据用户需要倒序排列统计结果，即按照单词出现次数从少到多排序。

工程源码包地址：https://coding.net/u/jx8zjs/p/wordCount/git

git@git.coding.net:jx8zjs/wordCount.git