1。 第一个bug

运行

echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer 后显示。提示：

- -bash: java: command not found。

那我就觉得可能是java没安装。然后，我就去官网

下载的是放到了/data 目录下，然后解压，

解压完成后，vim ./bashrc,打开，然后输入如下的内容。

保存退出，source ~/.bashrc 一下。

这时候再 echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer就可以了。

2.  第二个bug  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128) 这个bug

(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# ls

README.md  make_datafiles.py  url_lists

(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail#

(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl_abs_other/data/cnn/stories /data/rl_abs_other/data/dailymail/stories

Preparing to tokenize /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized...

Making list of files to tokenize...

Tokenizing 92579 files in /data/rl_abs_other/data/cnn/stories and saving in cnn_stories_tokenized...

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+20A9, decimal: 8361)

Untokenizable: ? (U+F06E, decimal: 61550)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+F022, decimal: 61474)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

PTBTokenizer tokenized 80043350 tokens at 42671.94 tokens per second.

Stanford CoreNLP Tokenizer has finished.

Successfully finished tokenizing /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized.

Preparing to tokenize /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized...

Making list of files to tokenize...

Tokenizing 219506 files in /data/rl_abs_other/data/dailymail/stories and saving in dm_stories_tokenized...

Untokenizable: ? (U+FFFC, decimal: 65532)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202D, decimal: 8237)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+2012, decimal: 8210)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202D, decimal: 8237)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202B, decimal: 8235)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202D, decimal: 8237)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+F001, decimal: 61441)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+F001, decimal: 61441)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+70E, decimal: 1806)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202F, decimal: 8239)

Untokenizable: ? (U+2010, decimal: 8208)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+206E, decimal: 8302)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+200D, decimal: 8205)

Untokenizable: ? (U+202A, decimal: 8234)

Untokenizable: ? (U+FFFD, decimal: 65533)

Untokenizable: ? (U+202C, decimal: 8236)

PTBTokenizer tokenized 203118231 tokens at 32507.27 tokens per second.

Stanford CoreNLP Tokenizer has finished.

Successfully finished tokenizing /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized.

Making bin file for URLs listed in url_lists/all_test.txt...

Writing story 0 of 11490; 0.00 percent done

Traceback (most recent call last):

  File "make_datafiles.py", line 253, in <module>

    write_to_tar(all_test_urls, os.path.join(finished_files_dir, "test.tar"))

  File "make_datafiles.py", line 182, in write_to_tar

    article_sents, abstract_sents = get_art_abs(story_file)

  File "make_datafiles.py", line 106, in get_art_abs

    lines = read_story_file(story_file)

  File "make_datafiles.py", line 78, in read_story_file

    lines = f.read().split('\n\n')

  File "/root/anaconda3/envs/jjenv_pytorch/lib/python3.6/encodings/ascii.py", line 26, in decode

    return codecs.ascii_decode(input, self.errors)[0]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128)

(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail#

然后我以为是编码问题，就去 make_datafiles.py 的文件开头加上 # coding: utf-8 ,但是没有解决问题，后来参考了一篇帖子https://blog.csdn.net/qq_36847641/article/details/78414718

所以就把我自己的代码，做如下更改，就可以了。

但是，

然后我就继续运行make_datafiles.py文件，然后一路都顺利直到完成。

(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl/rl_abs_other/data/dailymail/stories

Making bin file for URLs listed in url_lists/all_test.txt...

Writing story 0 of 11490; 0.00 percent done

Writing story 1000 of 11490; 8.70 percent done

Writing story 2000 of 11490; 17.41 percent done

Writing story 3000 of 11490; 26.11 percent done

Writing story 4000 of 11490; 34.81 percent done

Writing story 5000 of 11490; 43.52 percent done

Writing story 6000 of 11490; 52.22 percent done

Writing story 7000 of 11490; 60.92 percent done

Writing story 8000 of 11490; 69.63 percent done

Writing story 9000 of 11490; 78.33 percent done

Writing story 10000 of 11490; 87.03 percent done

Writing story 11000 of 11490; 95.74 percent done

Finished writing file finished_files/test.tar

Making bin file for URLs listed in url_lists/all_val.txt...

Writing story 0 of 13368; 0.00 percent done

Writing story 1000 of 13368; 7.48 percent done

Writing story 2000 of 13368; 14.96 percent done

Writing story 3000 of 13368; 22.44 percent done

Writing story 4000 of 13368; 29.92 percent done

Writing story 5000 of 13368; 37.40 percent done

Writing story 6000 of 13368; 44.88 percent done

Writing story 7000 of 13368; 52.36 percent done

Writing story 8000 of 13368; 59.84 percent done

Writing story 9000 of 13368; 67.32 percent done

Writing story 10000 of 13368; 74.81 percent done

Writing story 11000 of 13368; 82.29 percent done

Writing story 12000 of 13368; 89.77 percent done

Writing story 13000 of 13368; 97.25 percent done

Finished writing file finished_files/val.tar

Making bin file for URLs listed in url_lists/all_train.txt...

Writing story 0 of 287227; 0.00 percent done

Writing story 1000 of 287227; 0.35 percent done

Writing story 2000 of 287227; 0.70 percent done

Writing story 3000 of 287227; 1.04 percent done

Writing story 4000 of 287227; 1.39 percent done

Writing story 5000 of 287227; 1.74 percent done

Writing story 6000 of 287227; 2.09 percent done

Writing story 7000 of 287227; 2.44 percent done

Writing story 8000 of 287227; 2.79 percent done

Writing story 9000 of 287227; 3.13 percent done

Writing story 10000 of 287227; 3.48 percent done

Writing story 11000 of 287227; 3.83 percent done

Writing story 12000 of 287227; 4.18 percent done

Writing story 13000 of 287227; 4.53 percent done

Writing story 14000 of 287227; 4.87 percent done

Writing story 15000 of 287227; 5.22 percent done

Writing story 16000 of 287227; 5.57 percent done

Writing story 17000 of 287227; 5.92 percent done

Writing story 18000 of 287227; 6.27 percent done

Writing story 19000 of 287227; 6.61 percent done

Writing story 20000 of 287227; 6.96 percent done

Writing story 21000 of 287227; 7.31 percent done

Writing story 22000 of 287227; 7.66 percent done

Writing story 23000 of 287227; 8.01 percent done

Writing story 24000 of 287227; 8.36 percent done

Writing story 25000 of 287227; 8.70 percent done

Writing story 26000 of 287227; 9.05 percent done

Writing story 27000 of 287227; 9.40 percent done

Writing story 28000 of 287227; 9.75 percent done

Writing story 29000 of 287227; 10.10 percent done

Writing story 30000 of 287227; 10.44 percent done

Writing story 31000 of 287227; 10.79 percent done

Writing story 32000 of 287227; 11.14 percent done

Writing story 33000 of 287227; 11.49 percent done

Writing story 34000 of 287227; 11.84 percent done

Writing story 35000 of 287227; 12.19 percent done

Writing story 36000 of 287227; 12.53 percent done

Writing story 37000 of 287227; 12.88 percent done

Writing story 38000 of 287227; 13.23 percent done

Writing story 39000 of 287227; 13.58 percent done

Writing story 40000 of 287227; 13.93 percent done

Writing story 41000 of 287227; 14.27 percent done

Writing story 42000 of 287227; 14.62 percent done

Writing story 43000 of 287227; 14.97 percent done

Writing story 44000 of 287227; 15.32 percent done

Writing story 45000 of 287227; 15.67 percent done

Writing story 46000 of 287227; 16.02 percent done

Writing story 47000 of 287227; 16.36 percent done

Writing story 48000 of 287227; 16.71 percent done

Writing story 49000 of 287227; 17.06 percent done

Writing story 50000 of 287227; 17.41 percent done

Writing story 51000 of 287227; 17.76 percent done

Writing story 52000 of 287227; 18.10 percent done

Writing story 53000 of 287227; 18.45 percent done

Writing story 54000 of 287227; 18.80 percent done

Writing story 55000 of 287227; 19.15 percent done

Writing story 56000 of 287227; 19.50 percent done

Writing story 57000 of 287227; 19.84 percent done

Writing story 58000 of 287227; 20.19 percent done

Writing story 59000 of 287227; 20.54 percent done

Writing story 60000 of 287227; 20.89 percent done

Writing story 61000 of 287227; 21.24 percent done

Writing story 62000 of 287227; 21.59 percent done

Writing story 63000 of 287227; 21.93 percent done

Writing story 64000 of 287227; 22.28 percent done

Writing story 65000 of 287227; 22.63 percent done

Writing story 66000 of 287227; 22.98 percent done

Writing story 67000 of 287227; 23.33 percent done

Writing story 68000 of 287227; 23.67 percent done

Writing story 69000 of 287227; 24.02 percent done

Writing story 70000 of 287227; 24.37 percent done

Writing story 71000 of 287227; 24.72 percent done

Writing story 72000 of 287227; 25.07 percent done

Writing story 73000 of 287227; 25.42 percent done

Writing story 74000 of 287227; 25.76 percent done

Writing story 75000 of 287227; 26.11 percent done

Writing story 76000 of 287227; 26.46 percent done

Writing story 77000 of 287227; 26.81 percent done

Writing story 78000 of 287227; 27.16 percent done

Writing story 79000 of 287227; 27.50 percent done

Writing story 80000 of 287227; 27.85 percent done

Writing story 81000 of 287227; 28.20 percent done

Writing story 82000 of 287227; 28.55 percent done

Writing story 83000 of 287227; 28.90 percent done

Writing story 84000 of 287227; 29.25 percent done

Writing story 85000 of 287227; 29.59 percent done

Writing story 86000 of 287227; 29.94 percent done

Writing story 87000 of 287227; 30.29 percent done

Writing story 88000 of 287227; 30.64 percent done

Writing story 89000 of 287227; 30.99 percent done

Writing story 90000 of 287227; 31.33 percent done

Writing story 91000 of 287227; 31.68 percent done

Writing story 92000 of 287227; 32.03 percent done

Writing story 93000 of 287227; 32.38 percent done

Writing story 94000 of 287227; 32.73 percent done

Writing story 95000 of 287227; 33.07 percent done

Writing story 96000 of 287227; 33.42 percent done

Writing story 97000 of 287227; 33.77 percent done

Writing story 98000 of 287227; 34.12 percent done

Writing story 99000 of 287227; 34.47 percent done

Writing story 100000 of 287227; 34.82 percent done

Writing story 101000 of 287227; 35.16 percent done

Writing story 102000 of 287227; 35.51 percent done

Writing story 103000 of 287227; 35.86 percent done

Writing story 104000 of 287227; 36.21 percent done

Writing story 105000 of 287227; 36.56 percent done

Writing story 106000 of 287227; 36.90 percent done

Writing story 107000 of 287227; 37.25 percent done

Writing story 108000 of 287227; 37.60 percent done

Writing story 109000 of 287227; 37.95 percent done

Writing story 110000 of 287227; 38.30 percent done

Writing story 111000 of 287227; 38.65 percent done

Writing story 112000 of 287227; 38.99 percent done

Writing story 113000 of 287227; 39.34 percent done

Writing story 114000 of 287227; 39.69 percent done

Writing story 115000 of 287227; 40.04 percent done

Writing story 116000 of 287227; 40.39 percent done

Writing story 117000 of 287227; 40.73 percent done

Writing story 118000 of 287227; 41.08 percent done

Writing story 119000 of 287227; 41.43 percent done

Writing story 120000 of 287227; 41.78 percent done

Writing story 121000 of 287227; 42.13 percent done

Writing story 122000 of 287227; 42.48 percent done

Writing story 123000 of 287227; 42.82 percent done

Writing story 124000 of 287227; 43.17 percent done

Writing story 125000 of 287227; 43.52 percent done

Writing story 126000 of 287227; 43.87 percent done

Writing story 127000 of 287227; 44.22 percent done

Writing story 128000 of 287227; 44.56 percent done

Writing story 129000 of 287227; 44.91 percent done

Writing story 130000 of 287227; 45.26 percent done

Writing story 131000 of 287227; 45.61 percent done

Writing story 132000 of 287227; 45.96 percent done

Writing story 133000 of 287227; 46.30 percent done

Writing story 134000 of 287227; 46.65 percent done

Writing story 135000 of 287227; 47.00 percent done

Writing story 136000 of 287227; 47.35 percent done

Writing story 137000 of 287227; 47.70 percent done

Writing story 138000 of 287227; 48.05 percent done

Writing story 139000 of 287227; 48.39 percent done

Writing story 140000 of 287227; 48.74 percent done

Writing story 141000 of 287227; 49.09 percent done

Writing story 142000 of 287227; 49.44 percent done

Writing story 143000 of 287227; 49.79 percent done

Writing story 144000 of 287227; 50.13 percent done

Writing story 145000 of 287227; 50.48 percent done

Writing story 146000 of 287227; 50.83 percent done

Writing story 147000 of 287227; 51.18 percent done

Writing story 148000 of 287227; 51.53 percent done

Writing story 149000 of 287227; 51.88 percent done

Writing story 150000 of 287227; 52.22 percent done

Writing story 151000 of 287227; 52.57 percent done

Writing story 152000 of 287227; 52.92 percent done

Writing story 153000 of 287227; 53.27 percent done

Writing story 154000 of 287227; 53.62 percent done

Writing story 155000 of 287227; 53.96 percent done

Writing story 156000 of 287227; 54.31 percent done

Writing story 157000 of 287227; 54.66 percent done

Writing story 158000 of 287227; 55.01 percent done

Writing story 159000 of 287227; 55.36 percent done

Writing story 160000 of 287227; 55.71 percent done

Writing story 161000 of 287227; 56.05 percent done

Writing story 162000 of 287227; 56.40 percent done

Writing story 163000 of 287227; 56.75 percent done

Writing story 164000 of 287227; 57.10 percent done

Writing story 165000 of 287227; 57.45 percent done

Writing story 166000 of 287227; 57.79 percent done

Writing story 167000 of 287227; 58.14 percent done

Writing story 168000 of 287227; 58.49 percent done

Writing story 169000 of 287227; 58.84 percent done

Writing story 170000 of 287227; 59.19 percent done

Writing story 171000 of 287227; 59.53 percent done

Writing story 172000 of 287227; 59.88 percent done

Writing story 173000 of 287227; 60.23 percent done

Writing story 174000 of 287227; 60.58 percent done

Writing story 175000 of 287227; 60.93 percent done

Writing story 176000 of 287227; 61.28 percent done

Writing story 177000 of 287227; 61.62 percent done

Writing story 178000 of 287227; 61.97 percent done

Writing story 179000 of 287227; 62.32 percent done

Writing story 180000 of 287227; 62.67 percent done

Writing story 181000 of 287227; 63.02 percent done

Writing story 182000 of 287227; 63.36 percent done

Writing story 183000 of 287227; 63.71 percent done

Writing story 184000 of 287227; 64.06 percent done

Writing story 185000 of 287227; 64.41 percent done

Writing story 186000 of 287227; 64.76 percent done

Writing story 187000 of 287227; 65.11 percent done

Writing story 188000 of 287227; 65.45 percent done

Writing story 189000 of 287227; 65.80 percent done

Writing story 190000 of 287227; 66.15 percent done

Writing story 191000 of 287227; 66.50 percent done

Writing story 192000 of 287227; 66.85 percent done

Writing story 193000 of 287227; 67.19 percent done

Writing story 194000 of 287227; 67.54 percent done

Writing story 195000 of 287227; 67.89 percent done

Writing story 196000 of 287227; 68.24 percent done

Writing story 197000 of 287227; 68.59 percent done

Writing story 198000 of 287227; 68.94 percent done

Writing story 199000 of 287227; 69.28 percent done

Writing story 200000 of 287227; 69.63 percent done

Writing story 201000 of 287227; 69.98 percent done

Writing story 202000 of 287227; 70.33 percent done

Writing story 203000 of 287227; 70.68 percent done

Writing story 204000 of 287227; 71.02 percent done

Writing story 205000 of 287227; 71.37 percent done

Writing story 206000 of 287227; 71.72 percent done

Writing story 207000 of 287227; 72.07 percent done

Writing story 208000 of 287227; 72.42 percent done

Writing story 209000 of 287227; 72.76 percent done

Writing story 210000 of 287227; 73.11 percent done

Writing story 211000 of 287227; 73.46 percent done

Writing story 212000 of 287227; 73.81 percent done

Writing story 213000 of 287227; 74.16 percent done

Writing story 214000 of 287227; 74.51 percent done

Writing story 215000 of 287227; 74.85 percent done

Writing story 216000 of 287227; 75.20 percent done

Writing story 217000 of 287227; 75.55 percent done

Writing story 218000 of 287227; 75.90 percent done

Writing story 219000 of 287227; 76.25 percent done

Writing story 220000 of 287227; 76.59 percent done

Writing story 221000 of 287227; 76.94 percent done

Writing story 222000 of 287227; 77.29 percent done

Writing story 223000 of 287227; 77.64 percent done

Writing story 224000 of 287227; 77.99 percent done

Writing story 225000 of 287227; 78.34 percent done

Writing story 226000 of 287227; 78.68 percent done

Writing story 227000 of 287227; 79.03 percent done

Writing story 228000 of 287227; 79.38 percent done

Writing story 229000 of 287227; 79.73 percent done

Writing story 230000 of 287227; 80.08 percent done

Writing story 231000 of 287227; 80.42 percent done

Writing story 232000 of 287227; 80.77 percent done

Writing story 233000 of 287227; 81.12 percent done

Writing story 234000 of 287227; 81.47 percent done

Writing story 235000 of 287227; 81.82 percent done

Writing story 236000 of 287227; 82.16 percent done

Writing story 237000 of 287227; 82.51 percent done

Writing story 238000 of 287227; 82.86 percent done

Writing story 239000 of 287227; 83.21 percent done

Writing story 240000 of 287227; 83.56 percent done

Writing story 241000 of 287227; 83.91 percent done

Writing story 242000 of 287227; 84.25 percent done

Writing story 243000 of 287227; 84.60 percent done

Writing story 244000 of 287227; 84.95 percent done

Writing story 245000 of 287227; 85.30 percent done

Writing story 246000 of 287227; 85.65 percent done

Writing story 247000 of 287227; 85.99 percent done

Writing story 248000 of 287227; 86.34 percent done

Writing story 249000 of 287227; 86.69 percent done

Writing story 250000 of 287227; 87.04 percent done

Writing story 251000 of 287227; 87.39 percent done

Writing story 252000 of 287227; 87.74 percent done

Writing story 253000 of 287227; 88.08 percent done

Writing story 254000 of 287227; 88.43 percent done

Writing story 255000 of 287227; 88.78 percent done

Writing story 256000 of 287227; 89.13 percent done

Writing story 257000 of 287227; 89.48 percent done

Writing story 258000 of 287227; 89.82 percent done

Writing story 259000 of 287227; 90.17 percent done

Writing story 260000 of 287227; 90.52 percent done

Writing story 261000 of 287227; 90.87 percent done

Writing story 262000 of 287227; 91.22 percent done

Writing story 263000 of 287227; 91.57 percent done

Writing story 264000 of 287227; 91.91 percent done

Writing story 265000 of 287227; 92.26 percent done

Writing story 266000 of 287227; 92.61 percent done

Writing story 267000 of 287227; 92.96 percent done

Writing story 268000 of 287227; 93.31 percent done

Writing story 269000 of 287227; 93.65 percent done

Writing story 270000 of 287227; 94.00 percent done

Writing story 271000 of 287227; 94.35 percent done

Writing story 272000 of 287227; 94.70 percent done

Writing story 273000 of 287227; 95.05 percent done

Writing story 274000 of 287227; 95.39 percent done

Writing story 275000 of 287227; 95.74 percent done

Writing story 276000 of 287227; 96.09 percent done

Writing story 277000 of 287227; 96.44 percent done

Writing story 278000 of 287227; 96.79 percent done

Writing story 279000 of 287227; 97.14 percent done

Writing story 280000 of 287227; 97.48 percent done

Writing story 281000 of 287227; 97.83 percent done

Writing story 282000 of 287227; 98.18 percent done

Writing story 283000 of 287227; 98.53 percent done

Writing story 284000 of 287227; 98.88 percent done

Writing story 285000 of 287227; 99.22 percent done

Writing story 286000 of 287227; 99.57 percent done

Writing story 287000 of 287227; 99.92 percent done

Finished writing file finished_files/train.tar

Writing vocab file...

Finished writing vocab file

运行make_datafiles的过程的更多相关文章

  1. YARN(MapReduce 2)运行MapReduce的过程-源码分析

    这是我的分析,当然查阅书籍和网络.如有什么不对的,请各位批评指正.以下的类有的并不完全,只列出重要的方法. 如要转载,请注上作者以及出处. 一.源码阅读环境 需要安装jdk1.7.0版本及其以上版本, ...

  2. Windows7下的Java运行环境搭建过程图解

    第一步:下载JDK 地址:http://www.oracle.com/technetwork/java/javase/downloads/index-jsp-138363.html,(由于Sun于20 ...

  3. Qt入门之基础篇 ( 二 ) ：Qt项目建立、编译、运行和发布过程解析

    转载请注明出处:CN_Simo. 题解: 本篇内容主讲Qt应用从创建到发布的整个过程,旨在帮助读者能够快速走进Qt的世界. 本来计划是讲解Qt源码静态编译,如此的话读者可能并不能清楚地知道为何要静态编 ...

  4. JVM运行和类加载过程

    JAVA的JVM的内存可分为3个区:堆(heap).栈(stack)和方法区(method) (该知识点引用 http://www.cnblogs.com/dingyingsi/p/3760730.h ...

  5. MapReduce运行原理和过程

    原文 一．Map的原理和运行流程 Map的输入数据源是多种多样的,我们使用hdfs作为数据源.文件在hdfs上是以block(块,Hdfs上的存储单元)为单位进行存储的. 1.分片 我们将这一个个bl ...

  6. docker-machine create -d generic 运行的波折过程及遇见的问题

    这是一个愚蠢的学习过程,但是因为觉得过程还是值得记录的,还是写了下来 2>driver = generic 1)在这个过程中使用的都是本地的mac系统,然后尝试在mac本地create -d g ...

  7. 【原创】MapReduce运行原理和过程

    一．Map的原理和运行流程 Map的输入数据源是多种多样的,我们使用hdfs作为数据源.文件在hdfs上是以block(块,Hdfs上的存储单元)为单位进行存储的. 1.分片 我们将这一个个block ...

  8. C++源文件到可运行文件的过程

    一.四个步骤    对于C/C++编写的程序,从源码到可运行文件,一般经过以下四个步骤: 1).预处理,产生.ii文件 2).编译,产生汇编文件(.s文件) 3).汇编,产生目标文件(.o或.obj文 ...

  9. 记录从裸机到TensorFlow GPU版运行 的配置过程

    实验室原来有一台装Ubuntu Server系统的服务器,安装有tensorflow,在使用过程中经常出现断网.死机.自动关机等毛病,忍无可忍,决定重装系统 配置如下:Dell工作站,Xeon-E5 ...

随机推荐

  1. 神经网络3_M-P模型

    sklearn实战-乳腺癌细胞数据挖掘(博客主亲自录制视频教程,QQ:231469242) https://study.163.com/course/introduction.htm?courseId ...

  2. CMDB资产管理系统开发【day26】:admin action

    本节目标 审核写到数据库,我就单独写一个如下的 页面 单机go后就跳转到如下图界面,我们这节课的目标就是写一个这样的页面 asset\admin.py部分代码 注释如下: class NewAsset ...

  3. 分布式监控系统开发【day38】:报警阈值程序逻辑解析（三）

    一.需求讨论 1.请问如何解决延迟问题 1000台机器,每1分钟循环一次但是刚好第一次循环第一秒刚处理完了,结果还没等到第二分钟又出问题,你那必须等到第二次循环,假如我这个服务很重要必须实时知道,每次 ...

  4. DirectX11 With Windows SDK--15 几何着色器初探

    前言 从这一部分开始,感觉就像是踏入了无人深空一样,在之前初学DX11的时候,这部分内容都是基本上跳过的,现在打算重新认真地把它给拾回来. DirectX11 With Windows SDK完整目录 ...

  5. css中font-size为0的妙用（消除内联元素间的间隔）

    前言 <div> <input type="text"> <input type="button" value="提交& ...

  6. A Basic Course in Partial Differential Equations

    A Basic Course in Partial Differential Equations, Qing Han, 2011 [下载说明:点击链接,等待5秒, 点击右上角的跳过广告后调至下载页面, ...

  7. react中键盘enter事件处理

    对于常见的搜索需求业务场景,用户输入完成后,点击enter事件请求数据,要求不提交页面,实现数据局部更新,这需要用到react中的表单Forms. 处理方法: (1)html书写 form标签中去掉a ...

  8. go语言学习 一

    1.变量声明 指定变量类型,声明后若不赋值,使用默认值 根据值自行判定变量类型. 省略var, 注意 :=左侧的变量不应该是已经声明过的,否则会导致编译错误 2.go语言作用域 函数内定义的变量称为局 ...

  9. L2-001 紧急救援 （25 分) （最短路+路径打印）

    链接:https://pintia.cn/problem-sets/994805046380707840/problems/994805073643683840 题目: 作为一个城市的应急救援队伍的负 ...

  10. 帆软报表（finereport）参数组合/下拉参数联动

    首先实现下拉参数联动 1.参数的定义→添加控件绑定数据→参数过滤和参数面板样式 下拉复选框控件注意事项:select * from tablewhere 1=1${if(len(控件名称)=0,&qu ...