grep每次读取多大的文本

Most of the tools do not actually read a single line in from a file at a time, rather they use a buffer in memory to store chunks of lines. The tools operate a line at a time on the data in this buffer.

NOTE: By "line" I mean split by a \n, in grep's case, or whatever character is denoted as the "delimiter" when the "tool" is invoked.

Here's a example to illustrate this effect.

Sample data

Create a file with 100,000 lines in it. The file is called afile100k.

$ for i in $(seq 100000);do echo "file$i" >> afile100k; done

Using strace

We can utilize strace to peak inside a running process, in this case the grep command.

$ strace -s 2000 -o log100k grep 5 afile100k

This will log the output from strace, up to 2000 characters per line of output to the file log100k. The command we'll be tracing is grep 5 afile100k.

Here's some key output from the log:

ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff8bf73b20) = -1 ENOTTY (Inappropriate ioctl for device)

read(3, "file1\nfile2\nfile3\nfile4\nfile5\nfile6\nfile7\nfile8\nfile9\nfile10\nfile11\nfile12\nfile13\nfile14\nfile15\nfile16\nfile17\nfile18\nfile19\nfile20\nfile21\nfile22\nfile23\nfile24\nfile25\nfile26\nfile27\nfile28\nfile29\nfile30\nfile31\nfile32\nfile33\nfile34\nfile35\nfile36\nfile37\nfile38\nfile39\nfile40\nfile41\nfile42\nfile43\nfile44\nfile45\nfile46\nfile47\nfile48\nfile49\nfile50\nfile51\nfile52\nfile53\nfile54\nfile55\nfile56\nfile57\nfile58\nfile59\nfile60\nfile61\nfile62\nfile63\nfile64\nfile65\nfile66\nfile67\nfile68\nfile69\nfile70\nfile71\nfile72\nfile73\nfile74\nfile75\nfile76\nfile77\nfile78\nfile79\nfile80\nfile81\nfile82\nfile83\nfile84\nfile85\nfile86\nfile87\nfile88\nfile89\nfile90\nfile91\nfile92\nfile93\nfile94\nfile95\nfile96\nfile97\nfile98\nfile99\nfile100\nfile101\nfile102\nfile103\nfile104\nfile105\nfile106\nfile107\nfile108\nfile109\nfile110\nfile111\nfile112\nfile113\nfile114\nfile115\nfile116\nfile117\nfile118\nfile119\nfile120\nfile121\nfile122\nfile123\nfile124\nfile125\nfile126\nfile127\nfile128\nfile129\nfile130\nfile131\nfile132\nfile133\nfile134\nfile135\nfile136\nfile137\nfile138\nfile139\nfile140\nfile141\nfile142\nfile143\nfile144\nfile145\nfile146\nfile147\nfile148\nfile149\nfile150\nfile151\nfile152\nfile153\nfile154\nfile155\nfile156\nfile157\nfile158\nfile159\nfile160\nfile161\nfile162\nfile163\nfile164\nfile165\nfile166\nfile167\nfile168\nfile169\nfile170\nfile171\nfile172\nfile173\nfile174\nfile175\nfile176\nfile177\nfile178\nfile179\nfile180\nfile181\nfile182\nfile183\nfile184\nfile185\nfile186\nfile187\nfile188\nfile189\nfile190\nfile191\nfile192\nfile193\nfile194\nfile195\nfile196\nfile197\nfile198\nfile199\nfile200\nfile201\nfile202\nfile203\nfile204\nfile205\nfile206\nfile207\nfile208\nfile209\nfile210\nfile211\nfile212\nfile213\nfile214\nfile215\nfile216\nfile217\nfile218\nfile219\nfile220\nfile221\nfile222\nfile223\nfile224\nfile225\nfile226\nfile227\nfile228\nfile229\nfile230\nfile231\nfile232\nfile233\nfile234\nfile235\nfile236\nfile237\nfile238\nfile239\nfile240\nfile241\nfile242\nfile243\nfile244\nfile245\nfile246\nfile247\nfile248\nfile249\nfile250\nfile251\nfile252\nfile253\nfile254\nfile255\nfile256\nfile257\nfile258\nfile259\nfile260\nfile261\nfile262\nfile263\nfile"..., 32768) = 32768

lseek(3, 32768, SEEK_HOLE)              = 988895
lseek(3, 32768, SEEK_SET)               = 32768

Notice grep is reading in 32k (32768) at a time. NOTE: I've tried to break up the log a bit so that it's easier to read here on SE.

Now it starts writing out results:

fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fcafcfff000
write(1, "file5\n", 6)                  = 6
write(1, "file15\n", 7)                 = 7
write(1, "file25\n", 7)                 = 7
write(1, "file35\n", 7)                 = 7
write(1, "file45\n", 7)                 = 7
write(1, "file50\n", 7)                 = 7

After exhausting out the contents of this buffer it will re-read the next 32k (32768) chunk from the file.

write(1, "file3759\n", 9)               = 9
read(3, "\nfile3765\nfile3766\nfile3767\nfile3768\nfile3769\nfile3770\nfile3771\nfile3772\nfile3773....\nfile3986\nf"..., 32768) = 32768
Followed by more writes:

write(1, "file3765\n", 9)               = 9
write(1, "file3775\n", 9)               = 9

Grep continues to do this until it's completely exhausted the contents of the file, at which point it ends.

write(1, "file99995\n", 10)             = 10
read(3, "", 24576)                      = 0
close(3)                                = 0
close(1)                                = 0
munmap(0x7fcafcfff000, 4096)            = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

工作中遇到grep -w比较慢，从而strace -p跟踪进程过程，google了下比较通俗易懂，就不翻译了，记录下

原文link