假如现有数据：

{ "@timestamp": "2018-10-13T21:55:58+08:00", "remote_addr": "100.120.34.3", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.3" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
{ "@timestamp": "2018-10-13T21:56:06+08:00", "remote_addr": "100.120.34.101", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.101" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
{ "@timestamp": "2018-10-13T21:56:08+08:00", "remote_addr": "100.120.34.29", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.075, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.29" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.075 }
{ "@timestamp": "2018-10-13T21:56:10+08:00", "remote_addr": "100.120.34.75", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D166459D-E823-4847-9094-6F4BF90625B2&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.078, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.75" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.078 }
{ "@timestamp": "2018-10-13T21:56:18+08:00", "remote_addr": "100.120.34.39", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=08C65C3B-EED2-4A65-B0C1-67FC7FB78E18&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.082, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.39" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.082 }
{ "@timestamp": "2018-10-13T21:56:31+08:00", "remote_addr": "100.120.34.68", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.079, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.68" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.079 }

临时保存为tmp.log

awk -F 'idfa=' '{print $2}' tmp.log

出现如下结果：

58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.3" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.101" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.075, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.29" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.075 }

执行

awk -F 'idfa=' '{print $2}' tmp.log | awk -F '&source=' '{print $1}'

出现如下结果：
58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

执行

awk -F 'idfa=' '{print $2}' tmp.log | awk -F '&source=' '{print $1}'  | sort  (这一步将结果相同的放在一起)

出现如下结果：
58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

执行

awk -F 'idfa=' '{print $2}' cms_\(2\).log | awk -F '&source=' '{print $1}' |sort| uniq -c
出现最终结果(次数 和 各字符串)：
1 58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
1 E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
1 D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

centos下对文件某些特定字符串分组统计出现次数的更多相关文章

  1. centos下修改文件后如何保存退出

    centos下修改文件后如何保存退出 保存命令 按ESC键 跳到命令模式,然后: :w 保存文件但不退出vi :w file 将修改另外保存到file中,不退出vi :w! 强制保存,不推出vi :w ...

  2. 在文件夹下所有文件中查找字符串(linux/windows)

    在linux下可以用 grep "String" filename.txt#字符串 文件名grep -r "String" /home/#递归查找目录下所有文件 ...

  3. MyEclipse如何查找指定工程下所有或指定文件中特定字符串并且可进行批量替换

    查找操作步骤:(1)在myEclipse里菜单选择-Search-Search(快捷键:ctrl+h);(2)在弹出对话框中选File Search选项,然后在第一个文本框中输入“要查找的字符串”(为 ...

  4. CentOS下查看文件和文件夹大小

    当磁盘大小超过标准时会有报警提示,这时如果掌握df和du命令是非常明智的选择. df可以查看一级文件夹大小.使用比例.档案系统及其挂入点,但对文件却无能为力. 当磁盘大小超过标准时会有报警提示,这时如 ...

  5. Centos下删除文件后空间并未释放

    [root@DeviceSP /]# df -h Filesystem Size Used Avail Use% Mounted on /dev/vda3 90G 82G 3.8G 96% / tmp ...

  6. C#替换文件中特定字符串，按照原来的编码格式保存

    private void button1_Click(object sender, EventArgs e) { var txt1 = "E:\\Temp\\local"; str ...

  7. IO复制多级目录 控制台输入文件目录然后把目录下java文件复制到 D: 并统计java个数

    package cn.itcast_05; import java.io.BufferedInputStream; import java.io.BufferedOutputStream; impor ...

  8. 【转】【Linux】Linux下统计当前文件夹下的文件个数、目录个数

    [转][Linux]Linux下统计当前文件夹下的文件个数.目录个数 统计当前文件夹下文件的个数,包括子文件夹里的 ls -lR|grep "^-"|wc -l 统计文件夹下目录的 ...

  9. DataTable、List使用groupby进行分组和分组统计；List、DataTable查询筛选方法

    DataTable分组统计: .用两层循环计算,前提条件是数据已经按分组的列排好序的. DataTable dt = new DataTable(); dt.Columns.AddRange(new ...

随机推荐

  1. Katalon Studio简单使用(一)

    官网地址:https://www.katalon.com/ katalon 目前有两种产品 一个是studio 另外一个是katalon analytics,此处先来学习studio部分. 文章学习内 ...

  2. Linux CentOS环境下.Net Core SDK安装

    1.安装.Net SDK 在安装.NET之前,需要注册产品存储库并安装所需的依赖关系. 打开Linux命令提示符并运行以下命令: sudo rpm -Uvh https://packages.micr ...

  3. Http协议中关于Content-Length的解读【出现坑爹的视频中断】

    最近公司的视频设备在下载视频的时候,出现了很诡异的现象,在新旧服务器一样的tpp包,下载下来后,新服务器无法解析,旧服务器没问题.且tpp包并没有改动. 后面找了挺久,终于发现了视频下载的时候是断点续 ...

  4. 446. Arithmetic Slices II - Subsequence

    A sequence of numbers is called arithmetic if it consists of at least three elements and if the diff ...

  5. C#上传文件处理

    public class FileHelp : System.Web.UI.Page { /// <summary> /// 保存文件 /// 返回:数组:status[0]:文件名称:s ...

  6. jquery屏蔽掉键盘enter提交 onkeydown

    屏蔽掉enter提交onkeydown onkeydown="if(event.keyCode==13){return false;}"

  7. flask源码解析之DispatcherMiddleware

    DispatcherMiddleware作用 实现多app的应用,完成路由分发的功能 如何使用 from werkzeug.wsgi import DispatcherMiddleware from ...

  8. Usboot V1.68版本

    V1.68版本,我的收藏之一 官方的介绍: 市面上现在大多数U盘都支持启动机器的功能,但是要制作启动型U盘,需要进入WIN98,现在很多人机器 上都没有98了吧,呵呵.为了做个启动盘,装一个98,多冤 ...

  9. jquery源码解析：val方法和valHooks对象详解

    这一课,我们将讲解val方法,以及对value属性的兼容性处理,jQuery中通过valHooks对象来处理. 首先,我们先来看下val方法的使用: $("#input1").va ...

  10. [转] 打开 CMD 时自动执行命令

    [转] 打开 CMD 时自动执行命令 问题描述 在Windows中打开一个command-prompt时,我正在寻找一种方法来执行一些控制台命令,特别是设置一些命令别名. 例如,当打开command- ...