【439】Tweets processing by Python
参数说明:
- coordinates:Represents the geographic location of this Tweet as reported by the user or client application. The inner coordinates array is formatted as geoJSON (longitude first, then latitude).
1.文本文件转 json 格式
读取 txt 文件中的 tweets 文本,将其转为 json 格式,可以打印输出,也可以提取详细信息
代码:
import json
import os folderpath = r"D:\Twitter Data\Data\test"
files = os.listdir(folderpath)
os.chdir(folderpath) # get the first txt file
tweets_data_path = files[0] # store json format file in this array
tweets_data = []
tweets_file = open(tweets_data_path, "r")
for line in tweets_file:
try:
tweet = json.loads(line)
tweets_data.append(tweet)
except:
continue
# print json format file with indentation
print(json.dumps(tweets_data[0], indent=4))
输出:
{
"created_at": "Tue Jun 25 20:44:34 +0000 2019",
"id": 1143621025550049280,
"id_str": "1143621025550049280",
"text": "Australia beat the Poms overnight \ud83d\ude01\ud83c\udfcf\ud83c\udde6\ud83c\uddfa\ud83c\udff4\udb40\udc67\udb40\udc62\udb40\udc65\udb40\udc6e\udb40\udc67\udb40\udc7f #AUSvENG #CmonAussie #CWC19",
"source": "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>",
"truncated": false,
"in_reply_to_status_id": null,
"in_reply_to_status_id_str": null,
"in_reply_to_user_id": null,
"in_reply_to_user_id_str": null,
"in_reply_to_screen_name": null,
"user": {
"id": 252426781,
"id_str": "252426781",
"name": "Willy Aitch",
"screen_name": "WillyAitch",
"location": "Melbourne, Victoria",
"url": null,
"description": "September 2017 to February 2018, was the greatest 5 months ever. Richmond \ud83d\udc2f\ud83d\udc2f\ud83d\udc2fwon the 2017 AFL Premiership! Philadelphia Eagles \ud83e\udd85\ud83e\udd85\ud83e\udd85 won Super Bowl LII",
"translator_type": "none",
"protected": false,
"verified": false,
"followers_count": 417,
"friends_count": 1061,
"listed_count": 15,
"favourites_count": 18852,
"statuses_count": 17796,
"created_at": "Tue Feb 15 04:55:59 +0000 2011",
"utc_offset": null,
"time_zone": null,
"geo_enabled": true,
"lang": null,
"contributors_enabled": false,
"is_translator": false,
"profile_background_color": "C0DEED",
"profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png",
"profile_background_tile": false,
"profile_link_color": "1DA1F2",
"profile_sidebar_border_color": "C0DEED",
"profile_sidebar_fill_color": "DDEEF6",
"profile_text_color": "333333",
"profile_use_background_image": true,
"profile_image_url": "http://pbs.twimg.com/profile_images/1112669591342211072/rnbV0dCK_normal.jpg",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1112669591342211072/rnbV0dCK_normal.jpg",
"profile_banner_url": "https://pbs.twimg.com/profile_banners/252426781/1522377977",
"default_profile": true,
"default_profile_image": false,
"following": null,
"follow_request_sent": null,
"notifications": null
},
"geo": null,
"coordinates": null,
"place": {
"id": "01864a8a64df9dc4",
"url": "https://api.twitter.com/1.1/geo/id/01864a8a64df9dc4.json",
"place_type": "city",
"name": "Melbourne",
"full_name": "Melbourne, Victoria",
"country_code": "AU",
"country": "Australia",
"bounding_box": {
"type": "Polygon",
"coordinates": [
[
[
144.593742,
-38.433859
],
[
144.593742,
-37.511274
],
[
145.512529,
-37.511274
],
[
145.512529,
-38.433859
]
]
]
},
"attributes": {}
},
"contributors": null,
"is_quote_status": false,
"quote_count": 0,
"reply_count": 0,
"retweet_count": 0,
"favorite_count": 0,
"entities": {
"hashtags": [
{
"text": "AUSvENG",
"indices": [
46,
54
]
},
{
"text": "CmonAussie",
"indices": [
55,
66
]
},
{
"text": "CWC19",
"indices": [
67,
73
]
}
],
"urls": [],
"user_mentions": [],
"symbols": []
},
"favorited": false,
"retweeted": false,
"filter_level": "low",
"lang": "en",
"timestamp_ms": "1561495474599"
}
2. 读取关键字内容
通过 .keys() 获取所有的键值
代码:
import json
import os folderpath = r"D:\Twitter Data\Data\test"
files = os.listdir(folderpath)
os.chdir(folderpath) # get the first txt file
tweets_data_path = files[0] # store json format file in this array
tweets_data = []
tweets_file = open(tweets_data_path, "r")
for line in tweets_file:
try:
tweet = json.loads(line)
tweets_data.append(tweet)
except:
continue for k in tweets_data[0].keys():
print(k)
输出:
created_at
id
id_str
text
source
truncated
in_reply_to_status_id
in_reply_to_status_id_str
in_reply_to_user_id
in_reply_to_user_id_str
in_reply_to_screen_name
user
geo
coordinates
place
contributors
is_quote_status
quote_count
reply_count
retweet_count
favorite_count
entities
favorited
retweeted
filter_level
lang
timestamp_ms
3. 输出键值信息
代码:
import json
import os folderpath = r"D:\Twitter Data\Data\test"
files = os.listdir(folderpath)
os.chdir(folderpath) # get the first txt file
tweets_data_path = files[0] # store json format file in this array
tweets_data = []
tweets_file = open(tweets_data_path, "r")
for line in tweets_file:
try:
tweet = json.loads(line)
tweets_data.append(tweet)
except:
continue for k in tweets_data[0].keys():
print(k, ":", tweets_data[0][k])
print()
输出:
created_at : Tue Jun 25 20:44:34 +0000 2019 id : 1143621025550049280 id_str : 1143621025550049280 text : Australia beat the Poms overnight【439】Tweets processing by Python的更多相关文章
- 【转】实习小记-python 内置函数__eq__函数引发的探索
[转]实习小记-python 内置函数__eq__函数引发的探索 乱写__eq__会发生啥?请看代码.. >>> class A: ... def __eq__(self, othe ...
- 【转】实习小记-python中可哈希对象是个啥?what is hashable object in python?
[转]实习小记-python中可哈希对象是个啥?what is hashable object in python? 废话不多说直接祭上python3.3x的文档:(原文链接) object.__ha ...
- 【6】TensorFlow光速入门-python模型转换为tfjs模型并使用
本文地址:https://www.cnblogs.com/tujia/p/13862365.html 系列文章: [0]TensorFlow光速入门-序 [1]TensorFlow光速入门-tenso ...
- 【Linux】CentOS下升级Python和Pip版本全自动化py脚本
[Linux]CentOS下升级Python和Pip版本全自动化py脚本 CentOS7.6自带py2.7和py3.6 想要安装其它版本的话就要自己重新下载和编译py其它版本并且配置环境,主要是软链接 ...
- 【原】使用Json作为Python和C#混合编程时对象转换的中间文件
一.Python中自定义类对象json字符串化的步骤[1] 1. 用 json 或者simplejson 就可以: 2.定义转换函数: 3. 定义类 4. 生成对象 5.dumps执行,引入转换函 ...
- 【解决方案】django初始化执行python manage.py migrate命令后,除default数据库之外的其他数据库中的表没有创建出来
[问题原因]:django工程中存在多个应用,每个应用都指定了对应的数据库.执行python manage.py migrate命令时没有指定数据库,将只初始化默认的default数据库. [解决方案 ...
- 【转】利用Psyco提升Python运行速度
转自:http://www.leeon.me/a/use-Psyco-to-improve-Python-speed Psyco 是严格地在 Python 运行时进行操作的.也就是说,Python 源 ...
- 【转】你真的理解Python中MRO算法吗?
你真的理解Python中MRO算法吗? MRO(Method Resolution Order):方法解析顺序. Python语言包含了很多优秀的特性,其中多重继承就是其中之一,但是多重继承会引发很多 ...
- 【算法】八皇后问题 Python实现
[八皇后问题] 问题: 国际象棋棋盘是8 * 8的方格,每个方格里放一个棋子.皇后这种棋子可以攻击同一行或者同一列或者斜线(左上左下右上右下四个方向)上的棋子.在一个棋盘上如果要放八个皇后,使得她们互 ...
随机推荐
- C++面向程序设计(第二版)课后习题答案解析
最近没什么心情整理零散的知识点,就整理一下第四章的课后习题答案. 1.定义一个复数类Complex,重载运算符“+”,使之能用于复数的加法运算.将运算符函数重载为非成员函数,非友元的普通函数.编程序, ...
- OAuth 2.0攻击
参考文章:https://www.yuque.com/pmiaowu/web_security_1/oauth 作者:PHPoop 关于OAuth2.0协议的授权流程可以参考下面的流程图: 1.Cli ...
- HTML 009 select
本篇文章并非描述HTML中的select标签, 而是描述JSP中的<s:select> 关于HTML中的select标签, 以及和JSP中的<s:select>的相同以及差异后 ...
- learning java Runtime类中的exec
var rt = Runtime.getRuntime(); // 类c语言当中的system()函数. rt.exec("notepad.exe");
- mysqli扩展有一系列的优势,相对于mysql扩展的提升主要优势有哪些?
mysqli扩展有一系列的优势,相对于mysql扩展的提升主要优势有哪些? 面向对象接口 prepared语句支持(译注:关于prepare请参阅mysql相关文档) 多语句执行支持 事务支持 增强的 ...
- 影像优化 OptimizeRaster工具包介绍
Esri OptimizeRasters是一个高效.可配置的开源工具包. OptimizeRasters提供了以下功能: 影像格式转换和压缩.支持输出优化栅格格式:MRF.分块TIFF.云存储优化Ge ...
- Comet OJ - Contest #9 & X Round 3题解
传送门 \(A\) 咕咕 typedef long long ll; int a1,a2,n,d;ll res; int main(){ scanf("%d%d%d",&a ...
- mac eclipse 创建Java 工程
首先创建Java工程testjavapro,创建包testjavapro,接着创建类testjava 参考: https://www.jianshu.com/p/20280b850c95
- kafka集群搭建(图文并用)
将安装包上传服务器并解压 scp kafka_2.11-1.0.0.tgz username@{ip}:~/. mkdir /usr/local/kafka mv kafka_2.11-1.0.0.t ...
- [面试]快来测测你的C++水平
在32位编译环境下进行测试. 以下代码运行结果是什么? #include <iostream> using namespace std; class D { public: static ...