pandas 之 datetime 初识
import numpy as np
import pandas as pd
认识
Time series data is an impotant from of data in many different fields, such as finance, economics, ecology, neuroscience(神经学) and physics. Anything that is observed or measured at many points in time forms a time series.
Many time series are fixed frequency , which is to say that data points occur at regular intervals according to some rule, such as every 15 seconds, every 5 minutes, or once per month.
Time series can also be irregular(不规则的) without a fixed unit of time or offset between units. How you mark and refer to time series data depends on the application, and you may have one of the following:
- Timestamps, specific instants in time (时间戳)
- Fixed periods, such as the month January 2007 or the full year 2010 (时期)
- Intervals of time, indicated by a start and end timestamp. Periods can be thought of as special cases of intervals. (时间间隔)
- Experiment or elapsed time(试验时间逝去); each timestamp is a measure of time relative to a particular start time (e.g. the diameter(直径) of a cookie baking each second since being palced in the oven)
In this chapter, I am mainly concerned with time series in the first three categories, though many of the teachniques can applied to experimental time series where the index may be an integer or floating-point number indicating elapsed time from the start of the experiment. The simplest and most widely used kind of time series are those indexed by timestamp.
pandas also supports indexes based on timedeltas, which can be a useful way of representing experiment or elapsed time. We do not explore timedelta indexes in this book , but you can learn more in the pandas documenttaion.
pandas provides many buit-in time series tools and data algorithims. You can efficiently work with very large time series and easily slice and dice, aggregate, and resample(重采样) irrgular-and fixed-frequency time series. Some of these tools are especially useful financial and economics applications, but you could certainly use them to analyze server log, too.
The Pyhton standard library includes data types for date and time data, as well as calendar-related(日历相关) functionality. The datetime, time, calendar modules are the main places to start. the datetime.datetime type, or simply datetime, is widely used.
from datetime import datetime
now = datetime.now()
now
datetime.datetime(2019, 4, 27, 15, 3, 14, 103616)
now.year, now.month, now.day, now.hour, now.minute
(2019, 4, 27, 15, 3)
datetime stores(存储) both the date and time down to the microsecond timedelta reprecents the temporal(临时的) difference between two datetime objects:
"cj 特方便, 在时间相加上"
delta = datetime(2011, 1, 7) - datetime(2008, 6, 24, 8, 15)
delta
'cj 特方便, 在时间相加上'
datetime.timedelta(926, 56700)
delta.days, delta.seconds
(926, 56700)
You can add (or subtract) a timedelata or multiple thereof to a datetime object to yield a new shifted object:
from datetime import timedelta
start = datetime(2011, 1, 7)
"加12天"
start + timedelta(12)
'加12天'
datetime.datetime(2011, 1, 19, 0, 0)
" 减去24天"
start - 2*timedelta(12)
' 减去24天'
datetime.datetime(2010, 12, 14, 0, 0)
Table 11-1 summarizes the data types in the datetime module. While this chapter is mainly concerned with the data types in pandas and high-level time series manupulation, you may encounter the datetime-based types in many other places in Pyhton in the wild.
Type | Description |
---|---|
date | Store calendar date (year, month, day) using the Gregorian calendar |
time | Store time of day as hours,minutes, seconds, and microseconds |
datetime | Store both date and time |
timedelta | Reprecents the difference between tow datetime values(as days,second..) |
tzinfo | Base type for storing time zone infomation |
String和Datetime间的转换
You can format datetime object and pandas Timestamp objects, which I'll introduce later, as strings using str or the strftime method, passing a format specification:
stamp = datetime(2011, 1, 3)
stamp
str(stamp)
datetime.datetime(2011, 1, 3, 0, 0)
'2011-01-03 00:00:00'
stamp.strftime('%Y-%m-%d') # 四位数字的年
'2011-01-03'
stamp.strftime('%y-%m-%d') # 2位数字的年
'11-01-03'
See Table 11-2 for a complete list of the format codes.
Type | Description |
---|---|
%Y | Four-digit year(4个数字的年) |
%y | Two-digit year |
%m | Two-dight month [01, 12] |
%d | Two-dight day [01, 31] |
%H | Hour(24-hour clock) [00, 23] |
%I | Hour(12-hour clock) [00, 12]) |
%M | Two-dight minute [00, 59] |
%S | Second [00, 61] (second 60, 61 acccount for leap second) |
%w | Weekday as integer[0(Sundday), 6] |
%U | |
%W | |
%z | UTC time zone offset as +HHMM or -HHMM; empty if time zone naive |
%F | Shortcut for %Y-%m-%d (eg. 2012-4-8) |
%D | Shortcut for %m/%d/%y (eg. 04/18/12) |
You can use these same format codes to convert strings to dates using date time.strptime:
value = "2011-01-03"
datetime.strptime(value, '%Y-%m-%d')
datetime.datetime(2011, 1, 3, 0, 0)
datestrs = ['7/6/2011', '8/6/2011']
[datetime.strptime(x, '%m/%d/%Y') for x in datestrs]
[datetime.datetime(2011, 7, 6, 0, 0), datetime.datetime(2011, 8, 6, 0, 0)]
Datetime.strptime is a good way to parse a date with a know format. However, it can be a bit annoying to have to write a format spec each time, especially for common date formats.In this case, you can use the parse.parse method in the third-party dateutil package (this is installed automatically when you install pandas).
from dateutil.parser import parse
parse("2011-01-03")
datetime.datetime(2011, 1, 3, 0, 0)
parse("2011/01/03")
datetime.datetime(2011, 1, 3, 0, 0)
dateutil si capable of parsing most human-intelligble date representation:
parse('Jan 31, 1997, 10:45 PM')
datetime.datetime(1997, 1, 31, 22, 45)
In international locales, day appering before month is very common, so you can pass dayfirst=True to indicate this:
parse('6/12/2011', dayfirst=True)
datetime.datetime(2011, 12, 6, 0, 0)
pandas is generally oriented toward working with arrays of dates, whether used an axis index or a column in a DataFrame. The to_datetime method parses many different kinds of date representations. Standard date formats like ISO 8601 can be parsed very quickly:
datestrs = ['2011-07-06 12:00:00', '2011-08-06 00:00:00']
pd.to_datetime(datestrs)
DatetimeIndex(['2011-07-06 12:00:00', '2011-08-06 00:00:00'], dtype='datetime64[ns]', freq=None)
It also handles values that should be condidered missing (None, empty string. etc.):
idx = pd.to_datetime(datestrs + [None])
idx
DatetimeIndex(['2011-07-06 12:00:00', '2011-08-06 00:00:00', 'NaT'], dtype='datetime64[ns]', freq=None)
idx[2]
NaT
pd.isnull(idx)
array([False, False, True])
NaT (Not a Time) is pandas's null value for timestamp data.
dateutil.parser is a useful but imperfect tool. Notably, it will recognize some strings as dates that you might prefer that it didn't for example. '42' will be parsed as the year 2042 with today's ccalendar date.
datetime objects also have a number of locale-specific formatting options for systems in other countries or languages. For example, the abbreviated(缩写) month names will be different on German or French systems compared with English systme. See Table 11-3 for a listing.
- %a Abbreviated weekday name
- %A Full weekday name
- %b 缩写月份的名字
- %B 全写月份
- %c Full date and time (eg. Tue 01 May 2012 04:20:57 PM)
- %p 包含AM or PM
- %x (eg. '05/01/2012')
- %X (eg. '04:24:12 PM')
pandas 之 datetime 初识的更多相关文章
- python之pandas学习笔记-初识pandas
初识pandas python最擅长的就是数据处理,而pandas则是python用于数据分析的最常用工具之一,所以学python一定要学pandas库的使用. pandas为python提供了高性能 ...
- Pandas 数据处理 | Datetime 在 Pandas 中的一些用法!
Datatime 是 Python 中一种时间数据类型,对于不同时间格式之间的转换是比较方便的,而在 Pandas 中也同样支持 DataTime 数据机制,可以借助它实现许多有用的功能,例如 1,函 ...
- Pandas 基础(1) - 初识及安装 yupyter
Hello, 大家好, 昨天说了我会再更新一个关于 Pandas 基础知识的教程, 这里就是啦......Pandas 被广泛应用于数据分析领域, 是一个很好的分析工具, 也是我们后面学习 machi ...
- 整理总结 python 中时间日期类数据处理与类型转换(含 pandas)
我自学 python 编程并付诸实战,迄今三个月. pandas可能是我最高频使用的库,基于它的易学.实用,我也非常建议朋友们去尝试它.--尤其当你本身不是程序员,但多少跟表格或数据打点交道时,pan ...
- (转) Using the latest advancements in AI to predict stock market movements
Using the latest advancements in AI to predict stock market movements 2019-01-13 21:31:18 This blog ...
- Python基础 | 日期时间操作
目录 获取时间 时间映射 格式转换 字符串转日期 日期转字符串 unixtime 时间计算 时间偏移 时间差 "日期时间数据"作为三大基础数据类型之一,在数据分析中会经常遇到. 本 ...
- pandas中将timestamp转为datetime
参考自:http://stackoverflow.com/questions/35312981/using-pandas-to-datetime-with-timestamps 在pandas Dat ...
- pandas处理时间序列(1):pd.Timestamp()、pd.Timedelta()、pd.datetime( )、 pd.Period()、pd.to_timestamp()、datetime.strftime()、pd.to_datetime( )、pd.to_period()
Pandas库是处理时间序列的利器,pandas有着强大的日期数据处理功能,可以按日期筛选数据.按日期显示数据.按日期统计数据. pandas的实际类型主要分为: timestamp(时间戳) ...
- pandas初识
pandas初识 1.生成DataFrame型的数据 import pandas as pd import numpy as np dates = pd.date_range('20130101',p ...
随机推荐
- [日常] Redis中set集合的使用思考
公司部门同事有个需求,就是需要把当前另一个部门a中存储的数据全部导出来,自己当前业务b的数据全部导出来,两个要取一下差集,把a中存在,b中不存在的记下来,要去调用某接口把对应的文件删除.这个我感觉可以 ...
- Pwn-level3
题目地址 https://dn.jarvisoj.com/challengefiles/level3.rar.2047525b05c499c9dd189ba212bba1f8 借鉴 https://w ...
- Linux通过端口号查看使用进程-结束进程
1. 查看进程(参数带 - 与不带有区别): command [options] 例:ps -a(配合其他options参数以展示进程更多参数) ps -ef | grep 进程名(返回值是该进程的 ...
- [LOJ 2133][UOJ 131][BZOJ 4199][NOI 2015]品酒大会
[LOJ 2133][UOJ 131][BZOJ 4199][NOI 2015]品酒大会 题意 给定一个长度为 \(n\) 的字符串 \(s\), 对于所有 \(r\in[1,n]\) 求出 \(s\ ...
- FineUIPro v6.0.0 大版本更新,快来围观!
本月末(2019-09-20),我们会发布激动人心的 FineUI v6.0.0 版本,这个版本将带来一系列的重要更新! 在列举新版本特性之前,我们先来回顾下每次发布大版本的关键时间点: v1.0.0 ...
- netstat查看端口状态
netstat netstat -tunlp 用于显示 tcp,udp 的端口和进程等相关情况. netstat 查看端口占用语法格式: netstat -tunlp | grep 端口号 -t (t ...
- Jmockit 构造函数与初始代码块
from Jmockit 中文网 http://jmockit.cn/showArticle.htm?channel=4&id=14 有些编写不良好的类的构造函数,初始代码块,静态代码块存在大 ...
- 队列和 BFS —— 栈和 DFS
队列和 BFS: 广度优先搜索(BFS)的一个常见应用是找出从根结点到目标结点的最短路径. 示例 这里我们提供一个示例来说明如何使用 BFS 来找出根结点 A 和目标结点 G 之间的最短路径. 洞悉 ...
- Python 下载图片的三种方法
import os os.makedirs('./image/', exist_ok=True) IMAGE_URL = "http://image.nationalgeographic.c ...
- JVM的监控工具之jmap
参考博客:https://www.jianshu.com/p/a4ad53179df3 jmap(Memory Map for Java)命令用于生成堆转储快照(一般称为heapdump或dump文件 ...