pandas 之 datetime 初识
import numpy as np
import pandas as pd
认识
Time series data is an impotant from of data in many different fields, such as finance, economics, ecology, neuroscience(神经学) and physics. Anything that is observed or measured at many points in time forms a time series.
Many time series are fixed frequency , which is to say that data points occur at regular intervals according to some rule, such as every 15 seconds, every 5 minutes, or once per month.
Time series can also be irregular(不规则的) without a fixed unit of time or offset between units. How you mark and refer to time series data depends on the application, and you may have one of the following:
- Timestamps, specific instants in time (时间戳)
- Fixed periods, such as the month January 2007 or the full year 2010 (时期)
- Intervals of time, indicated by a start and end timestamp. Periods can be thought of as special cases of intervals. (时间间隔)
- Experiment or elapsed time(试验时间逝去); each timestamp is a measure of time relative to a particular start time (e.g. the diameter(直径) of a cookie baking each second since being palced in the oven)
In this chapter, I am mainly concerned with time series in the first three categories, though many of the teachniques can applied to experimental time series where the index may be an integer or floating-point number indicating elapsed time from the start of the experiment. The simplest and most widely used kind of time series are those indexed by timestamp.
pandas also supports indexes based on timedeltas, which can be a useful way of representing experiment or elapsed time. We do not explore timedelta indexes in this book , but you can learn more in the pandas documenttaion.
pandas provides many buit-in time series tools and data algorithims. You can efficiently work with very large time series and easily slice and dice, aggregate, and resample(重采样) irrgular-and fixed-frequency time series. Some of these tools are especially useful financial and economics applications, but you could certainly use them to analyze server log, too.
The Pyhton standard library includes data types for date and time data, as well as calendar-related(日历相关) functionality. The datetime, time, calendar modules are the main places to start. the datetime.datetime type, or simply datetime, is widely used.
from datetime import datetime
now = datetime.now()
now
datetime.datetime(2019, 4, 27, 15, 3, 14, 103616)
now.year, now.month, now.day, now.hour, now.minute
(2019, 4, 27, 15, 3)
datetime stores(存储) both the date and time down to the microsecond timedelta reprecents the temporal(临时的) difference between two datetime objects:
"cj 特方便, 在时间相加上"
delta = datetime(2011, 1, 7) - datetime(2008, 6, 24, 8, 15)
delta
'cj 特方便, 在时间相加上'
datetime.timedelta(926, 56700)
delta.days, delta.seconds
(926, 56700)
You can add (or subtract) a timedelata or multiple thereof to a datetime object to yield a new shifted object:
from datetime import timedelta
start = datetime(2011, 1, 7)
"加12天"
start + timedelta(12)
'加12天'
datetime.datetime(2011, 1, 19, 0, 0)
" 减去24天"
start - 2*timedelta(12)
' 减去24天'
datetime.datetime(2010, 12, 14, 0, 0)
Table 11-1 summarizes the data types in the datetime module. While this chapter is mainly concerned with the data types in pandas and high-level time series manupulation, you may encounter the datetime-based types in many other places in Pyhton in the wild.
Type | Description |
---|---|
date | Store calendar date (year, month, day) using the Gregorian calendar |
time | Store time of day as hours,minutes, seconds, and microseconds |
datetime | Store both date and time |
timedelta | Reprecents the difference between tow datetime values(as days,second..) |
tzinfo | Base type for storing time zone infomation |
String和Datetime间的转换
You can format datetime object and pandas Timestamp objects, which I'll introduce later, as strings using str or the strftime method, passing a format specification:
stamp = datetime(2011, 1, 3)
stamp
str(stamp)
datetime.datetime(2011, 1, 3, 0, 0)
'2011-01-03 00:00:00'
stamp.strftime('%Y-%m-%d') # 四位数字的年
'2011-01-03'
stamp.strftime('%y-%m-%d') # 2位数字的年
'11-01-03'
See Table 11-2 for a complete list of the format codes.
Type | Description |
---|---|
%Y | Four-digit year(4个数字的年) |
%y | Two-digit year |
%m | Two-dight month [01, 12] |
%d | Two-dight day [01, 31] |
%H | Hour(24-hour clock) [00, 23] |
%I | Hour(12-hour clock) [00, 12]) |
%M | Two-dight minute [00, 59] |
%S | Second [00, 61] (second 60, 61 acccount for leap second) |
%w | Weekday as integer[0(Sundday), 6] |
%U | |
%W | |
%z | UTC time zone offset as +HHMM or -HHMM; empty if time zone naive |
%F | Shortcut for %Y-%m-%d (eg. 2012-4-8) |
%D | Shortcut for %m/%d/%y (eg. 04/18/12) |
You can use these same format codes to convert strings to dates using date time.strptime:
value = "2011-01-03"
datetime.strptime(value, '%Y-%m-%d')
datetime.datetime(2011, 1, 3, 0, 0)
datestrs = ['7/6/2011', '8/6/2011']
[datetime.strptime(x, '%m/%d/%Y') for x in datestrs]
[datetime.datetime(2011, 7, 6, 0, 0), datetime.datetime(2011, 8, 6, 0, 0)]
Datetime.strptime is a good way to parse a date with a know format. However, it can be a bit annoying to have to write a format spec each time, especially for common date formats.In this case, you can use the parse.parse method in the third-party dateutil package (this is installed automatically when you install pandas).
from dateutil.parser import parse
parse("2011-01-03")
datetime.datetime(2011, 1, 3, 0, 0)
parse("2011/01/03")
datetime.datetime(2011, 1, 3, 0, 0)
dateutil si capable of parsing most human-intelligble date representation:
parse('Jan 31, 1997, 10:45 PM')
datetime.datetime(1997, 1, 31, 22, 45)
In international locales, day appering before month is very common, so you can pass dayfirst=True to indicate this:
parse('6/12/2011', dayfirst=True)
datetime.datetime(2011, 12, 6, 0, 0)
pandas is generally oriented toward working with arrays of dates, whether used an axis index or a column in a DataFrame. The to_datetime method parses many different kinds of date representations. Standard date formats like ISO 8601 can be parsed very quickly:
datestrs = ['2011-07-06 12:00:00', '2011-08-06 00:00:00']
pd.to_datetime(datestrs)
DatetimeIndex(['2011-07-06 12:00:00', '2011-08-06 00:00:00'], dtype='datetime64[ns]', freq=None)
It also handles values that should be condidered missing (None, empty string. etc.):
idx = pd.to_datetime(datestrs + [None])
idx
DatetimeIndex(['2011-07-06 12:00:00', '2011-08-06 00:00:00', 'NaT'], dtype='datetime64[ns]', freq=None)
idx[2]
NaT
pd.isnull(idx)
array([False, False, True])
NaT (Not a Time) is pandas's null value for timestamp data.
dateutil.parser is a useful but imperfect tool. Notably, it will recognize some strings as dates that you might prefer that it didn't for example. '42' will be parsed as the year 2042 with today's ccalendar date.
datetime objects also have a number of locale-specific formatting options for systems in other countries or languages. For example, the abbreviated(缩写) month names will be different on German or French systems compared with English systme. See Table 11-3 for a listing.
- %a Abbreviated weekday name
- %A Full weekday name
- %b 缩写月份的名字
- %B 全写月份
- %c Full date and time (eg. Tue 01 May 2012 04:20:57 PM)
- %p 包含AM or PM
- %x (eg. '05/01/2012')
- %X (eg. '04:24:12 PM')
pandas 之 datetime 初识的更多相关文章
- python之pandas学习笔记-初识pandas
初识pandas python最擅长的就是数据处理,而pandas则是python用于数据分析的最常用工具之一,所以学python一定要学pandas库的使用. pandas为python提供了高性能 ...
- Pandas 数据处理 | Datetime 在 Pandas 中的一些用法!
Datatime 是 Python 中一种时间数据类型,对于不同时间格式之间的转换是比较方便的,而在 Pandas 中也同样支持 DataTime 数据机制,可以借助它实现许多有用的功能,例如 1,函 ...
- Pandas 基础(1) - 初识及安装 yupyter
Hello, 大家好, 昨天说了我会再更新一个关于 Pandas 基础知识的教程, 这里就是啦......Pandas 被广泛应用于数据分析领域, 是一个很好的分析工具, 也是我们后面学习 machi ...
- 整理总结 python 中时间日期类数据处理与类型转换(含 pandas)
我自学 python 编程并付诸实战,迄今三个月. pandas可能是我最高频使用的库,基于它的易学.实用,我也非常建议朋友们去尝试它.--尤其当你本身不是程序员,但多少跟表格或数据打点交道时,pan ...
- (转) Using the latest advancements in AI to predict stock market movements
Using the latest advancements in AI to predict stock market movements 2019-01-13 21:31:18 This blog ...
- Python基础 | 日期时间操作
目录 获取时间 时间映射 格式转换 字符串转日期 日期转字符串 unixtime 时间计算 时间偏移 时间差 "日期时间数据"作为三大基础数据类型之一,在数据分析中会经常遇到. 本 ...
- pandas中将timestamp转为datetime
参考自:http://stackoverflow.com/questions/35312981/using-pandas-to-datetime-with-timestamps 在pandas Dat ...
- pandas处理时间序列(1):pd.Timestamp()、pd.Timedelta()、pd.datetime( )、 pd.Period()、pd.to_timestamp()、datetime.strftime()、pd.to_datetime( )、pd.to_period()
Pandas库是处理时间序列的利器,pandas有着强大的日期数据处理功能,可以按日期筛选数据.按日期显示数据.按日期统计数据. pandas的实际类型主要分为: timestamp(时间戳) ...
- pandas初识
pandas初识 1.生成DataFrame型的数据 import pandas as pd import numpy as np dates = pd.date_range('20130101',p ...
随机推荐
- 5-4 可视化库Seaborn-回归分析
In [2]: %matplotlib inline import numpy as np import pandas as pd from scipy import stats,integrate ...
- ACM-ICPC 2018 南京赛区网络预赛 I. Skr(回文树)
题意 https://nanti.jisuanke.com/t/A1955 求所有本质不同的回文串转成数后的和. 思路 如果了解回文树的构造原理,那么这题就很简单了,回文树每个结点代表一个回文串,每添 ...
- 在Rust中,cargo使用国内镜像源
一个编程语言依赖包管理的普通问题. cargo解决得比较优雅. 一,新建$HOME/.cargo/config文件 [source.crates-io] registry = "https: ...
- sudo apt-get 与 yum 常用命令
yum -RedHat:CentOS... -xxx.rpmsudo apt-get -Debian:Ubuntu... -xxx.deb 安装工具rpm -ivh yum-2.0.4-1.rh ...
- AtCoder Beginner Contest 147
A - Blackjack #include <bits/stdc++.h> int main() { int a, b, c; scanf("%d%d%d", &am ...
- mysql group by 的用法解析
1. group by的常规用法 group by的常规用法是配合聚合函数,利用分组信息进行统计,常见的是配合max等聚合函数筛选数据后分析,以及配合having进行筛选后过滤. 聚合函数max se ...
- Codeforces Round #596 (Div. 2, based on Technocup 2020 Elimination Round 2) A. Forgetting Things 水题
A. Forgetting Things Kolya is very absent-minded. Today his math teacher asked him to solve a simple ...
- Project Euler Problem 675
ORZ foreverlasting聚聚,QQ上问了他好久才会了这题(所以我们又聊了好久的Gal) 我们先来尝试推导一下\(S\)的性质,我们利用狄利克雷卷积来推: \[2^\omega=I\ast| ...
- Java后台+数据库+Java web前端——记账本
下面是本人实现的网页版(设计思路见上一篇https://www.cnblogs.com/sengzhao666/p/10445984.html) 代码如下: 运行截图: 首页: 创建: 账本删除:(先 ...
- RPC系列:基本概念
RPC(Remote Procedure Call):远程过程调用,它是一种通过网络从远程计算机程序上请求服务,而不需要了解底层网络技术的思想. RPC 是一种技术思想而非一种规范或协议,常见 RPC ...