代码:https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On

Chapter 1 What is Reinforcement Learning

Learning - supervised, unsupervised, and reinforcement

RL is not completely blind as in an unsupervised learning setup--we have a reward system.

(1) life is suffering, which could be totally wrong. In machine learning terms, it can be rephrased as having non-i.i.d data.

(2) exploration/exploitation dilemma is one of the open fundamental question in RL.

(3) the third complication factor lays in the fact that reward can be seriously delayed from actions.

RL fromalisms and realtions

RL entities and their communications

  • Agent和Environment是图的两个node
  • Actions作为edge由Agent指向Environment
  • Rewards和Observations作为edge由Environment指向Agent

Reward

We don't define how frequently the agent receives this reward. In the case of once-in-a-lifetime reward systems, all rewards except the last one will be zero.

The agent

The environment

Action

two types of actions: discrete or continuous.

Observations

Markov decision process

It is the theoretical foundation of RL, which makes it possible to start moving toward the methods used to solve the RL problem.

we start from the simplest case of a Markov Process(also known as a Markov chain), then extend it with rewards, which will turn it into a Markov reward processes. Then we'll put this idea into one other extra envelop by adding actions, which will lead us to Markov Decision Processes.

Markov process

you can always make your model more complex by extending your state space, which will allow you to capture more dependencies in the model at the cost of a large state space.

you can capture transition probabilities with a transition matrix, which is a square matrix of the size NxN, where N is the number of states in your model.

可以根据观测的episodes来估计transition matrix

Markov reward process

first thing is to add reward to Markov process model.

representation: reward transition matrix or a more compact representation, which is applicable only if the reward value depends only on the target state, which is not always the case.

second thing is to add discount factor gamma(from 0 to 1).

Markov decision process

add a dimension 'action' to transition matrix.

Chapter 2 OpenAI Gym

Chapter 3 Deep Learning with PyTorch

Chapter 4 The Cross-Entropy Method

Taxonomy of RL methods

  • Model-free or model-based
  • Value-based or policy-based
  • On-policy or off-policy

Practional cross-entropy

DRL Hands-on book的更多相关文章

  1. Drools mvel方言drl断点调试方法

    开发环境:myeclipse2014,  jdk1.8.0.91,drools6.4.0.Final, drools-eclipse-plugin,mvel2-2.2.6.Final问题描述:drl使 ...

  2. DRL之:策略梯度方法 (Policy Gradient Methods)

    DRL 教材 Chpater 11 --- 策略梯度方法(Policy Gradient Methods) 前面介绍了很多关于 state or state-action pairs 方面的知识,为了 ...

  3. 基于DRL和TORCS的自动驾驶仿真系统——之环境配置

    基于DRL和TORCS的自动驾驶仿真系统 --之环境配置 玩TORCS和DRL差不多有一整年了,开始的摸爬滚打都是不断碰壁过来的,近来在参与CMU的DRL10703课程学习和翻译志愿者工作,也将自己以 ...

  4. DRL前沿之:Benchmarking Deep Reinforcement Learning for Continuous Control

    1 前言 Deep Reinforcement Learning可以说是当前深度学习领域最前沿的研究方向,研究的目标即让机器人具备决策及运动控制能力.话说人类创造的机器灵活性还远远低于某些低等生物,比 ...

  5. DRL强化学习:

    IT博客网 热点推荐 推荐博客 编程语言 数据库 前端 IT博客网 > 域名隐私保护 免费 DRL前沿之:Hierarchical Deep Reinforcement Learning 来源: ...

  6. drools原生drl规则文件的使用

    在初识drools中对drl文件进行了简单的介绍.这里举个例子来具体说明下.主要是写了规则之后我们如何用java代码来run起来. drl文件内容如下: rule "ageUp12" ...

  7. Reinforcement Learning,微信公众号:DRL学习

    欢迎大家关注微信公众号:DRL学习,我们一起来学习强化学习和深度强化学习的算法及现状应用问题. 强化学习简单说就是学习如何最大化未来奖励的预期总和,以及agent学会在环境中做出的行动序列,其中随机状 ...

  8. Drools规则引擎详解-常用的drl实例

    package droolsDemo //说明:每个 drl 都必须声明一个包名,这个包名与 Java 里面的不同,它不需要与文件夹的层次结构一致, //主要用于可以根据kmodule.xml中不同的 ...

  9. allegro生成光绘文件时,通过cam打开,*.drl钻孔文件不识别,为Unknow类型

    生成钻孔文件时,NC_Parameters中,应该选Absolute

  10. DRL 教程 | 如何保持运动小车上的旗杆屹立不倒?TensorFlow利用A3C算法训练智能体玩CartPole游戏

    本教程讲解如何使用深度强化学习训练一个可以在 CartPole 游戏中获胜的模型.研究人员使用 tf.keras.OpenAI 训练了一个使用「异步优势动作评价」(Asynchronous Advan ...

随机推荐

  1. 【原创】大数据基础之ETL vs ELT or DataWarehouse vs DataLake

    ETL ETL is an abbreviation of Extract, Transform and Load. In this process, an ETL tool extracts the ...

  2. centos配置发送邮件

    邮件已经可以接收到了 CentOS下发送邮件有很多种方法,这里采用比较简单的mail客户端,配置第三方邮件服务商,例如:163.com 1.安装所用软件 yum install mailx sendm ...

  3. JS基础_属性名和属性值

    <!DOCTYPE html> <html> <head> <meta charset="UTF-8"> <title> ...

  4. luogu题解P2312解方程--暴力模+秦九韶

    题目链接 https://www.luogu.org/problemnew/show/P2312 分析 这道题很毒啊,这么大的数. 但是如果多项式\(\sum_{i=0}^N a[i]*X^i=0\) ...

  5. http请求204

    项目中发现一个奇怪的问题,请求的时候同一个接口有两个请求,而且有一个状态为204,有一个为200 在网上查看资料后得知,是因为跨域而引起的,OPTIONS是一种“预检请求”,浏览器在处理跨域访问的请求 ...

  6. 【转】ESXi主机出现“主机上的系统日志存储在非持久存储器中”解决办法

    原址:https://blog.csdn.net/mooncarp/article/details/50923483 当ESXi主机的底层操作系统安装在SD卡上时,在vCenter中配置该主机时,如果 ...

  7. 什么是PAM认证

    PAM(Pluggable Authentication Modules )是由 Sun 提出的一种用于实现应用程序的认证机制.其核心是一套共享库,目的是提供一个框架和一套编程接口,将认证工作由程序员 ...

  8. Delphi RS-232C标准

  9. Ansible简单介绍(一)

    一 :ansible简单介绍 此名取自 Ansible 作者最喜爱的<安德的游戏> 小说,而这部小说更被后人改编成电影 -<战争游戏>. 官网地址:https://www.an ...

  10. 2.java多线程_synchronized(Lock)同步

    1.synchronized同步关键词 线程安全是并发编程中的重要关注点,应该注意到的是,造成线程安全问题的主要诱因有两点,一是存在共享数据(也称临界资源),二是存在多条线程共同 操作共享数据.因此为 ...