Deepo is a series of Docker images that

and their Dockerfile generator that


GPU Version

Installation

Step 1. Install Docker and nvidia-docker.

Step 2. Obtain the all-in-one image from Docker Hub

docker pull ufoym/deepo

For users in China who may suffer from slow speeds when pulling the image from the public Docker registry, you can pull deepo images from the China registry mirror by specifying the full path, including the registry, in your docker pull command, for example:

docker pull registry.docker-cn.com/ufoym/deepo

or

docker pull hub-mirror.c.163.com/ufoym/deepo

or

docker pull docker.mirrors.ustc.edu.cn/ufoym/deepo

Usage

Now you can try this command:

docker run --runtime=nvidia --rm ufoym/deepo nvidia-smi

This should work and enables Deepo to use the GPU from inside a docker container. If this does not work, search the issues section on the nvidia-docker GitHub – many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do

docker run --runtime=nvidia -it ufoym/deepo bash

If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.

docker run --runtime=nvidia -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash

This will make /host/data from the host visible as /data in the container, and /host/config as /config. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.

Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with --ipc=host or --shm-size command line options to docker run.

docker run --runtime=nvidia -it --ipc=host ufoym/deepo bash

CPU Version

Installation

Step 1. Install Docker.

Step 2. Obtain the all-in-one image from Docker Hub

docker pull ufoym/deepo:cpu

Usage

Now you can try this command:

docker run -it ufoym/deepo:cpu bash

If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.

docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo:cpu bash

This will make /host/data from the host visible as /data in the container, and /host/config as /config. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.

Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with --ipc=host or --shm-size command line options to docker run.

docker run -it --ipc=host ufoym/deepo:cpu bash

You are now ready to begin your journey.

$ python

>>> import tensorflow
>>> import sonnet
>>> import torch
>>> import keras
>>> import mxnet
>>> import cntk
>>> import chainer
>>> import theano
>>> import lasagne
>>> import caffe
>>> import caffe2

$ caffe --version

caffe version 1.0.0

$ darknet

usage: darknet <function>

$ th

 │  ______             __   |  Torch7
│ /_ __/__ ________/ / | Scientific computing for Lua.
│ / / / _ \/ __/ __/ _ \ | Type ? for help
│ /_/ \___/_/ \__/_//_/ | https://github.com/torch
│ | http://torch.ch

│th>

Customization

Note that docker pull ufoym/deepo mentioned in Quick Start will give you a standard image containing all available deep learning frameworks. You can customize your own environment as well.

Unhappy with all-in-one solution?

If you prefer a specific framework rather than an all-in-one image, just append a tag with the name of the framework. Take tensorflow for example:

docker pull ufoym/deepo:tensorflow

Jupyter support

Step 1. pull the image with jupyter support

docker pull ufoym/deepo:all-jupyter

Step 2. run the image

docker run --runtime=nvidia -it -p 8888:8888 --ipc=host ufoym/deepo:all-jupyter jupyter notebook --no-browser --ip=0.0.0.0 --allow-root --NotebookApp.token= --notebook-dir='/root'

Build your own customized image with Lego-like modules

Step 1. prepare generator

git clone https://github.com/ufoym/deepo.git
cd deepo/generator

Step 2. generate your customized Dockerfile

For example, if you like pytorch and lasagne, then

python generate.py Dockerfile pytorch lasagne

This should generate a Dockerfile that contains everything for building pytorch and lasagne. Note that the generator can handle automatic dependency processing and topologically sort the lists. So you don’t need to worry about missing dependencies and the list order.

You can also specify the version of Python:

python generate.py Dockerfile pytorch lasagne python==3.6

Step 3. build your Dockerfile

docker build -t my/deepo .

This may take several minutes as it compiles a few libraries from scratch.

Comparison to alternatives

. modern-deep-learning dl-docker jupyter-deeplearning Deepo
ubuntu 16.04 14.04 14.04 18.04
cuda X 8.0 6.5-8.0 8.0-10.0/None
cudnn X v5 v2-5 v7
onnx X X X O
theano X O O O
tensorflow O O O O
sonnet X X X O
pytorch X X X O
keras O O O O
lasagne X O O O
mxnet X X X O
cntk X X X O
chainer X X X O
caffe O O O O
caffe2 X X X O
torch X O O O
darknet X X X O

Tags

Available Tags

. CUDA 10.0 / Python 3.6 CPU-only / Python 3.6
all-in-one latest all all-py36 py36-cu100 all-py36-cu100 all-py36-cpu all-cpu py36-cpu cpu
all-in-one with jupyter all-jupyter-py36-cu100 all-jupyter-py36 all-jupyter all-py36-jupyter-cpu py36-jupyter-cpu
Theano theano-py36-cu100 theano-py36 theano theano-py36-cpu theano-cpu
TensorFlow tensorflow-py36-cu100 tensorflow-py36 tensorflow tensorflow-py36-cputensorflow-cpu
Sonnet sonnet-py36-cu100 sonnet-py36 sonnet sonnet-py36-cpu sonnet-cpu
PyTorch / Caffe2 pytorch-py36-cu100 pytorch-py36pytorch pytorch-py36-cpu pytorch-cpu
Keras keras-py36-cu100 keras-py36 keras keras-py36-cpu keras-cpu
Lasagne lasagne-py36-cu100 lasagne-py36lasagne lasagne-py36-cpu lasagne-cpu
MXNet mxnet-py36-cu100 mxnet-py36 mxnet mxnet-py36-cpu mxnet-cpu
CNTK cntk-py36-cu100 cntk-py36 cntk cntk-py36-cpu cntk-cpu
Chainer chainer-py36-cu100 chainer-py36chainer chainer-py36-cpu chainer-cpu
Caffe caffe-py36-cu100 caffe-py36 caffe caffe-py36-cpu caffe-cpu
Torch torch-cu100 torch torch-cpu
Darknet darknet-cu100 darknet darknet-cpu

Deprecated Tags

. CUDA 9.0 / Python 3.6 CUDA 9.0 / Python 2.7 CPU-only / Python 3.6 CPU-only / Python 2.7
all-in-one py36-cu90 all-py36-cu90 all-py27-cu90all-py27 py27-cu90   all-py27-cpupy27-cpu
all-in-one with jupyter all-jupyter-py36-cu90 all-py27-jupyter py27-jupyter   all-py27-jupyter-cpu py27-jupyter-cpu
Theano theano-py36-cu90 theano-py27-cu90 theano-py27   theano-py27-cpu
TensorFlow tensorflow-py36-cu90 tensorflow-py27-cu90tensorflow-py27   tensorflow-py27-cpu
Sonnet sonnet-py36-cu90 sonnet-py27-cu90 sonnet-py27   sonnet-py27-cpu
PyTorch pytorch-py36-cu90 pytorch-py27-cu90 pytorch-py27   pytorch-py27-cpu
Keras keras-py36-cu90 keras-py27-cu90keras-py27   keras-py27-cpu
Lasagne lasagne-py36-cu90 lasagne-py27-cu90 lasagne-py27   lasagne-py27-cpu
MXNet mxnet-py36-cu90 mxnet-py27-cu90mxnet-py27   mxnet-py27-cpu
CNTK cntk-py36-cu90 cntk-py27-cu90cntk-py27   cntk-py27-cpu
Chainer chainer-py36-cu90 chainer-py27-cu90 chainer-py27   chainer-py27-cpu
Caffe caffe-py36-cu90 caffe-py27-cu90caffe-py27   caffe-py27-cpu
Caffe2 caffe2-py36-cu90 caffe2-py36 caffe2 caffe2-py27-cu90 caffe2-py27 caffe2-py36-cpucaffe2-cpu caffe2-py27-cpu
Torch torch-cu90 torch-cu90 torch   torch-cpu
Darknet darknet-cu90 darknet-cu90darknet   darknet-cpu

Citation

@misc{ming2017deepo,
author = {Ming Yang},
title = {Deepo: set up deep learning environment in a single command line.},
year = {2017},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/ufoym/deepo}}
}

Contributing

We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us.

Licensing

Deepo is MIT licensed.

Deepo的更多相关文章

  1. ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(一)

    ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(一) 本教程适用于想要在远程服务器上配置docker图形界面用于深度学习的用户. (一)ubuntu18.04配置n ...

  2. ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(三)

    ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(三) 本教程适用于想要在远程服务器上配置docker图形界面用于深度学习的用户. (三)配置远程桌面连接访问dock ...

  3. ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(二)

    ubuntu18.04配置nvidia docker和远程连接ssh+远程桌面连接(二) 本教程适用于想要在远程服务器上配置docker图形界面用于深度学习的用户. (二)nvidia docker配 ...

  4. Vmvare + Ubuntu 16.04环境搭建 + 相关软件安装配置笔记【深度学习】

    前言 由于学习与工作的需要,加上之前配置好的vmmachines都损坏了,我就重新弄一个ubuntu虚拟机,配置一下环境,给自己留个记录 1.文件 2.配置过程 1.在Vmware中新建虚拟机,自定义 ...

  5. 教你如何用Docker快速搭建深度学习环境

    本教程搭建集 Tensorflow.Keras.Coffe.PyTorch 等深度学习框架于一身的环境,及jupyter. 本教程使用nvidia-docker启动实例,通过本教程可以从一个全新的Ub ...

  6. [AI] 切换cuda版本的万金油

    1. 环境 ubuntu16.04 GTX1080Ti x 4 nvidia-418 cuda-10.1 pytorch1.0.0 目标:在最新的显卡驱动下,使用不同版本的cuda和深度学习框架来执行 ...

  7. docker出现如下错误:Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

    在docker中配置deepo时出现了错误: 在出现这个错误之前,我是先用如下命令查看NVIDIA-docker是否安装成功. docker run --runtime=nvidia --rm nvi ...

  8. 服务器搭建远程docker深度学习环境

    服务器搭建远程docker深度学习环境 本文大部分内容参考知乎文章 Docker+PyCharm快速搭建机器学习开发环境 搭建过程中出现ssh连接问题可以查看最后的注意事项 Docker Docker ...

随机推荐

  1. Kustomize安装配置入门文档

    一,简介 kustomize是sig-cli的一个子项目,它的设计目的是给kubernetes的用户提供一种可以重复使用同一套配置的声明式应用管理,从而在配置工作中用户只需要管理和维护kubernet ...

  2. nginx上传大文件,413错误解决

    在nginx里增加了配置. client_max_body_size 500m; proxy_connect_timeout 600; proxy_read_timeout 600; proxy_se ...

  3. Windows下Redis安装配置和使用注意事项

    Windows下Redis安装配置和使用注意事项 一:下载 下载地址: https://github.com/microsoftarchive/redis/releases 文件介绍: 本文以3.2. ...

  4. request有get,post,put,delete等方法大全

    注:本文为个人学习摘录,原文地址为:http://javaeedevelop.iteye.com/blog/1725299 An HTTP request is a class consisting ...

  5. 网络编程 UDP协议 TCP局域网客户端与服务端上传下载电影示例

    UDP协议 (了解) 称之为数据包协议,又称不可靠协议. 特点: 1) 不需要建立链接. 2) 不需要知道对方是否收到. 3) 数据不安全 4) 传输速度快 5)能支持并发 6) 不会粘包 7) 无需 ...

  6. zookeper分布式搭建

    zookeper的安装如下链接 https://www.cnblogs.com/wanerhu/p/11144815.html

  7. BEST FREE UNITY ASSETS – OVER 200 CURATED QUALITY ASSETS

    http://www.procedural-worlds.com/blog/best-free-unity-assets-categorised-mega-list/ BEST FREE UNITY ...

  8. html--前端基本标签内容讲解

    body里面分为两类标签:块级标签和内联标签. 1.块级标签:<p><h1><table><ol><ul><form><d ...

  9. LeetCode 1034. Coloring A Border

    原题链接在这里:https://leetcode.com/problems/coloring-a-border/ 题目: Given a 2-dimensional grid of integers, ...

  10. Error in readRDS(dest) : error reading from connection

    Error in readRDS(dest) : error reading from connection 解决办法:可能是镜像设置错误,导致无法抓取文件 修改 RStudio 中的镜像地址 设置成 ...