Docker registry垃圾回收

通过：

docker run -p 5000:5000 -v /netdata/xxxx/registry:/var/lib/registry registry.cn-hangzhou.aliyuncs.com/term/registry-gc:2.7 registry garbage-collect /var/lib/registry/config.yml

搭建好registry服务后，docker push alpine:3.8镜像至registry服务。其中alpine的dockerfile内容如下：

FROM scratch

ADD rootfs.tar.xz /

CMD ["/bin/sh"]

registry.cn-hangzhou.aliyuncs.com/term/registry-gc:2.7的dockerfile如下:

FROM alpine:3.4

RUN set -ex \

    && apk add --no-cache ca-certificates apache2-utils

COPY registry /bin/registry

COPY config.yml /etc/docker/registry/config.yml

VOLUME ["/var/lib/registry"]

EXPOSE 5000

COPY docker-entrypoint.sh /entrypoint.sh

RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

CMD ["/etc/docker/registry/config.yml"]

config.yml内容如下:

version: 0.1

log:

  fields:

    service: registry

storage:

  cache:

    blobdescriptor: inmemory

  filesystem:

    rootdirectory: /var/lib/registry

  delete:

    enabled: true

  maintenance:

    readonly:

      enabled: false

http:

  addr: :5000

  headers:

    X-Content-Type-Options: [nosniff]

health:

  storagedriver:

    enabled: true

    interval: 10s

    threshold: 3

后端存储

镜像推送至registry后，可查看registry后端的本地存储目录结构如下：

docker

`-- registry

    `-- v2

        |-- blobs

        |   `-- sha256

        |       |-- 01

        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

        |       |       `-- data

        |       |-- 1e

        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

        |       |       `-- data

        |       `-- 5a

        |           `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

        |               `-- data

        `-- repositories

            `-- alpine

                |-- _layers

                |   `-- sha256

                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

                |       |   `-- link

                |       `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

                |           `-- link

                |-- _manifests

                |   |-- revisions

                |   |   `-- sha256

                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

                |   |           `-- link

                |   `-- tags

                |       `-- 3.8

                |           |-- current

                |           |   `-- link

                |           `-- index

                |               `-- sha256

                |                   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

                |                       `-- link

                `-- _uploads

查看Manifest

查看alpine:3.8的manifest文件内容：

curl -i -H "Accept:application/vnd.docker.distribution.manifest.v2+json" http://127.0.0.1:5000/v2/alpine/manifests/3.8

Response Header:

HTTP/1.1 200 OK

Content-Length: 526

Content-Type: application/vnd.docker.distribution.manifest.v2+json

Docker-Content-Digest: sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

Docker-Distribution-Api-Version: registry/2.0

Etag: "sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667"

X-Content-Type-Options: nosniff

Date: Tue, 04 Sep 2018 11:59:14 GMT

Response body:

{

   "schemaVersion": 2,

   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",

   "config": {

      "mediaType": "application/vnd.docker.container.image.v1+json",

      "size": 1512,

      "digest": "sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc"

   },

   "layers": [

      {

         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",

         "size": 12456,

         "digest": "sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f"

      }

   ]

}

执行垃圾回收

sudo docker run --rm --name gctest -v /home/term/registry:/var/lib/registry registry:2.5.1 garbage-collect  -d /var/lib/registry/config.yml

得到的结果如下：

alpine

alpine: marking manifest sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

alpine: marking blob sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

alpine: marking configuration sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

3 blobs marked, 0 blobs eligible for deletion

发现所有的blobs都被mark, 无blobs可清理

删除Manifest

curl -X DELETE http://127.0.0.1:5000/v2/alpine/manifests/sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667从获取Manifest时的Header: Docker-Content-Digest里获取。

调用manifest API删除manifest文件后，再次查看registry后端存储目录结构:

docker

`-- registry

    `-- v2

        |-- blobs

        |   `-- sha256

        |       |-- 01

        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

        |       |       `-- data

        |       |-- 1e

        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

        |       |       `-- data

        |       |-- 5a

        |       |   `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

        |       |       `-- data

        |       `-- a3

        |           `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

        |               `-- data

        `-- repositories

            `-- alpine

                |-- _layers

                |   `-- sha256

                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

                |       |   `-- link

                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

                |       |   `-- link

                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

                |           `-- link

                |-- _manifests

                |   |-- revisions

                |   |   `-- sha256

                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

                |   `-- tags

                `-- _uploads

对比之前的目录，发现alpine下的tags目录内容已被删

再次垃圾回收

再次执行垃圾回收：

sudo docker run --rm --name gctest -v /home/term/registry:/var/lib/registry registry:2.5.1 garbage-collect  -d /var/lib/registry/config.yml

得到结果如下：

alpine

0 blobs marked, 4 blobs eligible for deletion

blob eligible for deletion: sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

blob eligible for deletion: sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

blob eligible for deletion: sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

blob eligible for deletion: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

以上为dry-run方式执行。

真正执行gc时，日志如下:

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/08/0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/11/11cd0b38bc3ceb958ffb2f9bd70be3fb317ce7d255c8a4c3f4af30e298aa1aab" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/1e/1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/5a/5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/8e/8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/01/0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

gc只清除blobs下的数据，但不会清除repositories里的内容及blobs下对应的目录。

docker

`-- registry

    `-- v2

        |-- blobs

        |   `-- sha256

        |       |-- 01

        |       |-- 08

        |       |-- 11

        |       |-- 1e

        |       |-- 5a

        |       |-- 8e

        |       `-- a3

        `-- repositories

            `-- alpine

                |-- _layers

                |   `-- sha256

                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

                |       |   `-- link

                |       |-- 11cd0b38bc3ceb958ffb2f9bd70be3fb317ce7d255c8a4c3f4af30e298aa1aab

                |       |   `-- link

                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

                |       |-- 8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6

                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

                |           `-- link

                |-- _manifests

                |   |-- revisions

                |   |   `-- sha256

                |   |       |-- 0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b

                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

                |   `-- tags

                `-- _uploads

删除blobs

调用registry api删除blob：

curl -i -X DELETE http://127.0.0.1:5000/v2/alpine/blobs/sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

Response Header:

HTTP/1.1 202 Accepted

Content-Length: 0

Docker-Distribution-Api-Version: registry/2.0

X-Content-Type-Options: nosniff

Date: Wed, 05 Sep 2018 01:05:20 GMT

Content-Type: text/plain; charset=utf-8

Reponse body:

docker

`-- registry

    `-- v2

        |-- blobs

        |   `-- sha256

        |       |-- 01

        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

        |       |       `-- data

        |       |-- 1e

        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

        |       |       `-- data

        |       |-- 5a

        |       |   `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

        |       |       `-- data

        |       `-- a3

        |           `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

        |               `-- data

        `-- repositories

            `-- alpine

                |-- _layers

                |   `-- sha256

                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

                |       |   `-- link

                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

                |           `-- link

                |-- _manifests

                |   |-- revisions

                |   |   `-- sha256

                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

                |   `-- tags

                `-- _uploads

对比之前目录结构，发现:docker/registry/v2/repositoires/<name>/_layers/<algorithm>/<digest>/link文件被删。

Config内容

{

    "architecture": "amd64",

    "config": {

        "ArgsEscaped": true,

        "AttachStderr": false,

        "AttachStdin": false,

        "AttachStdout": false,

        "Cmd": [

            "/bin/sh"

        ],

        "Domainname": "",

        "Entrypoint": null,

        "Env": [

            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

        ],

        "Hostname": "",

        "Image": "sha256:80c02d9956b63482b0e4344c56cc029d8c330e01a3f6093516b0a60b83118b9c",

        "Labels": null,

        "OnBuild": null,

        "OpenStdin": false,

        "StdinOnce": false,

        "Tty": false,

        "User": "",

        "Volumes": null,

        "WorkingDir": ""

    },

    "container": "bf174046ba695f7f6c1d841dd2894b2c34a96d73bb832c102b3ba2b6a0d20120",

    "container_config": {

        "ArgsEscaped": true,

        "AttachStderr": false,

        "AttachStdin": false,

        "AttachStdout": false,

        "Cmd": [

            "/bin/sh",

            "-c",

            "#(nop) ",

            "CMD [\"/bin/sh\"]"

        ],

        "Domainname": "",

        "Entrypoint": null,

        "Env": [

            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

        ],

        "Hostname": "bf174046ba69",

        "Image": "sha256:80c02d9956b63482b0e4344c56cc029d8c330e01a3f6093516b0a60b83118b9c",

        "Labels": {},

        "OnBuild": null,

        "OpenStdin": false,

        "StdinOnce": false,

        "Tty": false,

        "User": "",

        "Volumes": null,

        "WorkingDir": ""

    },

    "created": "2018-09-04T09:32:26.252138866Z",

    "docker_version": "17.06.2-ce",

    "history": [

        {

            "created": "2018-09-04T09:32:25.454551817Z",

            "created_by": "/bin/sh -c #(nop) ADD file:babb191309280a09293bff66ca4aead1ecad6119d9add3a803d489fa866d5282 in / "

        },

        {

            "created": "2018-09-04T09:32:26.252138866Z",

            "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",

            "empty_layer": true

        }

    ],

    "os": "linux",

    "rootfs": {

        "diff_ids": [

            "sha256:2a7f9859cf97588d6533a9ac63c988b34beff0cc2758182b70e7a8cc8383d7f3"

        ],

        "type": "layers"

    }

}

结论

  1. 要想调用registry API，请打开deletable=true的开关，可通过环境变量: REGISTRY_STORAGE_DELETE_ENABLED=true指定, 也可在config.yml里配置
  2. 镜像清除，删除manifest即可，无需调用API删除layer，真正Blob清理需执行garbage-collect.
  3. 执行garbage-collect前，请将regsitry的readonly=true开关打开，重启registry，保证gc时，registry只读。此时不允许docker push镜像，否则会清理掉gc过程中新push的blobs。清理完成后，将readonly开关关闭,再次重启registry。
  4. 目前registry暂无GC API，只能通过脚本执行GC工作。

坑点

  1. 目前registry不支持通过环境变量设置readonly=true，在registry:2.5.1版本试过，会导致registry起不来，报错:
goroutine 1 [running]:

panic(0xc2a220, 0xc820438620)

	/usr/local/go/src/runtime/panic.go:481 +0x3e6

github.com/docker/distribution/registry/handlers.NewApp(0x7fd101c70ad8, 0xc8203ba780, 0xc82000d600, 0x7fd101c70ad8)

	/go/src/github.com/docker/distribution/registry/handlers/app.go:135 +0xc44

github.com/docker/distribution/registry.NewRegistry(0x7fd101c70b18, 0xc8203ba780, 0xc82000d600, 0xc82000d600, 0x0, 0x0)

	/go/src/github.com/docker/distribution/registry/registry.go:86 +0x2b0

github.com/docker/distribution/registry.glob.func1(0x14cc660, 0xc820347f50, 0x1, 0x1)

	/go/src/github.com/docker/distribution/registry/registry.go:55 +0x2a9

github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x14cc660, 0xc820347f00, 0x1, 0x1, 0x0, 0x0)

	/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x6d4

github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x14cc800, 0x0, 0x0)

	/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x180

main.main()

须通过额外挂载配置文件解决

Docker registry垃圾回收的更多相关文章

  1. Docker垃圾回收机制

    由Docker垃圾回收机制引发的一场血案 AlstonWilliams 关注 2017.04.01 19:00* 字数 1398 阅读 253评论 0喜欢 0 今天早晨,在我还没睡醒的时候,我们团队中 ...

  2. Docker registry 私有仓库镜像查询、删除、上传、下载 shell

    #Docker官方私有仓库registry #官方只提供了API接口,不方便使用,就写了个shell #docker-registry安装配置http://www.cnblogs.com/elvi/p ...

  3. Docker——Registry 通过Shell管理私有仓库镜像

    使用方法: 复制代码保存为 image_registry.sh sh image_registry.sh  -h   #查看帮助 HUB=10.0.29.104:5000 改为自己的地址 #!/bin ...

  4. docker registry 删除镜像 垃圾回收

    操作步骤 通过环境变量修改默认配置,允许删除 获取image的sha值 进入registry容器中,执行垃圾回收 删除残留目录 #环境变量 REGISTRY_STORAGE_DELETE_ENABLE ...

  5. docker 垃圾回收机制

    docker垃圾回收机制 作者: 张首富 时间: 2019-04-10 个人博客: www.zhangshoufu.com QQ群: 895291458 说明 对于Docker来说,存在镜像/容器/存 ...

  6. Docker Registry私有仓库搭建

    部署registry 准备一个registry.mydocker.com 的证书 对私有registry取名registry.mydocker.com 目录规划 仓库数据目录:/data/docker ...

  7. Docker(十二)-Docker Registry镜像管理

    Registry删除镜像.垃圾回收 Docker仓库在2.1版本中支持了删除镜像的API,但这个删除操作只会删除镜像元数据,不会删除层数据.在2.4版本中对这一问题进行了解决,增加了一个垃圾回收命令, ...

  8. 企业级Docker Registry —— Harbor搭建和使用

    本节内容: Harbor介绍 安装部署Harbor 环境要求 环境信息 安装部署harbor 配置harbor 配置存储 完成安装和启动harbor 访问Harbor 修改管理员密码 启动后相关容器 ...

  9. docker registry 镜像删除

    registry:2.5.0版本的镜像,将镜像默认存放在了/var/lib/registry 目录下 /var/lib/registry/docker/registry/v2/repositories ...

随机推荐

  1. Python高级变量类型

    1.列表基础操作 2.列表循环遍历 3.元组 4.字典基础操作 5.字典循环遍历 6.字符串基础操作 7.字符串切片 8.公共方法

  2. Ubuntu16.04下安装Hyperledger Fabric 1.0.0

    系统环境 * Ubuntu: 16.04 * Go: 1.9.2 * NodeJS: v6.12.0 * Docker: 17.09.0-ce * HyperLedger Fabric: 1.0.0 ...

  3. c# 简单方便的连接oracle方式

    通过nuget安装ManagedDataAccess (自动生成的config里面的配置都可以删掉) winform程序,拖出一个datagridview和button using Oracle.Ma ...

  4. 2018.7.3 lnmp一键安装包无人值守版本 php7.2 + nginx1.14.0 + mariadb5.5 + centos7.1(1503) 环境搭建 + Thinkphp5.1.7 配置

    给自己练习用的,整个过程追求一个简单粗暴,没有配置虚拟主机,现在记录一下过程. 1. 进入到lnmp解压缩后的文件夹conf/rewrite,把thinkphp.conf复制一份到/usr/local ...

  5. LeetCode【110. 平衡二叉树】

    对于平衡二叉树,就是左右深度相差1 就可以另外弄一个函数,计算深度,然后, 在原函数上进行比较深度是否相差1,再输出true or false. 至于迭代就可以,比较完左右节点,再比较各自的左右节点. ...

  6. 5、分布式缓存Redis之bitmap、setbit

    基本语法: 1)SETBIT redis 127.0.0.1:6379> setbit KEY_NAME OFFSET VALUE //该命令用于对 key 所储存的字符串值,设置或清除指定偏移 ...

  7. 【算法习题】数组中任意2个（3个）数的和为sum的组合

    题1.给定一个int数组,一个数sum,求数组中和为sum的任意2个数的组合 @Test public void test_find2() { int[] arr = { -1, 0, 2, 3, 4 ...

  8. linux下各安装包的安装方法

    <转>linux下各安装包的安装方法   一.rpm包安装方式步骤: 1.找到相应的软件包,比如soft.version.rpm,下载到本机某个目录: 2.打开一个终端,su -成root ...

  9. Java笔记Spring（八）

    <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.sp ...

  10. HashTable源码简单介绍

    首先是继承了字典类Dictionary, 这说明HashTable的一些找位置的实现与Dictionary有关, 看一看数据结构,是一个entry数组, Entry,这个不陌生,先看一下它的结构吧,这 ...