Openshift 和Harbor的集成

1.安装配置Harbor

环境rhel 7.6

安装docker,python
安装docker-compose

sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

chmod +x /usr/local/docker-compose
 
[root@harbor harbor]# docker-compose version
docker-compose version 1.21., build a133471
docker-py version: 3.3.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.1t   May

下载harbor

https://github.com/goharbor/harbor/releases

我选择的是harbor-offline-installer-v1.6.2.tgz, 之前 1.5的版本没有镜像了，所以建议选择新一点的。

tar -xvf harbor-offline-installer-v1.6.2.tgz

修改配置harbor.cfg，我就修改了hostname

[root@harbor harbor]# cat harbor.cfg
## Configuration file of Harbor
 
#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version = 1.6.
#The IP address or hostname to access admin UI and registry service.
#DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname = 192.168.56.107:
 
#The protocol for accessing the UI and token/notification service, by default it is http.
#It can be set to https if ssl is enabled on nginx.
ui_url_protocol = http
 
#Maximum number of job workers in job service
max_job_workers =

修改配置docker-compose.yml,将端口映射修改为8060

    networks:
      - harbor
    ports:
      - :
      - :
      - :

根据docker-compose.yml中的镜像把镜像下载到本地

[root@harbor ~]# docker images
REPOSITORY                              TAG                 IMAGE ID            CREATED             SIZE
docker.io/openjdk                       latest              8e7eacedab93         days ago           MB
docker.io/goharbor/redis-photon         v1.6.2              473bfdd9d245         weeks ago          MB
docker.io/goharbor/registry-photon      v2.6.2-v1.6.2       62c30cdb384a         weeks ago          MB
docker.io/goharbor/nginx-photon         v1.6.2              c0602500e829         weeks ago          MB
docker.io/goharbor/harbor-log           v1.6.2              781ee4ceb5d3         weeks ago          MB
docker.io/goharbor/harbor-jobservice    v1.6.2              3419a2276f96         weeks ago          MB
docker.io/goharbor/harbor-ui            v1.6.2              66268686bb96         weeks ago          MB
docker.io/goharbor/harbor-adminserver   v1.6.2              4024440925a4         weeks ago          MB
docker.io/goharbor/harbor-db            v1.6.2              0ed4186be0d1         weeks ago          MB

修改docker配置 /etc/sysconfig/docker，主要是OPTIONS,ADD_REGISTRY,INSECURE_REGISTRY

[root@harbor harbor]# cat /etc/sysconfig/docker
# /etc/sysconfig/docker
 
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled=false --log-driver=journald --insecure-registry=192.168.56.107:8060'
if [ -z "${DOCKER_CERT_PATH}" ]; then
    DOCKER_CERT_PATH=/etc/docker
fi
 
# Do not add registries in this file anymore. Use /etc/containers/registries.conf
# instead. For more information reference the registries.conf() man page.
ADD_REGISTRY='--add-registry 192.168.56.107:8060'
INSECURE_REGISTRY='--insecure-registry=192.168.56.107:8060'
 
# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp
 
# Controls the /etc/cron.daily/docker-logrotate cron job status.
# To disable, uncomment the line below.
# LOGROTATE=false

重启docker服务

systemctl daemon-reload
systemctl restart docker.service

安装

[root@harbor harbor]# ./prepare
Generated and saved secret to file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/config.yml
Generated configuration file: ./common/config/log/logrotate.conf
Generated configuration file: ./common/config/registryctl/env
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.

[root@harbor harbor]# ./install.sh
 
[Step ]: checking installation environment ...
 
Note: docker version: 1.13.
 
Note: docker-compose version: 1.21.
 
[Step ]: loading Harbor images ...
Loaded image: goharbor/registry-photon:v2.6.2-v1.6.2
0155cb3a636c: Loading layer [==================================================>] 23.38 MB/23.38 MB
62f917db5fed: Loading layer [==================================================>] 12.16 MB/12.16 MB
2e192a070c25: Loading layer [==================================================>]  17.3 MB/17.3 MB
64fa72e486ec: Loading layer [==================================================>] 11.26 kB/11.26 kB
23afd47b0f1a: Loading layer [==================================================>] 3.072 kB/3.072 kB
3fa7415d357e: Loading layer [==================================================>] 29.46 MB/29.46 MB
Loaded image: goharbor/notary-server-photon:v0.5.1-v1.6.2
2f06068ec40a: Loading layer [==================================================>]    MB/ MB
d6e5bcc842f3: Loading layer [==================================================>] 10.93 MB/10.93 MB
c272c6b03ae0: Loading layer [==================================================>] 2.048 kB/2.048 kB
7b0653de0007: Loading layer [==================================================>] 48.13 kB/48.13 kB
484f0b8e979d: Loading layer [==================================================>] 3.072 kB/3.072 kB
72004696fb26: Loading layer [==================================================>] 10.98 MB/10.98 MB

验证

[root@harbor harbor]# docker-compose ps
       Name                     Command                  State                                     Ports
---------------------------------------------------------------------------------------------------------------------------------------
harbor-adminserver   /harbor/start.sh                 Up (healthy)
harbor-db            /entrypoint.sh postgres          Up (healthy)   /tcp
harbor-jobservice    /harbor/start.sh                 Up
harbor-log           /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:->/tcp
harbor-ui            /harbor/start.sh                 Up (healthy)
nginx                nginx -g daemon off;             Up (healthy)   0.0.0.0:->/tcp, 0.0.0.0:->/tcp, 0.0.0.0:->/tcp
redis                docker-entrypoint.sh redis ...   Up             /tcp
registry             /entrypoint.sh /etc/regist ...   Up (healthy)   /tcp

登录192.168.56.107:8060, admin/Harbor12345

添加用户，添加项目，然后push镜像验证。

2.Openshift配置

1.需要在每个拉取镜像的节点上配置docker的配置，配置方法参照步骤1的docker配置。

2.修改/etc/origin/master/master-config.yaml配置文件，加入下面这一句

imagePolicyConfig:
  internalRegistryHostname: docker-registry.default.svc:
  externalRegistryHostname: 192.168.56.107:

3.重新启动master

# master-restart api
# master-restart controllers

在项目中就可以对harbor的公有镜像进行随意拉取和启动了

==============================================================================

openshift相当于一个Harbor的客户端，但是oc自己有一套管理用户权限的机制(可以绑定LDAP），而Harbor自己也有一套管理镜像的权限机制(可以绑定LDAP做认证)

因此这两套机制如何统一联动是个问题。

在原来的interregistry下，每个项目是可以看到自己下面的镜像的

但以相同的路径加入Harbor的镜像明显不再管理之列

而且发现一个bug,通过命令get is的时候发现镜像前全部换成了外部镜像库的地址，不知道是不是我忘了什么地方没有配置