2017年4月1日星期六

OpenShift_071：从 3.4.0.39 升级到 3.4.1.10

环境：OCP 3.4

1. 删除旧的 repo，在 Registry 机器上做）
cd /opt/ose
rm -rf rhel-7-server-extras-rpms rhel-7-server-ose-3.4-rpms rhel-7-server-rpms

2. 上传新的 repo（在 MAC 机器上做）
scp -r rhel-7-server-extras-rpms root@192.168.56.112:/opt/ose/
scp -r rhel-7-server-ose-3.4-rpms root@192.168.56.112:/opt/ose/
scp -r rhel-7-server-rpms root@192.168.56.112:/opt/ose/

3. 删除旧的 repodata，重新生成新的 repodata（在 Registry 机器上做）
cd /opt/ose
rm -rf repodata

重新生成 repodata
createrepo --worker=5 /opt/ose

4. 升级 docker 以及安装程序（在所有机器上执行）
清除 yum 缓存（在所有机器上执行）
yum clean all

确认最新的 yum 安装包（在所有机器上执行）

yum list | grep -i docker
输出包含如下内容：
...
docker-latest.x86_64                    1.12.6-11.el7              OpenShift  
docker-latest-logrotate.x86_64          1.12.6-11.el7              OpenShift  
docker-latest-v1.10-migrator.x86_64     1.12.6-11.el7              OpenShift
...

yum list | grep -i atomic-openshift
输出包含如下内容：
...
atomic-openshift-docker-excluder.noarch 3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-dockerregistry.x86_64  3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-excluder.noarch        3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-master.x86_64          3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-node.x86_64            3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-pod.x86_64             3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-sdn-ovs.x86_64         3.4.1.10-1.git.0.c96aed3.el7
atomic-openshift-tests.x86_64           3.4.1.10-1.git.0.c96aed3.el7
...

更新安装包（在所有机器上执行）
yum update -y atomic-openshift-utils
yum update -y docker
yum install -y atomic-openshift-excluder atomic-openshift-docker-excluder

5. 删除 Registry 机器上的旧镜像（在 Registry 机器上执行）
删除本地（registry.access.redhat.com）的 core 镜像
REGISTRY="registry.access.redhat.com";PTH="openshift3";VERSION="v3.4.0.39";

docker rmi $REGISTRY/$PTH/ose:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-haproxy-router:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-deployer:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-sti-builder:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-docker-builder:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-pod:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-keepalived-ipfailover:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-docker-registry:$VERSION; \
docker rmi $REGISTRY/$PTH/ose-recycler:$VERSION; \
docker rmi $REGISTRY/$PTH/registry-console:$VERSION;

删除本地 Docker Registry（registry.example.com:5000）中的 core 镜像
REGISTRY="registry.example.com:5000";PTH="openshift3";VERSION="v3.4.0.39";

docker rmi $REGISTRY/$PTH/ose:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-haproxy-router:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-deployer:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-sti-builder:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-docker-builder:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-pod:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-keepalived-ipfailover:$VERSION ; \
docker rmi $REGISTRY/$PTH/ose-docker-registry:$VERSION; \
docker rmi $REGISTRY/$PTH/ose-recycler:$VERSION; \
docker rmi $REGISTRY/$PTH/registry-console:$VERSION;

删除本地（registry.access.redhat.com）的 logging 和 metrics 镜像镜像
REGISTRY="registry.access.redhat.com";PTH="openshift3";VERSION="v3.4";

docker rmi $REGISTRY/$PTH/logging-deployer:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-elasticsearch:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-kibana:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-fluentd:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-auth-proxy:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-curator:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-deployer:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-hawkular-metrics:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-cassandra:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-heapster:$VERSION;

删除本地 Docker Registry（registry.example.com:5000）中的 logging 和 metrics 镜像
REGISTRY="registry.example.com:5000";PTH="openshift3";VERSION="v3.4";

docker rmi $REGISTRY/$PTH/logging-deployer:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-elasticsearch:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-kibana:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-fluentd:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-auth-proxy:$VERSION; \
docker rmi $REGISTRY/$PTH/logging-curator:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-deployer:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-hawkular-metrics:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-cassandra:$VERSION; \
docker rmi $REGISTRY/$PTH/metrics-heapster:$VERSION;

删除本地（registry.access.redhat.com）的 apps 镜像
REGISTRY="registry.access.redhat.com";PTH="openshift3";

docker rmi $REGISTRY/jboss-webserver-3/webserver30-tomcat7-openshift:latest; \
docker rmi $REGISTRY/jboss-webserver-3/webserver30-tomcat8-openshift:latest; \
docker rmi $REGISTRY/jboss-eap-6/eap64-openshift:latest; \
docker rmi $REGISTRY/jboss-eap-7/eap70-openshift:latest; \

docker rmi $REGISTRY/jboss-amq-6/amq62-openshift:latest; \
docker rmi $REGISTRY/jboss-fuse-6/fis-java-openshift:latest; \
docker rmi $REGISTRY/jboss-fuse-6/fis-karaf-openshift:latest; \
docker rmi $REGISTRY/jboss-processserver-6/processserver63-openshift:latest; \
docker rmi $REGISTRY/jboss-decisionserver-6/decisionserver63-openshift:latest; \

docker rmi $REGISTRY/rhscl/mongodb-32-rhel7:latest; \
docker rmi $REGISTRY/rhscl/mysql-56-rhel7:latest; \
docker rmi $REGISTRY/rhscl/mysql-57-rhel7:latest; \
docker rmi $REGISTRY/rhscl/php-56-rhel7:latest; \
docker rmi $REGISTRY/rhscl/php-70-rhel7:latest; \
docker rmi $REGISTRY/rhscl/python-35-rhel7:latest; \
docker rmi $REGISTRY/rhscl/redis-32-rhel7:latest; \
docker rmi $REGISTRY/rhscl/ruby-23-rhel7:latest; \
docker rmi $REGISTRY/rhscl/s2i-base-rhel7:latest; \

docker rmi $REGISTRY/$PTH/jenkins-1-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-2-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-base-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-maven-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-nodejs-rhel7:latest; \
docker rmi $REGISTRY/$PTH/nodejs-010-rhel7:latest;

删除本地的 Docker Registry（registry.example.com:5000）中的 apps 镜像
REGISTRY="registry.example.com:5000";PTH="openshift3";

docker rmi $REGISTRY/jboss-webserver-3/webserver30-tomcat7-openshift:latest; \
docker rmi $REGISTRY/jboss-webserver-3/webserver30-tomcat8-openshift:latest; \
docker rmi $REGISTRY/jboss-eap-6/eap64-openshift:latest; \
docker rmi $REGISTRY/jboss-eap-7/eap70-openshift:latest; \

docker rmi $REGISTRY/jboss-amq-6/amq62-openshift:latest; \
docker rmi $REGISTRY/jboss-fuse-6/fis-java-openshift:latest; \
docker rmi $REGISTRY/jboss-fuse-6/fis-karaf-openshift:latest; \
docker rmi $REGISTRY/jboss-processserver-6/processserver63-openshift:latest; \
docker rmi $REGISTRY/jboss-decisionserver-6/decisionserver63-openshift:latest; \

docker rmi $REGISTRY/rhscl/mongodb-32-rhel7:latest; \
docker rmi $REGISTRY/rhscl/mysql-56-rhel7:latest; \
docker rmi $REGISTRY/rhscl/mysql-57-rhel7:latest; \
docker rmi $REGISTRY/rhscl/php-56-rhel7:latest; \
docker rmi $REGISTRY/rhscl/php-70-rhel7:latest; \
docker rmi $REGISTRY/rhscl/python-35-rhel7:latest; \
docker rmi $REGISTRY/rhscl/redis-32-rhel7:latest; \
docker rmi $REGISTRY/rhscl/ruby-23-rhel7:latest; \
docker rmi $REGISTRY/rhscl/s2i-base-rhel7:latest; \

docker rmi $REGISTRY/$PTH/jenkins-1-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-2-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-base-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-maven-rhel7:latest; \
docker rmi $REGISTRY/$PTH/jenkins-slave-nodejs-rhel7:latest; \

docker rmi $REGISTRY/$PTH/nodejs-010-rhel7:latest;

确认 Registry 机器上没有任何镜像
docker images

6. 上传新的镜像（在 MAC 机器上做）
scp ose-images-core-v3.4.tar.gz root@192.168.56.112:/opt/ose/images/
scp ose-images-logging_metric-v3.4.tar.gz root@192.168.56.112:/opt/ose/images/
scp ose-images-apps-latest_20170402.tar.gz root@192.168.56.112:/opt/ose/images/

7. 加载并推送新的镜像（在 Registry 机器上做）
加载镜像
cd /opt/ose/images/
for i in `ls *.tar.gz` ; do docker load -i $i; done;
说明：如果硬盘空间紧张，建议一个一个做，做完一个删除一个。

推送镜像到本地Docker Registry
REDHAT_REG="registry.access.redhat.com";
PRIVATE_REG="registry.example.com:5000";
for i in $(docker images|grep $REDHAT_REG|awk '{print $1":"$2}') ; do docker tag  $i "$PRIVATE_REG$(echo $i|awk -F 'com' {'print $2'})" ; done;
for i in `docker images|grep $PRIVATE_REG|awk '{print $1":"$2}'` ; do  docker push $i; done;

对于 core 镜像，还必须重新打上详细版本的 Tag，否则部署时会报告找不到这些镜像。
REGISTRY="registry.example.com:5000";PTH="openshift3";VERSION1="v3.4";VERSION2="v3.4.1.10";

docker tag $REGISTRY/$PTH/ose:$VERSION1 $REGISTRY/$PTH/ose:$VERSION2
docker push $REGISTRY/$PTH/ose:$VERSION2

docker tag $REGISTRY/$PTH/ose-haproxy-router:$VERSION1 $REGISTRY/$PTH/ose-haproxy-router:$VERSION2
docker push $REGISTRY/$PTH/ose-haproxy-router:$VERSION2

docker tag $REGISTRY/$PTH/ose-deployer:$VERSION1 $REGISTRY/$PTH/ose-deployer:$VERSION2
docker push $REGISTRY/$PTH/ose-deployer:$VERSION2

docker tag $REGISTRY/$PTH/ose-sti-builder:$VERSION1 $REGISTRY/$PTH/ose-sti-builder:$VERSION2
docker push $REGISTRY/$PTH/ose-sti-builder:$VERSION2

docker tag $REGISTRY/$PTH/ose-docker-builder:$VERSION1 $REGISTRY/$PTH/ose-docker-builder:$VERSION2
docker push $REGISTRY/$PTH/ose-docker-builder:$VERSION2

docker tag $REGISTRY/$PTH/ose-pod:$VERSION1 $REGISTRY/$PTH/ose-pod:$VERSION2
docker push $REGISTRY/$PTH/ose-pod:$VERSION2

docker tag $REGISTRY/$PTH/ose-keepalived-ipfailover:$VERSION1 $REGISTRY/$PTH/ose-keepalived-ipfailover:$VERSION2
docker push $REGISTRY/$PTH/ose-keepalived-ipfailover:$VERSION2

docker tag $REGISTRY/$PTH/ose-docker-registry:$VERSION1 $REGISTRY/$PTH/ose-docker-registry:$VERSION2
docker push $REGISTRY/$PTH/ose-docker-registry:$VERSION2

docker tag $REGISTRY/$PTH/ose-recycler:$VERSION1 $REGISTRY/$PTH/ose-recycler:$VERSION2
docker push $REGISTRY/$PTH/ose-recycler:$VERSION2

docker tag $REGISTRY/$PTH/registry-console:$VERSION1 $REGISTRY/$PTH/registry-console:$VERSION2
docker push $REGISTRY/$PTH/registry-console:$VERSION2

8. 将 atomic-openshift 相关包从 yum 排除列表中移除（在所有机器上执行）
atomic-openshift-excluder unexclude

9. 编辑、确认 ansible 的 inventory 文件（在 Master 机器上执行）
cat /etc/ansible/hosts
输出如下：
[OSEv3:children]
masters
nodes

[OSEv3:vars]
ansible_ssh_user=root
deployment_type=openshift-enterprise
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]

[masters]
master.example.com

[nodes]
master.example.com
node1.example.com
node2.example.com

开始升级
ansible-playbook -i /etc/ansible/hosts \
 /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_4/upgrade.yml
大约经过 10 分钟，最后输出如下:
......
PLAY RECAP *********************************************************************
localhost                  : ok=35   changed=3    unreachable=0    failed=0   
master.example.com         : ok=332  changed=48   unreachable=0    failed=0   
node1.example.com          : ok=101  changed=12   unreachable=0    failed=0   
node2.example.com          : ok=101  changed=12   unreachable=0    failed=0   

10. 将 atomic-openshift 相关包添加至 yum 排除列表（在所有机器上执行）
atomic-openshift-excluder exclude

11. 重启所有机器

 12. 修改 docker 配置文件并重启 docker（在所有机器上执行）
cp /etc/sysconfig/docker /etc/sysconfig/docker.bak.$(date "+%Y%m%d%H%M%S");
sed -i s/".*OPTIONS=.*"/"OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0\/16 --insecure-registry registry.example.com:5000'"/g /etc/sysconfig/docker;
sed -i 's/registry.access.redhat.com/registry.example.com:5000/g' /etc/sysconfig/docker;
systemctl restart docker

13. 确认升级成功（在 Master 机器上执行）
oc get node
输出如下：
NAME                 STATUS                     AGE
master.example.com   Ready,SchedulingDisabled   71d
node1.example.com    Ready                      71d
node2.example.com    Ready                      71d

oc get -n default dc/docker-registry -o json | grep \"image\"
输出如下：
"image": "openshift3/ose-docker-registry:v3.4.1.10",

oc get -n default dc/ose-router -o json | grep \"image\"
输出如下：
"image": "openshift3/ose-haproxy-router:v3.4.1.10",

确认 default project 中 ose-router 和 docker-registry pod 启动成功
oc project default
oc get pod
输出如下：
NAME                      READY     STATUS    RESTARTS   AGE
docker-registry-2-dgyaa   1/1       Running   0          56s
ose-router-2-5xeuo        1/1       Running   0          57s

创建创建一个用户并分配权限，然后登录控制台，快速创建一个 PHP 应用
htpasswd -cb /etc/origin/master/htpasswd redhat welcome1
oc adm policy add-cluster-role-to-user cluster-admin redhat
请参考《离线安装 OCP 3.4 之测试 PHP 应用》。

14. 删除 Node1 和 Node2 机器上的旧镜像（在 Node1 和 Node2 机器上执行）
docker rmi registry.example.com:5000/openshift3/ose-haproxy-router:v3.4.0.39
docker rmi registry.example.com:5000/openshift3/ose-deployer:v3.4.0.39
docker rmi registry.example.com:5000/openshift3/ose-docker-registry:v3.4.0.39
docker rmi registry.example.com:5000/openshift3/ose-pod:v3.4.0.39

15. 删除原有的 Image Stream，导入新的 Image Stream

删除原有的 Image Stream
for i in $(oc get is -n openshift --no-headers|awk '{print $1}') ; do oc delete is $i -n openshift; done;

创建指向本地 Docker Registry 的 Image Stream
REDHAT_REG="registry.access.redhat.com";
PRIVATE_REG="registry.example.com:5000";

sed s/"${REDHAT_REG}"/"${PRIVATE_REG}"/g  /usr/share/openshift/examples/image-streams/image-streams-rhel7.json |sed '/"creationTimestamp": null/a\\t,"annotations": {"openshift.io/image.insecureRepository": "true"}' |oc create -n openshift -f - ;

重新导入 Image
for i in $(oc get is -n openshift --no-headers|awk '{print $1}'); do oc import-image $i --insecure  -n openshift;done

16. 使用 oc adm diagnostics 检测 OpenShift（在 Master 机器上执行）
oc adm diagnostics
大约需要等待 3 分钟，因为需要创建 2 个 network project 验证网络，输出如下（只显示有错误的部分）：
......
ERROR: [DCli0014 from diagnostic ConfigContexts@openshift/origin/pkg/diagnostics/client/config_contexts.go:285]
       For client config context '/master-example-com:8443/redhat':
       The server URL is 'https://master.example.com:8443'
       The user authentication is 'redhat/master-example-com:8443'
       The current project is 'default'
       (*errors.StatusError) the server has asked for the client to provide credentials
       
       This means that when we tried to make a request to the master API
       server, the request required credentials that were not presented. This
       can happen with an expired or invalid authentication token. Try logging
       in with this user again.
......
ERROR: [DNet2005 from diagnostic NetworkCheck@openshift/origin/pkg/diagnostics/network/run_pod.go:117]
       Setting up test environment for network diagnostics failed: Failed to run network diags test pod and service: [timed out waiting for the condition, timed out waiting for the condition]
......

 oc login -u redhat -p welcome1
第 1 个错误没有了，第 2 个错误还在，第 2 个错误跟.Net 有关，可以忽略。

参考文献：
1. https://bbs.archlinux.org/viewtopic.php?id=208295
标签：

没有评论:

发表评论

订阅： 博文评论 (Atom)