OpenShift_020：部署与运行第三方的 docker image

环境：OCP 3.3

本文以 https://hub.docker.com/r/gitlab/gitlab-ce/ 为例说明，如何部署与运行第三方的 docker image。

1. 在 MAC OS X 上操作
docker pull gitlab/gitlab-ce
docker save -o gitlab-ce.tar gitlab/gitlab-ce
gzip -v gitlab-ce.tar
scp gitlab-ce.tar.gz root@registry.example.com:/opt/ocp/images

2. 在 registry.example.com 上操作
cd /opt/ocp/images
docker load -i gitlab-ce.tar.gz
docker images | grep gitlab
docker tag gitlab/gitlab-ce:latest registry.example.com:5000/gitlab-ce:latest
docker push registry.example.com:5000/gitlab-ce:latest

3. 在 master.example.com 上操作
oc login -u system:admin
oc label node node2.example.com app="yes"

oc login -u admin -p admin
oc new-project applications

oc login -u system:admin
oc annotate namespace applications openshift.io/node-selector='app=yes' --overwrite

oc login -u admin -p admin
oc new-app registry.example.com:5000/gitlab-ce:latest --name=gitlab-ce --insecure-registry=true
输出如下：
--> Found Docker image f086a16 (5 days old) from registry.example.com:5000 for "registry.example.com:5000/gitlab-ce:latest"

    * An image stream will be created as "gitlab-ce:latest" that will track this image
    * This image will be deployed in deployment config "gitlab-ce"
    * Ports 22/tcp, 443/tcp, 80/tcp will be load balanced by service "gitlab-ce"
      * Other containers can access this service through the hostname "gitlab-ce"
    * This image declares volumes and will default to use non-persistent, host-local storage.
      You can add persistent volumes later by running 'volume dc/gitlab-ce --add ...'
    * WARNING: Image "registry.example.com:5000/gitlab-ce:latest" runs as the 'root' user which may not be permitted by your cluster administrator

--> Creating resources with label app=gitlab-ce ...
    imagestream "gitlab-ce" created
    deploymentconfig "gitlab-ce" created
    service "gitlab-ce" created
--> Success
    Run 'oc status' to view your app.

oc get pods -o wide
最终输出如下：
NAME                 READY     STATUS         RESTARTS   AGE       IP          NODE
gitlab-ce-1-deploy   1/1       Running        0          59s       10.1.1.10   node2.example.com
gitlab-ce-1-xvpi8    0/1       ErrImagePull   0          55s       10.1.1.11   node2.example.com

4. 为了解决 ErrImagePull 问题，在 node2.example.com 上操作
systemctl stop docker
rm -rf /var/lib/docker/*
systemctl start docker

5. 在 master.example.com 上操作，重新部署
oc delete all --all
oc new-app registry.example.com:5000/gitlab-ce:latest --name=gitlab-ce --insecure-registry=true
oc get pods -o wide
输出如下：
NAME                READY     STATUS             RESTARTS   AGE       IP          NODE
gitlab-ce-1-8jyg2   0/1       CrashLoopBackOff   2          2m        10.1.1.11   node2.example.com

oc logs gitlab-ce-1-8jyg2
输出如下：
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=8.15.2-ce.0

Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:

  docker exec -it gitlab vim /etc/gitlab/gitlab.rb
  docker restart gitlab

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md

If this container fails to start due to permission problems try to fix it by executing:

  docker exec -it gitlab update-permissions
  docker restart gitlab

Generating ssh_host_rsa_key...
No user exists for uid 1000070000

6. 修改 SCC（Security Context Control），允许 sa 以任何用户在容器中操作。
修改 SCC（Security Context Control），为 sa default  授予 anyuid SCC，允许该 sa 以任何用户在容器中操作。
如果 Dockerfile 中定义了 USER，就是使用该 USER；如果 Dockerfile 中没有定义了 USER，就使用 root。

oc login -u system:admin
oadm policy add-scc-to-user anyuid -z default
oc edit scc anyuid
看到如下信息就对了：
users:
- system:serviceaccount:applications:default

7. 在 master.example.com 上操作，再次重新部署
oc login -u admin -p admin
oc delete all --all
oc new-app registry.example.com:5000/gitlab-ce:latest --name=gitlab-ce --insecure-registry=true
oc get pods -o wide
输出如下：
NAME                READY     STATUS    RESTARTS   AGE       IP          NODE
gitlab-ce-1-i9fqp   1/1       Running   0          7m        10.1.1.11   node2.example.com

oc get svc gitlab-ce
输出如下：
NAME        CLUSTER-IP      EXTERNAL-IP   PORT(S)                 AGE
gitlab-ce   172.30.38.120           22/TCP,80/TCP,443/TCP   1m

curl http://172.30.38.120
等待一会，输出页面内容，说明正确。

oc expose service gitlab-ce --hostname=gitlab-ce.apps.example.com
访问  http://gitlab-ce.apps.example.com

8. 在 master.example.com 上操作，导出 template
oc get pods --show-labels
输出如下:
NAME                READY     STATUS    RESTARTS   AGE       LABELS
gitlab-ce-1-i9fqp   1/1       Running   0          5m        app=gitlab-ce,deployment=gitlab-ce-1,deploymentconfig=gitlab-ce

oc export all --as-template=gitlab --selector='app=gitlab-ce' -o json
输出如下：
{
    "kind": "Template",
    "apiVersion": "v1",
    "metadata": {
        "name": "gitlab",
        "creationTimestamp": null
    },
    "objects": [
        {
            "kind": "ImageStream",
            "apiVersion": "v1",
            "metadata": {
                "name": "gitlab-ce",
                "generation": 2,
                "creationTimestamp": null,
                "labels": {
                    "app": "gitlab-ce"
                },
                "annotations": {
                    "openshift.io/generated-by": "OpenShiftNewApp",
                    "openshift.io/image.dockerRepositoryCheck": "2017-01-03T10:11:59Z"
                }
            },
            "spec": {
                "tags": [
                    {
                        "name": "latest",
                        "annotations": {
                            "openshift.io/imported-from": "registry.example.com:5000/gitlab-ce:latest"
                        },
                        "from": {
                            "kind": "DockerImage",
                            "name": "172.30.123.246:5000/applications/gitlab-ce:latest"
                        },
                        "generation": 2,
                        "importPolicy": {
                            "insecure": true
                        }
                    }
                ]
            },
            "status": {
                "dockerImageRepository": ""
            }
        },
        {
            "kind": "DeploymentConfig",
            "apiVersion": "v1",
            "metadata": {
                "name": "gitlab-ce",
                "generation": 2,
                "creationTimestamp": null,
                "labels": {
                    "app": "gitlab-ce"
                },
                "annotations": {
                    "openshift.io/generated-by": "OpenShiftNewApp"
                }
            },
            "spec": {
                "strategy": {
                    "type": "Rolling",
                    "rollingParams": {
                        "updatePeriodSeconds": 1,
                        "intervalSeconds": 1,
                        "timeoutSeconds": 600,
                        "maxUnavailable": "25%",
                        "maxSurge": "25%"
                    },
                    "resources": {}
                },
                "triggers": [
                    {
                        "type": "ConfigChange"
                    },
                    {
                        "type": "ImageChange",
                        "imageChangeParams": {
                            "automatic": true,
                            "containerNames": [
                                "gitlab-ce"
                            ],
                            "from": {
                                "kind": "ImageStreamTag",
                                "namespace": "applications",
                                "name": "gitlab-ce:latest"
                            }
                        }
                    }
                ],
                "replicas": 1,
                "test": false,
                "selector": {
                    "app": "gitlab-ce",
                    "deploymentconfig": "gitlab-ce"
                },
                "template": {
                    "metadata": {
                        "creationTimestamp": null,
                        "labels": {
                            "app": "gitlab-ce",
                            "deploymentconfig": "gitlab-ce"
                        },
                        "annotations": {
                            "openshift.io/container.gitlab-ce.image.entrypoint": "[\"/assets/wrapper\"]",
                            "openshift.io/generated-by": "OpenShiftNewApp"
                        }
                    },
                    "spec": {
                        "volumes": [
                            {
                                "name": "gitlab-ce-volume-1",
                                "emptyDir": {}
                            },
                            {
                                "name": "gitlab-ce-volume-2",
                                "emptyDir": {}
                            },
                            {
                                "name": "gitlab-ce-volume-3",
                                "emptyDir": {}
                            }
                        ],
                        "containers": [
                            {
                                "name": "gitlab-ce",
                                "image": "registry.example.com:5000/gitlab-ce:latest",
                                "ports": [
                                    {
                                        "containerPort": 443,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 80,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 22,
                                        "protocol": "TCP"
                                    }
                                ],
                                "resources": {},
                                "volumeMounts": [
                                    {
                                        "name": "gitlab-ce-volume-1",
                                        "mountPath": "/var/log/gitlab"
                                    },
                                    {
                                        "name": "gitlab-ce-volume-2",
                                        "mountPath": "/var/opt/gitlab"
                                    },
                                    {
                                        "name": "gitlab-ce-volume-3",
                                        "mountPath": "/etc/gitlab"
                                    }
                                ],
                                "terminationMessagePath": "/dev/termination-log",
                                "imagePullPolicy": "Always"
                            }
                        ],
                        "restartPolicy": "Always",
                        "terminationGracePeriodSeconds": 30,
                        "dnsPolicy": "ClusterFirst",
                        "securityContext": {}
                    }
                }
            },
            "status": {
                "observedGeneration": 2,
                "replicas": 1,
                "updatedReplicas": 1,
                "availableReplicas": 1
            }
        },
        {
            "kind": "ReplicationController",
            "apiVersion": "v1",
            "metadata": {
                "name": "gitlab-ce-1",
                "generation": 1,
                "creationTimestamp": null,
                "labels": {
                    "app": "gitlab-ce",
                    "openshift.io/deployment-config.name": "gitlab-ce"
                },
                "annotations": {
                    "openshift.io/deployer-pod.name": "gitlab-ce-1-deploy",
                    "openshift.io/deployment-config.latest-version": "1",
                    "openshift.io/deployment-config.name": "gitlab-ce",
                    "openshift.io/deployment.phase": "Complete",
                    "openshift.io/deployment.replicas": "1",
                    "openshift.io/deployment.status-reason": "caused by an image change",
                    "openshift.io/encoded-deployment-config": "{\"kind\":\"DeploymentConfig\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"gitlab-ce\",\"namespace\":\"applications\",\"selfLink\":\"/oapi/v1/namespaces/applications/deploymentconfigs/gitlab-ce\",\"uid\":\"0dc44461-d19d-11e6-8a12-0800277f2689\",\"resourceVersion\":\"18803\",\"generation\":2,\"creationTimestamp\":\"2017-01-03T10:11:55Z\",\"labels\":{\"app\":\"gitlab-ce\"},\"annotations\":{\"openshift.io/generated-by\":\"OpenShiftNewApp\"}},\"spec\":{\"strategy\":{\"type\":\"Rolling\",\"rollingParams\":{\"updatePeriodSeconds\":1,\"intervalSeconds\":1,\"timeoutSeconds\":600,\"maxUnavailable\":\"25%\",\"maxSurge\":\"25%\"},\"resources\":{}},\"triggers\":[{\"type\":\"ConfigChange\"},{\"type\":\"ImageChange\",\"imageChangeParams\":{\"automatic\":true,\"containerNames\":[\"gitlab-ce\"],\"from\":{\"kind\":\"ImageStreamTag\",\"namespace\":\"applications\",\"name\":\"gitlab-ce:latest\"},\"lastTriggeredImage\":\"registry.example.com:5000/gitlab-ce:latest\"}}],\"replicas\":1,\"test\":false,\"selector\":{\"app\":\"gitlab-ce\",\"deploymentconfig\":\"gitlab-ce\"},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"gitlab-ce\",\"deploymentconfig\":\"gitlab-ce\"},\"annotations\":{\"openshift.io/container.gitlab-ce.image.entrypoint\":\"[\\\"/assets/wrapper\\\"]\",\"openshift.io/generated-by\":\"OpenShiftNewApp\"}},\"spec\":{\"volumes\":[{\"name\":\"gitlab-ce-volume-1\",\"emptyDir\":{}},{\"name\":\"gitlab-ce-volume-2\",\"emptyDir\":{}},{\"name\":\"gitlab-ce-volume-3\",\"emptyDir\":{}}],\"containers\":[{\"name\":\"gitlab-ce\",\"image\":\"registry.example.com:5000/gitlab-ce:latest\",\"ports\":[{\"containerPort\":443,\"protocol\":\"TCP\"},{\"containerPort\":80,\"protocol\":\"TCP\"},{\"containerPort\":22,\"protocol\":\"TCP\"}],\"resources\":{},\"volumeMounts\":[{\"name\":\"gitlab-ce-volume-1\",\"mountPath\":\"/var/log/gitlab\"},{\"name\":\"gitlab-ce-volume-2\",\"mountPath\":\"/var/opt/gitlab\"},{\"name\":\"gitlab-ce-volume-3\",\"mountPath\":\"/etc/gitlab\"}],\"terminationMessagePath\":\"/dev/termination-log\",\"imagePullPolicy\":\"Always\"}],\"restartPolicy\":\"Always\",\"terminationGracePeriodSeconds\":30,\"dnsPolicy\":\"ClusterFirst\",\"securityContext\":{}}}},\"status\":{\"latestVersion\":1,\"observedGeneration\":2,\"details\":{\"message\":\"caused by an image change\",\"causes\":[{\"type\":\"ImageChange\",\"imageTrigger\":{\"from\":{\"kind\":\"ImageStreamTag\",\"namespace\":\"applications\",\"name\":\"gitlab-ce:latest\"}}}]}}}\n"
                }
            },
            "spec": {
                "replicas": 1,
                "selector": {
                    "app": "gitlab-ce",
                    "deployment": "gitlab-ce-1",
                    "deploymentconfig": "gitlab-ce"
                },
                "template": {
                    "metadata": {
                        "creationTimestamp": null,
                        "labels": {
                            "app": "gitlab-ce",
                            "deployment": "gitlab-ce-1",
                            "deploymentconfig": "gitlab-ce"
                        },
                        "annotations": {
                            "openshift.io/container.gitlab-ce.image.entrypoint": "[\"/assets/wrapper\"]",
                            "openshift.io/deployment-config.latest-version": "1",
                            "openshift.io/deployment-config.name": "gitlab-ce",
                            "openshift.io/deployment.name": "gitlab-ce-1",
                            "openshift.io/generated-by": "OpenShiftNewApp"
                        }
                    },
                    "spec": {
                        "volumes": [
                            {
                                "name": "gitlab-ce-volume-1",
                                "emptyDir": {}
                            },
                            {
                                "name": "gitlab-ce-volume-2",
                                "emptyDir": {}
                            },
                            {
                                "name": "gitlab-ce-volume-3",
                                "emptyDir": {}
                            }
                        ],
                        "containers": [
                            {
                                "name": "gitlab-ce",
                                "image": "registry.example.com:5000/gitlab-ce:latest",
                                "ports": [
                                    {
                                        "containerPort": 443,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 80,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 22,
                                        "protocol": "TCP"
                                    }
                                ],
                                "resources": {},
                                "volumeMounts": [
                                    {
                                        "name": "gitlab-ce-volume-1",
                                        "mountPath": "/var/log/gitlab"
                                    },
                                    {
                                        "name": "gitlab-ce-volume-2",
                                        "mountPath": "/var/opt/gitlab"
                                    },
                                    {
                                        "name": "gitlab-ce-volume-3",
                                        "mountPath": "/etc/gitlab"
                                    }
                                ],
                                "terminationMessagePath": "/dev/termination-log",
                                "imagePullPolicy": "Always"
                            }
                        ],
                        "restartPolicy": "Always",
                        "terminationGracePeriodSeconds": 30,
                        "dnsPolicy": "ClusterFirst",
                        "securityContext": {}
                    }
                }
            },
            "status": {
                "replicas": 0
            }
        },
        {
            "kind": "Service",
            "apiVersion": "v1",
            "metadata": {
                "name": "gitlab-ce",
                "creationTimestamp": null,
                "labels": {
                    "app": "gitlab-ce"
                },
                "annotations": {
                    "openshift.io/generated-by": "OpenShiftNewApp"
                }
            },
            "spec": {
                "ports": [
                    {
                        "name": "22-tcp",
                        "protocol": "TCP",
                        "port": 22,
                        "targetPort": 22
                    },
                    {
                        "name": "80-tcp",
                        "protocol": "TCP",
                        "port": 80,
                        "targetPort": 80
                    },
                    {
                        "name": "443-tcp",
                        "protocol": "TCP",
                        "port": 443,
                        "targetPort": 443
                    }
                ],
                "selector": {
                    "app": "gitlab-ce",
                    "deploymentconfig": "gitlab-ce"
                },
                "type": "ClusterIP",
                "sessionAffinity": "None"
            },
            "status": {
                "loadBalancer": {}
            }
        },
        {
            "kind": "Pod",
            "apiVersion": "v1",
            "metadata": {
                "generateName": "gitlab-ce-1-",
                "creationTimestamp": null,
                "labels": {
                    "app": "gitlab-ce",
                    "deployment": "gitlab-ce-1",
                    "deploymentconfig": "gitlab-ce"
                },
                "annotations": {
                    "kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"applications\",\"name\":\"gitlab-ce-1\",\"uid\":\"102f7e75-d19d-11e6-8a12-0800277f2689\",\"apiVersion\":\"v1\",\"resourceVersion\":\"18818\"}}\n",
                    "openshift.io/container.gitlab-ce.image.entrypoint": "[\"/assets/wrapper\"]",
                    "openshift.io/deployment-config.latest-version": "1",
                    "openshift.io/deployment-config.name": "gitlab-ce",
                    "openshift.io/deployment.name": "gitlab-ce-1",
                    "openshift.io/generated-by": "OpenShiftNewApp",
                    "openshift.io/scc": "anyuid"
                }
            },
            "spec": {
                "volumes": [
                    {
                        "name": "gitlab-ce-volume-1",
                        "emptyDir": {}
                    },
                    {
                        "name": "gitlab-ce-volume-2",
                        "emptyDir": {}
                    },
                    {
                        "name": "gitlab-ce-volume-3",
                        "emptyDir": {}
                    },
                    {
                        "name": "default-token-uukq6",
                        "secret": {
                            "secretName": "default-token-uukq6"
                        }
                    }
                ],
                "containers": [
                    {
                        "name": "gitlab-ce",
                        "image": "registry.example.com:5000/gitlab-ce:latest",
                        "ports": [
                            {
                                "containerPort": 443,
                                "protocol": "TCP"
                            },
                            {
                                "containerPort": 80,
                                "protocol": "TCP"
                            },
                            {
                                "containerPort": 22,
                                "protocol": "TCP"
                            }
                        ],
                        "resources": {},
                        "volumeMounts": [
                            {
                                "name": "gitlab-ce-volume-1",
                                "mountPath": "/var/log/gitlab"
                            },
                            {
                                "name": "gitlab-ce-volume-2",
                                "mountPath": "/var/opt/gitlab"
                            },
                            {
                                "name": "gitlab-ce-volume-3",
                                "mountPath": "/etc/gitlab"
                            },
                            {
                                "name": "default-token-uukq6",
                                "readOnly": true,
                                "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
                            }
                        ],
                        "terminationMessagePath": "/dev/termination-log",
                        "imagePullPolicy": "Always",
                        "securityContext": {
                            "capabilities": {
                                "drop": [
                                    "MKNOD",
                                    "SYS_CHROOT"
                                ]
                            },
                            "privileged": false,
                            "seLinuxOptions": {
                                "level": "s0:c8,c7"
                            }
                        }
                    }
                ],
                "restartPolicy": "Always",
                "terminationGracePeriodSeconds": 30,
                "dnsPolicy": "ClusterFirst",
                "nodeSelector": {
                    "app": "yes"
                },
                "host": "node2.example.com",
                "serviceAccountName": "default",
                "serviceAccount": "default",
                "nodeName": "node2.example.com",
                "securityContext": {
                    "seLinuxOptions": {
                        "level": "s0:c8,c7"
                    }
                },
                "imagePullSecrets": [
                    {
                        "name": "default-dockercfg-ves7r"
                    }
                ]
            },
            "status": {
                "phase": "Pending"
            }
        }
    ]
}


参考文献：
1. https://blog.openshift.com/getting-any-docker-image-running-in-your-own-openshift-cluster/