OpenShift_010：综合实验（OSE 3.0.1）

环境：OSE 3.0.1

0. 开始实验之前, 重置所有虚机，重置后会自动重新启动所有虚机
[kiosk@fundation0 ~]$ rht-vmctl reset all

1. 安装 OSE 3.0

1.1 OSE 3.0 安装先决条件
（1）确认 master 节点主机已经注册了必须的软件频道（node 节点主机也做此步）
      [root@master ~]# yum repolist
（2）安装必要软件，一共 9 个软件（node 节点主机也做此步）
      [root@master ~]# yum -y install docker wget git net-tools bind-utils iptables-services bridge-utils python-virtualenv gcc
（3）确认主机名和 IP 地址对应（node 节点主机也做此步）
      [root@master ~]# host $(hostname -f)（node 节点主机也做此步）
（4）生成 SSH 无密码的公钥和私钥对
      [root@master ~]# ssh-keygen -N ''
（5）复制 SSH 公钥到所有主机
      [root@master ~]# ssh-copy-id root@node.pod0.example.com 口令 redhat
      [root@master ~]# ssh-copy-id root@master.pod0.example.com 口令 redhat
（6）stop 并 disable NetworkManager 和 firewalld（node 节点主机也做此步）
      [root@master ~]# systemctl stop NetworkManager
      [root@master ~]# systemctl disable NetworkManager
      [root@master ~]# systemctl stop firewalld
      [root@master ~]# systemctl disable firewalld

1.2. 配置 docker daemon
 （1）安装 private registry 证书作为受信证书（node 节点主机也做此步）
      [root@master ~]# yum -y install ca-certificates
      [root@master ~]# update-ca-trust enable
      [root@master ~]# scp root@workstation.pod0.example.com:/etc/pki/tls/certs/self.crt /etc/pki/ca-trust/source/anchors/
      [root@master ~]# update-ca-trust extract
（2）在继续下一步之前, 确认停止 docker（node 节点主机也做此步）
      [root@master ~]# systemctl stop docker
（3）修改文件, 确认只搜索 private registry（node 节点主机也做此步）
      [root@master ~]# vi /etc/sysconfig/docker
       #ADD_REGISTRY='--add-registry registry.access.redhat.com'
       ADD_REGISTRY='--add-registry workstation.pod0.example.com:5000'
       BLOCK_REGISTRY='--block-registry public --block-registry registry.access.redhat.com'
（4）创建新文件, 指定容器（非持久存储）使用 LVM thinpool（node 节点主机也做此步）
      [root@master ~]# vi /etc/sysconfig/docker-storage-setup
       DEVS=/dev/vdb
       VG=docker-vg
       SETUP_LVM_THIN_POOL=yes
（5）配置 LVM thinpool（node 节点主机也做此步）
      [root@master ~]# lvmconf --disable-cluster
      [root@master ~]# docker-storage-setup
（6）再次 start 并 enable docker（node 节点主机也做此步）
      [root@master ~]# systemctl start docker
      [root@master ~]# systemctl enable docker

1.3. 预取所有 OSE 以及运行时容器 images 到所有主机
（1）用脚本 pull 所有 images （node 节点主机也做此步）
      [root@master ~]# /root/DO280/labs/lab-review/fetch.sh
fetch.sh 脚本内容如下：
#!/bin/bash

# OSE 3 containerized services images
# Needs version because the private repo do not have ":latest" tags
for image in \
    openshift3/ose-haproxy-router openshift3/ose-deployer openshift3/ose-sti-builder openshift3/ose-sti-image-builder \
    penshift3/ose-docker-builder openshift3/ose-pod openshift3/ose-docker-registry openshift3/ose-keepalived-ipfailover
do docker pull $image:v3.0.1.0; done

# OSE 3 runtime images
for image in \
    openshift3/ruby-20-rhel7 openshift3/php-55-rhel7 openshift3/nodejs-010-rhel7 openshift3/mysql-55-rhel7 \
    jboss-eap-6/eap-openshift
do docker pull $image; done

# sample images
for image in \
    openshift/hello-openshift php-quote
do docker pull $image; done


重要说明:
（1）如果没有脚本, 搜索 openshift3, 然后再 pull 具体的 image （node 节点主机也做此步）
      [root@master ~]# docker search openshift3
      [root@master ~]# docker pull openshift3/ose-haproxy-router
（2）某些 images, 比如 xPaaS images 没有 openshift3 作为名字的一部分。
      作为本实验, 记得下载 jboss-eap-6/eap-openshift image。
（3）不要搜索 openshift, 然后下载所有返回的 images, 因为大多数 images 来自社区, 没有经过认证。
      [root@master ~]# docker search openshift     

1.4. 运行 OSE 快速安装脚本
（1）解压
      [root@master ~]# mkdir ose-installer ; cd ose-installer
      [root@master ose-installer]# tar xzf /root/DO280/installers/oo-install-ose.tgz
（2）安装
      [root@master ose-installer]# ./oo-install-ose
       • 输入 Y 确认继续
       • 回车确认 root 作为 ssh 访问用户
       • 按任意键继续, 准备添加 master 主机名列表
       • 安装程序会打开一个 vi 编辑器, 输入 master 主机名列表:
         master.pod0.example.com
         wq！保存
       • 输入 Y 确认 master 主机名列表
       • 按任意键继续, 准备添加 node 主机名列表, 注意 master 也是一种 node
       • 安装程序会打开一个 vi 编辑器, 输入 node 主机名列表: (注意 master 也是一种 node, 并且已经添加了 master)
       • 输入 node 主机名列表:
          master.pod0.example.com // 已添加
          node.pod0.example.com
          wq！保存
       • 输入 Y 确认 node 主机名列表
       • 按任意键继续
       • 安装程序会打开一个 vi 编辑器, 包含所有的 master 和 node 主机名和 IP 地址列表
         确认无误后, wq！保存
       • 输入 Y 确认继续
       • 出现 The installation was successful！信息后, 按任意键继续
       • 确认信息正确
       • 如果一切正确, 输入 Y 开始安装
（3）安装完毕后, 检查主机状态
      [root@master ~]# oc get nodes
（4）确认 openshift services 都正常启动了
      [root@master ~]# systemctl status openshift-master
      [root@master ~]# systemctl status openshift-node
      [root@node ~]# systemctl status openshift-node

1.5. 修改 OSE image streams
S2I 时, 会从 registry pull images, 默认指向的地址是 redhat private registry, 需要修改成实验环境下的 registry。
    [root@master ~]# oc edit is -n openshift
    会打开一个 vi 编辑器, 执行如下替换
    :%s/registry.access.redhat.com/workstation.pod0.example.com:5000

1.6. 使用 Apache htpasswd 作为 OSE 的认证方式
（1）安装 httpd-tools package
      [root@master ~]# yum -y install httpd-tools
（2）修改 /etc/openshift/master/master-config.yaml
      找到 oauthConfig 部分
      增加一个 file 属性 /etc/openshift/openshift-passwd
      kind 属性 替换 DenyAllPasswordIdentityProvider 为 HTPasswdPasswordIdentityProvider
（3）创建 /etc/openshift/openshift-passwd 文件
      [root@master ~]# touch /etc/openshift/openshift-passwd
（4）重启 openshift-master
      [root@master ~]# systemctl restart openshift-master

1.7. 发布 OSE router 和 internal registry
（1）配置 OSE master 使用正确的 DNS wild-card domain 名称
      [root@master ~]# vi /etc/openshift/master/master-config.yaml
        routingConfig
          subdomain: cloudapps0.example.com
      [root@master ~]# systemctl restart openshift-master
（2）创建 router service account
      [root@master ~]# echo '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"router"}}' | oc create -f -
（3）修改 privileged security context constraints (SCC), 加入 router service account
      [root@master ~]# oc edit scc privileged
      找到 users:，在其下面增加一个新用户 system:serviceaccount:default:router :
      users:
      - system:serviceaccount:default:router
      - system:serviceaccount:openshift-infra:build-controller
      注意：可以运行脚本 /root/DO280/labs/lab-review/router_scc.sh 创建 service account 并且加到 privileged SCC。
router_scc.sh 脚本内容如下：
#!/bin/bash

echo \
    '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"router"}}' \
    | oc create -f -
   
oc get scc privileged -o yaml > scc_privileged.yaml

sed -i '
/users:/ a \
- system:serviceaccount:default:router
' scc_privileged.yaml

oc replace scc privileged -f scc_privileged.yaml

（4）发布默认的 HAProxy router
      [root@master ~]# oadm router oserouter --credentials='/etc/openshift/master/openshift-router.kubeconfig' --images='openshift3/ose-haproxy-router:${version}' --service-account=router
      确认 router pod 运行起来后, 再执行下一步
      [root@master ~]# oc get pods
（5）发布 OSE internal container image registry
      [root@master ~]# oadm registry --credentials=/etc/openshift/master/openshift-registry.kubeconfig --images='openshift3/ose-docker-registry:${version}'
      确认 registry pod 运行起来后, 再执行下一步
      [root@master ~]# oc get pods

1.8. 为 OSE internal registry 提供持久化存储
（1）安装并配置 NFS Server（master）和 NFS  Client（node）
      [root@master ~]# bash /root/DO280/labs/lab-review/config-nfs-server.sh
config-nfs-server.sh 脚本内容如下：
#!/bin/bash

# OSE 3.0.1.0 packages already add nfs-utils as dependency
#yum -y install nfs-utils
#ssh root@node yum -y install nfs-utils

iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 111 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2049 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 20048 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 50825 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 53248 -j ACCEPT

sed -i '/COMMIT/ i \
# BEGIN NFS server \
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 53248 -j ACCEPT \
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 50825 -j ACCEPT \
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 20048 -j ACCEPT \
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2049 -j ACCEPT \
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 111 -j ACCEPT \
# END NFS server' /etc/sysconfig/iptables

# DO NOT restart iptables. Restarting it looses rules added by docker and openshift-sdn

sed -i '
s/RPCMOUNTDOPTS=""/RPCMOUNTDOPTS="-p 20048"/
s/STATDARG=""/STATDARG="-p 50825"/
' /etc/sysconfig/nfs

echo '
fs.nfs.nlm_tcpport=53248
fs.nfs.nlm_udpport=53248
' >> /etc/sysctl.conf

modprobe nfsd
sysctl -p

for u in rpcbind nfs-server nfs-lock nfs-idmap; do
    systemctl enable $u
    systemctl start $u
done

setsebool -P virt_use_nfs=true
ssh root@node setsebool -P virt_use_nfs=true
（2）发布 OSE internal container image registry
      [root@master ~]# mkdir -p /var/export/registryvol
      [root@master ~]# chown nfsnobody:nfsnobody /var/export/registryvol
      [root@master ~]# chmod 700 /var/export/registryvol
（3）Export 该目录
      [root@master ~]# vi /etc/exports
      增加一行内容如下:
      /var/export/registryvol *(rw,async,all_squash)
（4）允许客户端访问
      [root@master ~]# exportfs -a
（5）复制 PV 定义文件
      [root@master ~]# cp /root/DO280/labs/lab-review/registry-volume.json /root
      [root@master ~]# vi /root/registry-volume.json
      修改内容如下：
      "server": "master.pod0.example.com"
修改后的 registry-volume.json 的完整内容如下：
{
    "apiVersion": "v1",
    "kind": "PersistentVolume",
    "metadata": {
        "name": "registry-volume",
        "labels": {
            "deploymentconfig": "docker-registry"
        }
    },
    "spec": {
        "capacity": {
            "storage": "10Gi"
        },
        "accessModes": [ "ReadWriteMany" ],
        "nfs": {
            "path": "/var/export/registryvol",
            "server": "master.pod0.example.com"
        }
    }
}
       说明: OpenShift Enterprise Developer Guide 有 YAML 格式的 PV 定义。
（6）创建 PV
      [root@master ~]# oc create -f /root/registry-volume.json

  1.9. 绑定 PV 到 registry pod
（1）创建 PVClaim
      [root@master ~]# oc create -f /root/DO280/labs/lab-review/registry-pvclaim.json
registry-pvclaim.json 的完整内容如下：
{
    "apiVersion": "v1",
    "kind": "PersistentVolumeClaim",
    "metadata": {
        "name": "registry-pvclaim",
        "labels": {
            "deploymentconfig": "docker-registry"
        }
    },
    "spec": {
        "accessModes": [ "ReadWriteMany" ],
        "resources": {
            "requests": {
                "storage": "10Gi"
            }
        }
    }
}
（2）确认 PVClaim 绑定到了 PV
      [root@master ~]# oc get pvc
（3）修改 DeploymentConfig docker-registry to use the PVClaim:
      [root@master ~]# oc volume dc docker-registry --add --overwrite -t persistentVolumeClaim --claim-name=registry-pvclaim --name=registry-storage
（4）修改 docker-registry dc 后，确认 DeploymentConfig docker-registry 启动了一个新的 registry pod
      [root@master ~]# oc status
（5）确认 registry 工作正常
      [root@master ~]# oc describe service docker-registry
（6）访问 registry API，确认 registry pod 工作正常
      [root@master ~]# curl http://172.30.10.176:5000/healthz

2. 发布 Bookstore 应用

2.1. 创建用户 developer
（1）使用 htpasswd 创建用户
      [root@master ~]# htpasswd -b /etc/openshift/openshift-passwd developer openshift

2.2. 创建 project 并增加 quota 限制
（1）安装 OSE client (workstation 主机)
      [student@workstation ~]$ sudo yum -y install openshift
      输入口令 student
（2）以 developer 账户登录 (workstation 主机)
      [student@workstation ~]$ oc login -u developer -p openshift
      由于是第一次登录，会询问 master 的信息, 输入 https://master.pod0.example.com:8443
      并使用 insecure connections? (y/n): y
（3）创建 project bookstore (workstation 主机)
      [student@workstation ~]$ oc new-project bookstore
（4）为 project bookstore 增加限制 quota (master 主机)
      [root@master ~]# oc create -f /root/DO280/labs/lab-review/project-quotas.json -n bookstore
project-quotas.json 的完整内容如下：
{
    "apiVersion": "v1",
    "kind": "ResourceQuota",
    "metadata": {
        "name": "bookstore-project-quotas"
    },
    "spec": {
        "hard": {
            "memory": "16Gi",
            "cpu": "6000m",
            "pods": "6",
            "services": "8",
            "replicationcontrollers": "5",
            "persistentvolumeclaims": "2"
        }
    }
}
（5）为 project bookstore 增加限制 limit (master 主机)
      [root@master ~]# oc create -f /root/DO280/labs/lab-review/project-limits.json -n bookstore
project-limits.json 的完整内容如下：
{
    "kind": "LimitRange",
    "apiVersion": "v1",
    "metadata": {
        "name": "bookstore-project-limits"
    },
    "spec": {
        "limits": [
            {
                "type": "Pod",
                "max": {
                    "cpu": "500m",
                    "memory": "4Gi"
                },
                "min": {
                    "cpu": "200m",
                    "memory": "512Mi"
                }
            },
            {
                "type": "Container",
                "max": {
                    "cpu": "500m",
                    "memory": "4Gi"
                },
                "min": {
                    "cpu": "200m",
                    "memory": "512Mi"
                },
                "default": {
                    "cpu": "200m",
                    "memory": "1.5Gi"
                }
            }
        ]
    }
}

2.3. 查看并确认 application template 的参数 (全部在 workstation 主机)
（1）列出 template eap6-mysql-persistent-sti 的所有参数
      [student@workstation ~]$ oc process --parameters eap6-mysql-persistent-sti -n openshift | sort
输出如下：
DB_DATABASE                          Database name
DB_USERNAME                        Database user name
DB_PASSWORD                        Database user password
EAP_HTTPS_SECRET              The name of the secret containing the keystore file
EAP_HTTPS_KEYSTORE         The name of the keystore file within the secret
EAP_HTTPS_NAME                  The name associated with the server certificate
EAP_HTTPS_PASSWORD        The password for the keystore and certificate
GIT_URI                                     Git source URI for application
VOLUME_CAPACITY                Size of persistent storage for database volume
（2）导出 application template
      [student@workstation ~]$ oc export template eap6-mysql-persistent-sti -o json -n openshift > eap6-mysql-persistent-sti.json
（3）分析导出的 template 文件 eap6-mysql-persistent-sti.json
      [student@workstation ~]$ vim eap6-mysql-persistent-sti.json
      检查如下参数配置:
      ◦ 2 个 pods: 一个是 application pod，一个是 database pod, 分别在各自的 DeploymentConfig 中配置。
      ◦ 1 个 BuildConfig：用来从 source code 生成 application image。
      ◦ 1 个 ImageStream：用来生成 application image。
      ◦ 2 个 services：一个是 application，一个是 database。
      ◦ 1 个额外的 service：用做 JBoss EAP clustering。
      ◦ 2 个 routes：全都是为了 application pod service，一个用 HTTP，一个用 HTTPS。
      ◦ 1 个 PVClaim：用做 database data volume。

eap6-mysql-persistent-sti.json 完整内容如下：
{
    "kind": "Template",
    "apiVersion": "v1",
    "metadata": {
        "name": "eap6-mysql-persistent-sti",
        "creationTimestamp": null,
        "annotations": {
            "description": "Application template for EAP 6 MySQL applications with persistent storage built using STI.",
            "iconClass": "icon-jboss"
        }
    },
    "objects": [
        {
            "kind": "Service",
            "apiVersion": "v1",
            "spec": {
                "ports": [
                    {
                        "port": 8080,
                        "targetPort": 8080
                    }
                ],
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}"
                }
            },
            "metadata": {
                "name": "${APPLICATION_NAME}",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "The web server's http port."
                }
            }
        },
        {
            "kind": "Service",
            "apiVersion": "v1",
            "spec": {
                "ports": [
                    {
                        "port": 8443,
                        "targetPort": 8443
                    }
                ],
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}"
                }
            },
            "metadata": {
                "name": "secure-${APPLICATION_NAME}",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "The web server's https port."
                }
            }
        },
        {
            "kind": "Service",
            "apiVersion": "v1",
            "spec": {
                "ports": [
                    {
                        "port": 8888,
                        "targetPort": 8888
                    }
                ],
                "portalIP": "None",
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}"
                }
            },
            "metadata": {
                "name": "${APPLICATION_NAME}-ping",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "Ping service for clustered applications."
                }
            }
        },
        {
            "kind": "Service",
            "apiVersion": "v1",
            "spec": {
                "ports": [
                    {
                        "port": 3306,
                        "targetPort": 3306
                    }
                ],
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}-mysql"
                }
            },
            "metadata": {
                "name": "${APPLICATION_NAME}-mysql",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "The database server's port."
                }
            }
        },
        {
            "kind": "Route",
            "apiVersion": "v1",
            "id": "${APPLICATION_NAME}-http-route",
            "metadata": {
                "name": "${APPLICATION_NAME}-http-route",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "Route for application's http service."
                }
            },
            "spec": {
                "host": "${APPLICATION_HOSTNAME}",
                "to": {
                    "name": "${APPLICATION_NAME}"
                }
            }
        },
        {
            "kind": "Route",
            "apiVersion": "v1",
            "id": "${APPLICATION_NAME}-https-route",
            "metadata": {
                "name": "${APPLICATION_NAME}-https-route",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                },
                "annotations": {
                    "description": "Route for application's https service."
                }
            },
            "spec": {
                "host": "${APPLICATION_HOSTNAME}",
                "to": {
                    "name": "secure-${APPLICATION_NAME}"
                },
                "tls": {
                    "termination": "passthrough"
                }
            }
        },
        {
            "kind": "ImageStream",
            "apiVersion": "v1",
            "metadata": {
                "name": "${APPLICATION_NAME}",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                }
            }
        },
        {
            "kind": "BuildConfig",
            "apiVersion": "v1",
            "metadata": {
                "name": "${APPLICATION_NAME}",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                }
            },
            "spec": {
                "source": {
                    "type": "Git",
                    "git": {
                        "uri": "${GIT_URI}",
                        "ref": "${GIT_REF}"
                    },
                    "contextDir": "${GIT_CONTEXT_DIR}"
                },
                "strategy": {
                    "type": "Source",
                    "sourceStrategy": {
                        "from": {
                            "kind": "ImageStreamTag",
                            "namespace": "openshift",
                            "name": "jboss-eap6-openshift:${EAP_RELEASE}"
                        }
                    }
                },
                "output": {
                    "to": {
                        "kind": "ImageStreamTag",
                        "name": "${APPLICATION_NAME}:latest"
                    }
                },
                "triggers": [
                    {
                        "type": "GitHub",
                        "github": {
                            "secret": "${GITHUB_TRIGGER_SECRET}"
                        }
                    },
                    {
                        "type": "Generic",
                        "generic": {
                            "secret": "${GENERIC_TRIGGER_SECRET}"
                        }
                    },
                    {
                        "type": "ImageChange",
                        "imageChange": {}
                    }
                ]
            }
        },
        {
            "kind": "DeploymentConfig",
            "apiVersion": "v1",
            "metadata": {
                "name": "${APPLICATION_NAME}",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                }
            },
            "spec": {
                "strategy": {
                    "type": "Recreate"
                },
                "triggers": [
                    {
                        "type": "ImageChange",
                        "imageChangeParams": {
                            "automatic": true,
                            "containerNames": [
                                "${APPLICATION_NAME}"
                            ],
                            "from": {
                                "kind": "ImageStream",
                                "name": "${APPLICATION_NAME}"
                            }
                        }
                    }
                ],
                "replicas": 1,
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}"
                },
                "template": {
                    "metadata": {
                        "name": "${APPLICATION_NAME}",
                        "labels": {
                            "deploymentConfig": "${APPLICATION_NAME}",
                            "application": "${APPLICATION_NAME}"
                        }
                    },
                    "spec": {
                        "serviceAccount": "eap-service-account",
                        "containers": [
                            {
                                "name": "${APPLICATION_NAME}",
                                "image": "${APPLICATION_NAME}",
                                "imagePullPolicy": "Always",
                                "volumeMounts": [
                                    {
                                        "name": "eap-keystore-volume",
                                        "mountPath": "/etc/eap-secret-volume",
                                        "readOnly": true
                                    }
                                ],
                                "readinessProbe": {
                                    "exec": {
                                        "command": [
                                            "/bin/bash",
                                            "-c",
                                            "/opt/eap/bin/readinessProbe.sh"
                                        ]
                                    }
                                },
                                "ports": [
                                    {
                                        "name": "http",
                                        "containerPort": 8080,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "name": "https",
                                        "containerPort": 8443,
                                        "protocol": "TCP"
                                    },
                                    {
                                        "name": "ping",
                                        "containerPort": 8888,
                                        "protocol": "TCP"
                                    }
                                ],
                                "env": [
                                    {
                                        "name": "DB_SERVICE_PREFIX_MAPPING",
                                        "value": "${APPLICATION_NAME}-mysql=DB"
                                    },
                                    {
                                        "name": "DB_JNDI",
                                        "value": "${DB_JNDI}"
                                    },
                                    {
                                        "name": "DB_USERNAME",
                                        "value": "${DB_USERNAME}"
                                    },
                                    {
                                        "name": "DB_PASSWORD",
                                        "value": "${DB_PASSWORD}"
                                    },
                                    {
                                        "name": "DB_DATABASE",
                                        "value": "${DB_DATABASE}"
                                    },
                                    {
                                        "name": "TX_DATABASE_PREFIX_MAPPING",
                                        "value": "${APPLICATION_NAME}-mysql=DB"
                                    },
                                    {
                                        "name": "DB_MIN_POOL_SIZE",
                                        "value": "${DB_MIN_POOL_SIZE}"
                                    },
                                    {
                                        "name": "DB_MAX_POOL_SIZE",
                                        "value": "${DB_MAX_POOL_SIZE}"
                                    },
                                    {
                                        "name": "DB_TX_ISOLATION",
                                        "value": "${DB_TX_ISOLATION}"
                                    },
                                    {
                                        "name": "OPENSHIFT_DNS_PING_SERVICE_NAME",
                                        "value": "${APPLICATION_NAME}-ping"
                                    },
                                    {
                                        "name": "OPENSHIFT_DNS_PING_SERVICE_PORT",
                                        "value": "8888"
                                    },
                                    {
                                        "name": "EAP_HTTPS_KEYSTORE_DIR",
                                        "value": "/etc/eap-secret-volume"
                                    },
                                    {
                                        "name": "EAP_HTTPS_KEYSTORE",
                                        "value": "${EAP_HTTPS_KEYSTORE}"
                                    },
                                    {
                                        "name": "EAP_HTTPS_NAME",
                                        "value": "${EAP_HTTPS_NAME}"
                                    },
                                    {
                                        "name": "EAP_HTTPS_PASSWORD",
                                        "value": "${EAP_HTTPS_PASSWORD}"
                                    },
                                    {
                                        "name": "HORNETQ_CLUSTER_PASSWORD",
                                        "value": "${HORNETQ_CLUSTER_PASSWORD}"
                                    },
                                    {
                                        "name": "HORNETQ_QUEUES",
                                        "value": "${HORNETQ_QUEUES}"
                                    },
                                    {
                                        "name": "HORNETQ_TOPICS",
                                        "value": "${HORNETQ_TOPICS}"
                                    }
                                ]
                            }
                        ],
                        "volumes": [
                            {
                                "name": "eap-keystore-volume",
                                "secret": {
                                    "secretName": "${EAP_HTTPS_SECRET}"
                                }
                            }
                        ]
                    }
                }
            }
        },
        {
            "kind": "DeploymentConfig",
            "apiVersion": "v1",
            "metadata": {
                "name": "${APPLICATION_NAME}-mysql",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                }
            },
            "spec": {
                "strategy": {
                    "type": "Recreate"
                },
                "triggers": [
                    {
                        "type": "ImageChange",
                        "imageChangeParams": {
                            "automatic": true,
                            "containerNames": [
                                "${APPLICATION_NAME}-mysql"
                            ],
                            "from": {
                                "kind": "ImageStreamTag",
                                "namespace": "openshift",
                                "name": "mysql:latest"
                            }
                        }
                    }
                ],
                "replicas": 1,
                "selector": {
                    "deploymentConfig": "${APPLICATION_NAME}-mysql"
                },
                "template": {
                    "metadata": {
                        "name": "${APPLICATION_NAME}-mysql",
                        "labels": {
                            "deploymentConfig": "${APPLICATION_NAME}-mysql",
                            "application": "${APPLICATION_NAME}"
                        }
                    },
                    "spec": {
                        "containers": [
                            {
                                "name": "${APPLICATION_NAME}-mysql",
                                "image": "mysql",
                                "imagePullPolicy": "Always",
                                "ports": [
                                    {
                                        "containerPort": 3306,
                                        "protocol": "TCP"
                                    }
                                ],
                                "volumeMounts": [
                                    {
                                        "mountPath": "/var/lib/mysql/data",
                                        "name": "${APPLICATION_NAME}-mysql-pvol"
                                    }
                                ],
                                "env": [
                                    {
                                        "name": "MYSQL_USER",
                                        "value": "${DB_USERNAME}"
                                    },
                                    {
                                        "name": "MYSQL_PASSWORD",
                                        "value": "${DB_PASSWORD}"
                                    },
                                    {
                                        "name": "MYSQL_DATABASE",
                                        "value": "${DB_DATABASE}"
                                    },
                                    {
                                        "name": "MYSQL_LOWER_CASE_TABLE_NAMES",
                                        "value": "${MYSQL_LOWER_CASE_TABLE_NAMES}"
                                    },
                                    {
                                        "name": "MYSQL_MAX_CONNECTIONS",
                                        "value": "${MYSQL_MAX_CONNECTIONS}"
                                    },
                                    {
                                        "name": "MYSQL_FT_MIN_WORD_LEN",
                                        "value": "${MYSQL_FT_MIN_WORD_LEN}"
                                    },
                                    {
                                        "name": "MYSQL_FT_MAX_WORD_LEN",
                                        "value": "${MYSQL_FT_MAX_WORD_LEN}"
                                    },
                                    {
                                        "name": "MYSQL_AIO",
                                        "value": "${MYSQL_AIO}"
                                    }
                                ]
                            }
                        ],
                        "volumes": [
                            {
                                "name": "${APPLICATION_NAME}-mysql-pvol",
                                "persistentVolumeClaim": {
                                    "claimName": "${APPLICATION_NAME}-mysql-claim"
                                }
                            }
                        ]
                    }
                }
            }
        },
        {
            "apiVersion": "v1",
            "kind": "PersistentVolumeClaim",
            "metadata": {
                "name": "${APPLICATION_NAME}-mysql-claim",
                "labels": {
                    "application": "${APPLICATION_NAME}"
                }
            },
            "spec": {
                "accessModes": [
                    "ReadWriteOnce"
                ],
                "resources": {
                    "requests": {
                        "storage": "${VOLUME_CAPACITY}"
                    }
                }
            }
        }
    ],
    "parameters": [
        {
            "name": "EAP_RELEASE",
            "description": "EAP Release version, e.g. 6.4, etc.",
            "value": "6.4"
        },
        {
            "name": "APPLICATION_NAME",
            "description": "The name for the application.",
            "value": "eap-app"
        },
        {
            "name": "APPLICATION_HOSTNAME",
            "description": "Custom hostname for service routes.  Leave blank for default hostname, e.g.: \u003capplication-name\u003e.\u003cproject\u003e.\u003cdefault-domain-suffix\u003e"
        },
        {
            "name": "GIT_URI",
            "description": "Git source URI for application"
        },
        {
            "name": "GIT_REF",
            "description": "Git branch/tag reference",
            "value": "master"
        },
        {
            "name": "GIT_CONTEXT_DIR",
            "description": "Path within Git project to build; empty for root project directory."
        },
        {
            "name": "DB_JNDI",
            "description": "Database JNDI name used by application to resolve the datasource, e.g. java:/jboss/datasources/mysql"
        },
        {
            "name": "DB_DATABASE",
            "description": "Database name",
            "value": "root"
        },
        {
            "name": "VOLUME_CAPACITY",
            "description": "Size of persistent storage for database volume.",
            "value": "512Mi"
        },
        {
            "name": "HORNETQ_QUEUES",
            "description": "Queue names"
        },
        {
            "name": "HORNETQ_TOPICS",
            "description": "Topic names"
        },
        {
            "name": "EAP_HTTPS_SECRET",
            "description": "The name of the secret containing the keystore file",
            "value": "eap-app-secret"
        },
        {
            "name": "EAP_HTTPS_KEYSTORE",
            "description": "The name of the keystore file within the secret",
            "value": "keystore.jks"
        },
        {
            "name": "EAP_HTTPS_NAME",
            "description": "The name associated with the server certificate"
        },
        {
            "name": "EAP_HTTPS_PASSWORD",
            "description": "The password for the keystore and certificate"
        },
        {
            "name": "DB_MIN_POOL_SIZE",
            "description": "Sets xa-pool/min-pool-size for the configured datasource."
        },
        {
            "name": "DB_MAX_POOL_SIZE",
            "description": "Sets xa-pool/max-pool-size for the configured datasource."
        },
        {
            "name": "DB_TX_ISOLATION",
            "description": "Sets transaction-isolation for the configured datasource."
        },
        {
            "name": "MYSQL_LOWER_CASE_TABLE_NAMES",
            "description": "Sets how the table names are stored and compared."
        },
        {
            "name": "MYSQL_MAX_CONNECTIONS",
            "description": "The maximum permitted number of simultaneous client connections."
        },
        {
            "name": "MYSQL_FT_MIN_WORD_LEN",
            "description": "The minimum length of the word to be included in a FULLTEXT index."
        },
        {
            "name": "MYSQL_FT_MAX_WORD_LEN",
            "description": "The maximum length of the word to be included in a FULLTEXT index."
        },
        {
            "name": "MYSQL_AIO",
            "description": "Controls the innodb_use_native_aio setting value if the native AIO is broken."
        },
        {
            "name": "HORNETQ_CLUSTER_PASSWORD",
            "description": "HornetQ cluster admin password",
            "generate": "expression",
            "from": "[a-zA-Z0-9]{8}"
        },
        {
            "name": "DB_USERNAME",
            "description": "Database user name",
            "generate": "expression",
            "from": "user[a-zA-Z0-9]{3}"
        },
        {
            "name": "DB_PASSWORD",
            "description": "Database user password",
            "generate": "expression",
            "from": "[a-zA-Z0-9]{8}"
        },
        {
            "name": "GITHUB_TRIGGER_SECRET",
            "description": "Github trigger secret",
            "generate": "expression",
            "from": "[a-zA-Z0-9]{8}"
        },
        {
            "name": "GENERIC_TRIGGER_SECRET",
            "description": "Generic build trigger secret",
            "generate": "expression",
            "from": "[a-zA-Z0-9]{8}"
        }
    ],
    "labels": {
        "template": "eap6-mysql-persistent-sti"
    }
}

2.4. 为数据库提供持久化存储
（1）创建一个目录, 只有用户和组 nfsnobody 对其拥有全部权限:
        [root@master ~]# mkdir -p /var/export/bookstoredbvol
        [root@master ~]# chown nfsnobody:nfsnobody /var/export/bookstoredbvol
        [root@master ~]# chmod 700 /var/export/bookstoredbvol
（2）export 该目录
        [root@master ~]# vi /etc/exports
          /var/export/bookstoredbvol *(rw,async,all_squash)
（3）允许客户端使用 NFS share:
        [root@master ~]# exportfs -a
（4）复制 PV 定义文件到 root 用户主目录下, 并修改复制后的文件
        [root@master ~]# cp /root/DO280/labs/lab-review/bookstoredb-volume.json /root/
          "server": "master.pod0.example.com"
修改后的 bookstoredb-volume.json 的完整内容如下：
{
    "apiVersion": "v1",
    "kind": "PersistentVolume",
    "metadata": {
        "name": "bookstoredb-volume",
        "labels": {
            "application": "bookstore"
        }
    },
    "spec": {
        "capacity": {
            "storage": "1Gi"
        },
        "accessModes": [ "ReadWriteOnce" ],
        "nfs": {
            "path": "/var/export/bookstoredbvol",
            "server": "master.podX.example.com"
        }
    }
}
（5）注意 PV size 要和 template VOLUME_SIZE 参数相匹配:
        "capacity": {
          "storage": "1Gi"
        },
（6）创建 PV:
        [root@master ~]# oc create -f /root/bookstoredb-volume.json

2.5. 创建一个 secret 用于 application SSL certificate (全部在 workstation 主机)
（1）创建 secret:
        [student@workstation ~]$ oc create -f /home/student/DO280/labs/lab-review/secret.json
        说明: 手册中的 secret 定义文件使用的是 YAML 格式。
secret.json 完整内容如下：
{
      "kind": "List",
      "apiVersion": "v1",
      "metadata": {},
      "items": [
          {
              "kind": "ServiceAccount",
              "apiVersion": "v1",
              "metadata": {
                  "name": "eap-service-account"
              },
              "secrets": [
                  {
                      "name": "eap-app-secret-member"
                  }
              ]
          },
          {
              "kind": "Secret",
              "apiVersion": "v1",
              "metadata": {
                  "annotations": {
                      "description": "Default secret file with name 'jboss' and password 'mykeystorepass'"
                  },
                  "name": "eap-app-secret-member"
              },
              "data": {
                  "keystore.jks": "/u3+7QAAAAIAAAABAAAAAQAFamJvc3MAAAFNbVtLLAAABQMwggT/MA4GCisGAQQBKgIRAQEFAASCBOsxl4wqa+E+XP8+qMZY9XLhvKrRX8V1MHdwFZQaLTEVURCizqYXoMnbhtfV0oMAUFsE7013TTA9Q2l+pSs+cqz6HH/vwjEEIkqJx5wD8WcD/bu9e9F9EHQ+zrjZFmpMFvXsvj9+ux1o/YLBDGY3kd4MoDcJy0yJ/ZpzNYLkXanlrMhWqxC7MAliCBsdyVgNn5RFb4Nn+JZgJuNSIGo/K292+0IFaFv9vsXbX889W9HPCvfO0mQIzoy8In0NhzdKli/67y4kbDkWaI0fRONckZTxNpxn6rMc0nN9zKrGVToLxj1Ufcoj/tCvR8agtPpv7KIWUqBYDg83ad+i4EE5XYISovlsl6RmtrrTb39PJcL86+wJ+x2ZrLuyzh6C9sAOdSBiKt/DY97ICIYltRMrb+cNwWdnJvT+PeYvv3vKo7YThha+akoJDjsWMp1HWpbIC9zg9ZjugU+/ao6nHtmoZmCaYjLuEE+sYl5s179uyQjE3LRc+0cVY2+bYCOD6P6JLH9GdfjkR40OhjryiWy2Md6vAGaATh6kjjreRHfSie4KCgIZx9Ngb1+uAwauYSM8d9OIwT5lRmLd4Go9CaFXtFdq/IZv3x5ZEPVqMjxcq0KXcs1QcfK3oSYL/rrkxXxKFTrd0N3KgvwATWx/KS90tdHBg65dF3PpBjK1AYQL3Q7KV3t45SVyYHd92TUsaduY1nUQk4TukNC8l9f8xYVeOFXoFHZRx9edqn8fjDMmCYn5PTPNuMPHQm7nKxeWhV2URY5jt774gmvHLNcXeEgrM7US81wOvs2y1jY/paJWn+OACf2x2a75MWFFkZH67bZoh9pPWAwOUEtegXTL5QVicHjzZrop8Qb7K7hlGgD0RP5YYOFYF4DD+SL5BHKr6fw/LS6MMJaK1wKsJd0oGg9HcHXjph9Kb+mqXrQ54C1KI42LpFftU3DCg8wGoqvg/zO/UtVeHX3rBZDUIkeQrCULEkki9oL5diDxe9mNx9Qua5FJ6FJGIffQmsC4b0+Xys6NyqUu1aeWLcAPA/5hcs6ZTiSRTHTBe3vxapyBjnAL5uij4ILbWbEGH1e0mAHBeiihRx+w4oxH4OGCvXOhwIDHETLJJUcnJe1CouECdqdfVy/eEsIfiEheVs8OwogJLiWgzB7PoebXM4SKsAWL3NcDtC1LV3KuPgFuTDH7MjPIR83eSxkKlJLMNGfEpUHyg+lm7aJ98PVIS+l1YV9oUzLfbo3S6S2sMjVgyviS90vNIPo5JOTEFHsg5aWJNHL0OV4zRUeILzwwdQz+VkTk9DobnkLWUeLnwUNWheOpaQh79Mk0IfwfLj4D0Vx9p+PShKKZCGs0wjckmCFBM5Pc1x2lwMdaP5yATzrw+jUc+/3UY4PF/4Ya66m/DRsBKEcXjVAHcTce6OdNdGlBNT8VgkxPiylwO8hvyvpf6j+wdb9iXi6eOnk0AiEJ6mUAXs/eyDD/cqQjnUBKRGLQUSdHhvtpw8RfvyVhAAxNOnBsOT0WYol9iK6pSclGTF5mZleASRzZhH69GgdebfFhXimb0j/wYj3uLgf6mrKMDwlrXJ80SiWkXxd5TX/7XtB9lbPzNpaR12M8U8UVg16VOtMwCR2Gss2vmhqQnQFLsUsAKcYM0TRp1pWqbzpGebCvJkVWiIYocN3ZI1csAhGX3G86ewAAAAEABVguNTA5AAADeTCCA3UwggJdoAMCAQICBGekovEwDQYJKoZIhvcNAQELBQAwazELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAk5DMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYDVQQKEw1teWNvbXBhbnkuY29tMRQwEgYDVQQLEwtFbmdpbmVlcmluZzEPMA0GA1UEAxMGanNtaXRoMB4XDTE1MDUxOTE4MDYxOFoXDTE1MDgxNzE4MDYxOFowazELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAk5DMRAwDgYDVQQHEwdSYWxlaWdoMRYwFAYDVQQKEw1teWNvbXBhbnkuY29tMRQwEgYDVQQLEwtFbmdpbmVlcmluZzEPMA0GA1UEAxMGanNtaXRoMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk0zbGtem+If//jw0OTszIcpX4ydOCC0PeqktulYkm4pG0qEVBB+HuMj7yeTBc1KCDl2xm+Q6LPeTzUufk7BXFEg4Ru1l3PSW70LyJBfHy5ns0dYE5M1I0Avv9rvjgC1VTsiBmdXh+tIIQDPknIKpWpcs79XPOURGLvuGjfyj08EZWFvAZzYrk3lKwkceDHpYYb5i+zxFRz5K6of/h9gQ9CzslqNd7uxxvyy/yTtNFk2J797Vk3hKtbiATqc9+egEHcEQrzADejPYol5ke3DA1NPRBqFGku5n215i2eYzYvVV1xmifID/3lzvNWN0bWlOxl74VsPnWa/2JPP3hZ6p5QIDAQABoyEwHzAdBgNVHQ4EFgQURLJKk/gaSrMjDyX8iYtCzPtTBqAwDQYJKoZIhvcNAQELBQADggEBAA4ESTKsWevv40hFv11t+lGNHT16u8Xk+WnvB4Ko5sZjVhvRWTTKOEBE5bDYfMhf0esn8gg0B4Qtm4Rb5t9PeaG/0d6xxD0BIV6eWihJVtEGOH47Wf/UzfC88fqoIxZ6MMBPik/WeafvOK+HIHfZSwAmqlXgl4nNVDdMNHtBhNAvikL3osxrSbqdi3eyI7rqSpb41Lm9v+PF+vZTOGRQf22Gq30/Ie85DlqugtRKimWHJYL2HeL4ywTtQKgde6JDRCOHwbDcsl6CbMjugt3yyI7Yo9EJdKb5p6YoVOpnCz7369W9Uim+Xrl2ELZWM5WTiQFxd6S36Ql2TUk+s8zj/GoN9ov0Y/yNNCxAibwyzo94N+Q4vA=="
              }
          }
      ]
}

2.6. 根据导出的 template 创建 application (全部在 workstation 主机)
（1）为 oc process 命令提供参数
        [student@workstation ~]$ vi /home/student/DO280/labs/lab-review/process-template.sh
          #!/bin/bash
          # do not add nor remove whitespace from the following command:
          oc process -f eap6-mysql-persistent-sti.json -v \
          APPLICATION_NAME=bookstore,\
          APPLICATION_HOSTNAME=bookstore.cloudapps0.example.com,\
          GIT_URI=http://workstation.pod0.example.com/bookstore,\
          DB_JNDI=java:/jboss/datasources/mysql,\
          DB_DATABASE=bookstoredb,\
          DB_USERNAME=bookstoreapp,\
          DB_PASSWORD=secret,\
          EAP_HTTPS_SECRET=eap-app-secret-member,\
          EAP_HTTPS_KEYSTORE=keystore.jks,\
          EAP_HTTPS_NAME=jboss,\
          EAP_HTTPS_PASSWORD=mykeystorepass,\
          VOLUME_CAPACITY=1Gi \
          > processed-template.json
        [student@workstation ~]$ cp /home/student/DO280/labs/lab-review/process-template.sh .
（2）运行 oc process 命令产生资源定义文件
        [student@workstation ~]$ bash process-template.sh

生成后的 processed-template.json 的完整内容如下：
{
    "kind": "List",
    "apiVersion": "v1beta3",
    "metadata": {},
    "items": [
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "description": "The web server's http port."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore"
            },
            "spec": {
                "ports": [
                    {
                        "port": 8080,
                        "targetPort": 8080
                    }
                ],
                "selector": {
                    "deploymentConfig": "bookstore"
                }
            }
        },
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "description": "The web server's https port."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "secure-bookstore"
            },
            "spec": {
                "ports": [
                    {
                        "port": 8443,
                        "targetPort": 8443
                    }
                ],
                "selector": {
                    "deploymentConfig": "bookstore"
                }
            }
        },
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "description": "Ping service for clustered applications."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-ping"
            },
            "spec": {
                "portalIP": "None",
                "ports": [
                    {
                        "port": 8888,
                        "targetPort": 8888
                    }
                ],
                "selector": {
                    "deploymentConfig": "bookstore"
                }
            }
        },
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "description": "The database server's port."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-mysql"
            },
            "spec": {
                "ports": [
                    {
                        "port": 3306,
                        "targetPort": 3306
                    }
                ],
                "selector": {
                    "deploymentConfig": "bookstore-mysql"
                }
            }
        },
        {
            "apiVersion": "v1",
            "id": "bookstore-http-route",
            "kind": "Route",
            "metadata": {
                "annotations": {
                    "description": "Route for application's http service."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-http-route"
            },
            "spec": {
                "host": "bookstore.cloudapps0.example.com",
                "to": {
                    "name": "bookstore"
                }
            }
        },
        {
            "apiVersion": "v1",
            "id": "bookstore-https-route",
            "kind": "Route",
            "metadata": {
                "annotations": {
                    "description": "Route for application's https service."
                },
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-https-route"
            },
            "spec": {
                "host": "bookstore.cloudapps0.example.com",
                "tls": {
                    "termination": "passthrough"
                },
                "to": {
                    "name": "secure-bookstore"
                }
            }
        },
        {
            "apiVersion": "v1",
            "kind": "ImageStream",
            "metadata": {
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore"
            }
        },
        {
            "apiVersion": "v1",
            "kind": "BuildConfig",
            "metadata": {
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore"
            },
            "spec": {
                "output": {
                    "to": {
                        "kind": "ImageStreamTag",
                        "name": "bookstore:latest"
                    }
                },
                "source": {
                    "contextDir": "",
                    "git": {
                        "ref": "master",
                        "uri": "http://workstation.pod0.example.com/bookstore"
                    },
                    "type": "Git"
                },
                "strategy": {
                    "sourceStrategy": {
                        "from": {
                            "kind": "ImageStreamTag",
                            "name": "jboss-eap6-openshift:6.4",
                            "namespace": "openshift"
                        }
                    },
                    "type": "Source"
                },
                "triggers": [
                    {
                        "github": {
                            "secret": "xwrWajXC"
                        },
                        "type": "GitHub"
                    },
                    {
                        "generic": {
                            "secret": "doVsDd7R"
                        },
                        "type": "Generic"
                    },
                    {
                        "imageChange": {},
                        "type": "ImageChange"
                    }
                ]
            }
        },
        {
            "apiVersion": "v1",
            "kind": "DeploymentConfig",
            "metadata": {
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore"
            },
            "spec": {
                "replicas": 1,
                "selector": {
                    "deploymentConfig": "bookstore"
                },
                "strategy": {
                    "type": "Recreate"
                },
                "template": {
                    "metadata": {
                        "labels": {
                            "application": "bookstore",
                            "deploymentConfig": "bookstore"
                        },
                        "name": "bookstore"
                    },
                    "spec": {
                        "containers": [
                            {
                                "env": [
                                    {
                                        "name": "DB_SERVICE_PREFIX_MAPPING",
                                        "value": "bookstore-mysql=DB"
                                    },
                                    {
                                        "name": "DB_JNDI",
                                        "value": "java:/jboss/datasources/mysql"
                                    },
                                    {
                                        "name": "DB_USERNAME",
                                        "value": "bookstoreapp"
                                    },
                                    {
                                        "name": "DB_PASSWORD",
                                        "value": "secret"
                                    },
                                    {
                                        "name": "DB_DATABASE",
                                        "value": "bookstoredb"
                                    },
                                    {
                                        "name": "TX_DATABASE_PREFIX_MAPPING",
                                        "value": "bookstore-mysql=DB"
                                    },
                                    {
                                        "name": "DB_MIN_POOL_SIZE",
                                        "value": ""
                                    },
                                    {
                                        "name": "DB_MAX_POOL_SIZE",
                                        "value": ""
                                    },
                                    {
                                        "name": "DB_TX_ISOLATION",
                                        "value": ""
                                    },
                                    {
                                        "name": "OPENSHIFT_DNS_PING_SERVICE_NAME",
                                        "value": "bookstore-ping"
                                    },
                                    {
                                        "name": "OPENSHIFT_DNS_PING_SERVICE_PORT",
                                        "value": "8888"
                                    },
                                    {
                                        "name": "EAP_HTTPS_KEYSTORE_DIR",
                                        "value": "/etc/eap-secret-volume"
                                    },
                                    {
                                        "name": "EAP_HTTPS_KEYSTORE",
                                        "value": "keystore.jks"
                                    },
                                    {
                                        "name": "EAP_HTTPS_NAME",
                                        "value": "jboss"
                                    },
                                    {
                                        "name": "EAP_HTTPS_PASSWORD",
                                        "value": "mykeystorepass"
                                    },
                                    {
                                        "name": "HORNETQ_CLUSTER_PASSWORD",
                                        "value": "CxVGPLVP"
                                    },
                                    {
                                        "name": "HORNETQ_QUEUES",
                                        "value": ""
                                    },
                                    {
                                        "name": "HORNETQ_TOPICS",
                                        "value": ""
                                    }
                                ],
                                "image": "bookstore",
                                "imagePullPolicy": "Always",
                                "name": "bookstore",
                                "ports": [
                                    {
                                        "containerPort": 8080,
                                        "name": "http",
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 8443,
                                        "name": "https",
                                        "protocol": "TCP"
                                    },
                                    {
                                        "containerPort": 8888,
                                        "name": "ping",
                                        "protocol": "TCP"
                                    }
                                ],
                                "readinessProbe": {
                                    "exec": {
                                        "command": [
                                            "/bin/bash",
                                            "-c",
                                            "/opt/eap/bin/readinessProbe.sh"
                                        ]
                                    }
                                },
                                "volumeMounts": [
                                    {
                                        "mountPath": "/etc/eap-secret-volume",
                                        "name": "eap-keystore-volume",
                                        "readOnly": true
                                    }
                                ]
                            }
                        ],
                        "serviceAccount": "eap-service-account",
                        "volumes": [
                            {
                                "name": "eap-keystore-volume",
                                "secret": {
                                    "secretName": "eap-app-secret-member"
                                }
                            }
                        ]
                    }
                },
                "triggers": [
                    {
                        "imageChangeParams": {
                            "automatic": true,
                            "containerNames": [
                                "bookstore"
                            ],
                            "from": {
                                "kind": "ImageStream",
                                "name": "bookstore"
                            }
                        },
                        "type": "ImageChange"
                    }
                ]
            }
        },
        {
            "apiVersion": "v1",
            "kind": "DeploymentConfig",
            "metadata": {
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-mysql"
            },
            "spec": {
                "replicas": 1,
                "selector": {
                    "deploymentConfig": "bookstore-mysql"
                },
                "strategy": {
                    "type": "Recreate"
                },
                "template": {
                    "metadata": {
                        "labels": {
                            "application": "bookstore",
                            "deploymentConfig": "bookstore-mysql"
                        },
                        "name": "bookstore-mysql"
                    },
                    "spec": {
                        "containers": [
                            {
                                "env": [
                                    {
                                        "name": "MYSQL_USER",
                                        "value": "bookstoreapp"
                                    },
                                    {
                                        "name": "MYSQL_PASSWORD",
                                        "value": "secret"
                                    },
                                    {
                                        "name": "MYSQL_DATABASE",
                                        "value": "bookstoredb"
                                    },
                                    {
                                        "name": "MYSQL_LOWER_CASE_TABLE_NAMES",
                                        "value": ""
                                    },
                                    {
                                        "name": "MYSQL_MAX_CONNECTIONS",
                                        "value": ""
                                    },
                                    {
                                        "name": "MYSQL_FT_MIN_WORD_LEN",
                                        "value": ""
                                    },
                                    {
                                        "name": "MYSQL_FT_MAX_WORD_LEN",
                                        "value": ""
                                    },
                                    {
                                        "name": "MYSQL_AIO",
                                        "value": ""
                                    }
                                ],
                                "image": "mysql",
                                "imagePullPolicy": "Always",
                                "name": "bookstore-mysql",
                                "ports": [
                                    {
                                        "containerPort": 3306,
                                        "protocol": "TCP"
                                    }
                                ],
                                "volumeMounts": [
                                    {
                                        "mountPath": "/var/lib/mysql/data",
                                        "name": "bookstore-mysql-pvol"
                                    }
                                ]
                            }
                        ],
                        "volumes": [
                            {
                                "name": "bookstore-mysql-pvol",
                                "persistentVolumeClaim": {
                                    "claimName": "bookstore-mysql-claim"
                                }
                            }
                        ]
                    }
                },
                "triggers": [
                    {
                        "imageChangeParams": {
                            "automatic": true,
                            "containerNames": [
                                "bookstore-mysql"
                            ],
                            "from": {
                                "kind": "ImageStreamTag",
                                "name": "mysql:latest",
                                "namespace": "openshift"
                            }
                        },
                        "type": "ImageChange"
                    }
                ]
            }
        },
        {
            "apiVersion": "v1",
            "kind": "PersistentVolumeClaim",
            "metadata": {
                "labels": {
                    "application": "bookstore",
                    "template": "eap6-mysql-persistent-sti"
                },
                "name": "bookstore-mysql-claim"
            },
            "spec": {
                "accessModes": [
                    "ReadWriteOnce"
                ],
                "resources": {
                    "requests": {
                        "storage": "1Gi"
                    }
                }
            }
        }
    ]
}

（3）发布 application
        [student@workstation ~]$ oc create -f processed-template.json

  2.7. 等待直到 application build 并且运行，这一步等待时间较长，请耐心
（1）确认 build 启动
        [student@workstation ~]$ oc get builds
（2）查看 build 日志:
        [student@workstation ~]$ oc build-logs bookstore-1
（3）确认 application 和 database pods 启动:
        [student@workstation ~]$ oc get pods
        注意: REASON ExitCode:0 表示 build 成功。

  2.8. 确认 application 没有启动成功, 因为数据库没有数据
（1）查看 application pod 日志
        [student@workstation ~]$ oc logs bookstore-1-m6ynh | tail -n 6
        [student@workstation ~]$ oc logs bookstore-1-m6ynh | less

  2.9. 为数据库增加数据
（1）转发本地端口到 database pod:
        [student@workstation ~]$ oc port-forward -p bookstore-mysql-1-28zyg 13306:3306
（2）新开一个终端, 安装 MySQL client:
        [student@workstation ~]$ sudo yum -y install mysql
（3）git 克隆, 获取 SQL script:
        [student@workstation ~]$ git clone http://workstation.pod0.example.com/bookstore
（4）运行 SQL script
        [student@workstation ~]$ mysql -h127.0.0.1 -ubookstoreapp -psecret -P13306 bookstoredb < bookstore/sql/bookstore.sql
（5）确认 database volume 包含有所有 tables
        [root@master ~]# ls /var/export/bookstoredbvol/bookstoredb/
（6）停止本地端口转发
Ctrl + C 终止另一个终端运行的 oc port-forward 进程。

2.10. 强制 OSE 创建一个新的 application pod, 并确认启动成功
（1）删除原来失败的 application pod:
        [student@workstation ~]$ oc delete pod bookstore-1-m6ynh
（2）确认一个新的 pod 生成, 并等待其启动成功
        [student@workstation ~]$ oc get pod
（3）查看 application pod 日志, 确认连接数据库成功
        [student@workstation ~]$ oc logs bookstore-1-wp81p | tail -n 5

3. 访问 Bookstore Application

3.1. 确认 application route 到正确的主机
（1）template 参数 APPLICATION_URL
        [student@workstation ~]$ oc describe route bookstore
（2）如果 route host 属性不正确, 可以使用 oc edit 修改

3.2. 访问 application
打开浏览器, 访问 bookstore.cloudapps0.example.com