OpenShift_003：离线安装 OSE 3.1 之二：安装 YUM 源及第三方服务

环境：OS X EI Capitan 10.11.3 + Open Shift Enterprise 3.1

1. 网络、主机名、安全配置
(1) 设定固定 IP 地址：vim /etc/sysconfig/network-scripts/ifcfg-enp0s3
TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=enp0s3
UUID=8fe5a4c7-fbcd-4218-8e2f-31ec4e956fa4
DEVICE=enp0s3
ONBOOT=yes
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPADDR=192.168.56.105
NETMASK=255.255.255.0
GATEWAY=192.168.56.105
(2) 设定 hostname
hostnamectl set-hostname yum.example.com
(3) 设定 selinux 安全级别为 permissive
sed -i 's/=enforcing/=permissive/g' /etc/selinux/config

2. 安装和配置 yum 源
（1）创建本地 yum 源
tar zxvf /media/sf_Share/ose3.1/ose31-yum.tar.gz -C /opt/
（2）安装 createrepo 等工具
cd /opt/ose/rhel-7-server-rpms/Packages/;
rpm -ivh deltarpm-3.6-3.el7.x86_64.rpm;
rpm -ivh python-deltarpm-3.6-3.el7.x86_64.rpm;
rpm -ivh createrepo-0.9.9-23.el7.noarch.rpm;
（3）创建可识别的 yum 源
cd /opt/ose/;
createrepo .;
chmod 777 -R /opt/ose;
（4）指定本地 yum 源文件路径
cat << EOF > /etc/yum.repos.d/local.repo
[local]
baseurl = file:///opt/ose
gpgcheck = 0
enabled = 1
EOF
（5）安装 httpd
yum install -y httpd;
cat << EOF > /etc/httpd/conf.d/yum.conf
Alias /repo "/opt/ose"

  Options +Indexes +FollowSymLinks
Require all granted

SetHandler None
EOF
（6）使 yum 源可以被访问
systemctl disable firewalld;
systemctl stop  firewalld;
systemctl enable httpd;
systemctl restart httpd;
（7）在其它机器上（master，node1，node2）测试确认 yum 源可以被访问

3. 复制 docker images 和 应用程序
（1）复制 docker images
mkdir /opt/images;
cd /opt/images;
cp /media/sf_Share/ose3.1/docker-images/*.tar .
（2）复制应用程序
mkdir /opt/software;
cd /opt/software;
cp /media/sf_Share/ose3.1/*.zip .;
cp /media/sf_Share/ose3.1/*.json .;
cp -r /media/sf_Share/ose3.1/customized .;

4. 安装 docker 和相应软件
(1) 安装基础软件包 （如果没有 yum 源，需要先按照 5.2 安装和配置 yum 源）
yum -y remove NetworkManager*;
yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion vim lrzsz unzip;
yum -y update;
yum -y install atomic-openshift-utils;
reboot;
(2) 安装 docker
yum -y install docker;
systemctl enable docker;
systemctl stop docker;
(3) 修改 docker registry 地址，蓝色字体根据实际地址修改
cp /etc/sysconfig/docker /etc/sysconfig/docker.bak.$(date "+%Y%m%d%H%M%S");
sed  -i s/".*--selinux-enabled.*"/"OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0\/16 --insecure-registry registry.example.com:5000'"/g /etc/sysconfig/docker;
sed -i 's/registry.access.redhat.com/registry.example.com:5000/g' /etc/sysconfig/docker;
(4) 修改 docker 存储
cat < /etc/sysconfig/docker-storage-setup
VG=rhel
SETUP_LVM_THIN_POOL=yes
EOF
docker-storage-setup;
输出如下：// TODO 这个有没有问题？
Rounding up size to full physical extent 52.00 MiB
Volume group "rhel" has insufficient free space (11 extents): 13 required.
(4) 将 docker 的容器存储部分挂到另一个盘，蓝色字体根据实际设备修改（本次安装未执行此步）
vgextend rhel  /dev/vdb
lvextend -l +100%FREE /dev/rhel/docker-pool
(5) 重启 docker
systemctl start docker;

5. 安装和配置 dns server
（1）安装 bind 相关包
yum -y install bind* ;
systemctl enable named ;
（2）修改 dns 访问配置
cp /etc/named.conf /etc/named.conf.bak.$(date "+%Y%m%d%H%M%S");
sed -i s/"listen-on port 53 { 127.0.0.1; };"/"listen-on port 53 { any; };"/g /etc/named.conf;
sed -i s/"listen-on-v6 port 53 { ::1; };"/"listen-on port 53 { any; };"/g /etc/named.conf;
sed -i s/"allow-query     { localhost; };"/"allow-query     { any; };"/g /etc/named.conf;
sed -i '/rfc1912/i\zone "apps.example.com" IN { type master; file "dynamic/apps.example.com.db"; };' /etc/named.conf
sed -i '/rfc1912/i\zone "example.com" IN { type master; file "dynamic/example.com.db"; };' /etc/named.conf
（3）修改各主机别名，蓝色字体根据实际机器 IP 地址修改
cat << EOF > /var/named/dynamic/example.com.db
\$ORIGIN .
\$TTL 1 ; 1 sec
example.com     IN  SOA  ns1.example.com. hostmaster.example.com. (
                2011112904 ; serial
                60         ; refresh
                15         ; retry
                1800       ; expire
                10         ; minimum
           )
       NS ns1.example.com.
       MX 10 mail.example.com.
\$ORIGIN example.com.
ns1       A 192.168.56.105
master    A 192.168.56.106
node1     A 192.168.56.107
node2     A 192.168.56.108
registry    A 192.168.56.105
git        A 192.168.56.105
yum        A 192.168.156.105

EOF
（4）修改各主机别名，蓝色字体根据实际机器 IP 地址修改
cat << EOF > /var/named/dynamic/apps.example.com.db
\$ORIGIN .
\$TTL 1 ; 1 sec
apps.example.com     IN  SOA  ns1.apps.example.com. hostmaster.apps.example.com. (
                2011112904 ; serial
                60         ; refresh
                15         ; retry
                1800       ; expire
                10         ; minimum
           )
       NS ns1.apps.example.com.
       MX 10 mail.apps.example.com.
\$ORIGIN apps.example.com.
*        A 192.168.56.107
EOF
systemctl restart named;
（5）配置 iptables
cp /etc/sysconfig/iptables /etc/sysconfig/iptables.bak.$(date "+%Y%m%d%H%M%S");
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 53 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p udp -m state --state NEW -m udp --dport 53 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT' /etc/sysconfig/iptables;
systemctl enable iptables
systemctl restart iptables;
(6) 在其它3台机器（master，node1，node2）测试，确认 dns 解析成功

6. 安装和配置 docker private registry
在 registry.example.com 所在的机器上安装，这里 registry.example.com 机器就是 yum.example.com 机器。
(1) 安装 docker registry 包
yum -y install docker-registry;
systemctl enable docker-registry;
systemctl start docker-registry;
(2) 把之前备份的本地 docker images 文件导入
cd /opt/images/;
for i in `ls` ; do docker load < $i ; done;
docker images;
(3) 把本地 docker images push 到仓库
REDHAT_REG="registry.access.redhat.com";
PRIVATE_REG="registry.example.com:5000";
for i in $(docker images|grep $REDHAT_REG|awk '{print $1":"$2}') ; do docker tag  $i "$PRIVATE_REG$(echo $i|awk -F 'com' {'print $2'})" ; done;
for i in `docker images|grep $PRIVATE_REG|awk '{print $1}'` ; do  docker push $i; done;
(4) 在其它3台机器（master，node1，node2）上测试，确认可以从 docker private registry 把 docker image pull 下来
 
7. 安装和配置 git server
在 git.example.com 所在的机器上安装，这里 git.example.com 机器就是 yum.example.com 机器。
(1) 初始化仓库
mkdir -p /opt/git/repo/cakephp.git;
cd /opt/git/repo/cakephp.git;
git init --bare;
git update-server-info;
mv hooks/post-update.sample hooks/post-update;
(2) 本地克隆
cd /opt;
git clone file:///opt/git/repo/cakephp.git/;
(3) 把 cakephp-ex-master push 到仓库
unzip cakephp-ex-master.zip;
cp cakephp-ex-master/* cakephp -rf;
cp cakephp-ex-master/.sti cakephp -rf;
cp cakephp-ex-master/.htaccess cakephp -rf;
cp cakephp-ex-master/.gitignore cakephp -rf;
cd cakephp;
git add .;
git commit -m 'initial upload';
git push origin master;
(4) 在 httpd 中打开 git 服务
cat << EOF > /etc/httpd/conf.d/git.conf
Alias /git "/opt/git/repo"

    Dav On
    Options +Indexes +FollowSymLinks
Require all granted
EOF
chown -R apache:apache /opt/git/repo/cakephp.git;
systemctl restart httpd;
(5) 在其它3台机器（master，node1，node2）测试，确认可以 git clone 项目代码