docker实战之jenkins集群

环境

centos 7
jenkins 2.121.1

前言

jenkins集群是必须搭建的,可以提高部署效率,每次只是部署几个job自然不会出现问题,如果一次要执行100个呢?试过就知道有多慢了!

搭建

jenkins-master只负责分发构建任务.

搭建之前,我先把几个关键点梳理下:

jenkins主节点创建job到目标部署主机的ssh免密登录密钥信息,在集群中的jenkins从节点如何获取?
jenkins主节点数据必须持久化到本地磁盘

Pull jenkins images

1	$ docker pull jenkins/jenkins:lts

Jenkins master and Mount data volume

Build Dockerfile: jenkins-data and jenkins-master

$ docker build -f jenkins-data-dockerfile -t jenkins-data .
$ docker build -f jenkins-master-dockerfile -t jenins-master .
# 从节点宿主机上执行
$ docker build -f jenkins-slave-dockerfile -t jenins-slave .

Run jenkins-master

# 创建用户持久化jenkins数据文件的目录及日志目录.
$ mkdir -p {/data/jenkins,/data/logs/jenkins}
# 容器内启动jenkins服务使用的是jenkins用户,所以必须修改目录用户拥有者为
$ chown 1000:1000 {/data/jenkins,/data/logs/jenkins}
# 启动数据卷jenkins-datas
$ docker run --name jenkins-data \
-v /data/jenkins:/var/jenkins_home \
-v /data/logs/jenkins:/var/log/jenkins \
jenkins-data
# 启动jenkins-master.(构建dockerfile见附录.)
$ docker run -d \
-p 19875:8080 \
-p 50000:50000 \
--volumes-from jenkins-data \
--name jenkins-master \
jenkins-master

Run jenkins-slave in slave machine

$ docker run -ti -d \
-p 19222:22 \
--restart on-failure \
--network host \ 
--name jenkins-slave \
jenkins-slave

Jenkins master add slave node

添加登录slave的用户名密码

Jenkins->Credentials

这里可以添加root用户,避免一些权限问题.

添加节点

Jenkins->系统管理->管理节点

指定远程工作目录,如果是用jenkins_slave用户时注意/home/jenkins_home需要jenkins_slave权限.为了方便起见,这里就直接使用root用户

同步jenkins-master的密钥及相关脚本

这里我采用的做法是配置job,通过job将.ssh/id_rsa, .ssh/id_rsa.pub,.ssh/known_hosts拷贝到slave中(见rsa_sysnc.sh脚本).
(所有脚本proxy-scripts已同步到git上.)

新建同步job

上传dockerfile到docker hub个人仓库.

参考

Getting started with Docker 官方,推荐

Docker Volume 之权限管理

Get Started with Jenkins 2.0 with Docker

Official Jenkins Docker image

附录

jenkins-data-dockerfile

$ vim jenkins-data-dockerfile
FROM debian:jessie
# Create the jenkins user
RUN useradd -d "/var/jenkins_home" -u 1000 -m -s /bin/bash jenkins
# Create the folders and volume mount points
RUN mkdir -p /var/log/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
VOLUME ["/var/log/jenkins", "/var/jenkins_home"]
USER jenkins
CMD ["echo", "Data container for Jenkins"]

jenkins-master-dockerfile(待补充插件列表)

$ vim jenkins-master-dockerfile
FROM jenkins/jenkins:lts
# 增加swarm:3.13插件.https://plugins.jenkins.io/swarm
USER root
RUN /usr/local/bin/install-plugins.sh docker_swarm Ansible Multijob Pipeline
USER jenkins

jenkins-slave-dockerfile

FROM centos:centos7
# 创建用户及修改密码
RUN groupadd -g 1000 jenkins_slave
RUN useradd -d /home/jenkins_home -s /bin/bash \
-m jenkins_slave -u 1000 -g jenkins_slave
RUN echo 'ys123456' | passwd --stdin root
RUN echo 'jpass' | passwd --stdin jenkins_slave
# 安装依赖包
RUN yum install -y passwd openssl openssh-server wget git vim java-1.8.0-openjdk java-1.8.0-openjdk-devel ansible
# 设置maven,安装路径保持和jenkins-master中的一致.
RUN rm -rf /opt/apache-maven* && \
wget -O /opt/apache-maven-3.5.4.tar.gz http://mirrors.shu.edu.cn/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz && \
tar -xvf /opt/apache-maven-3.5.4.tar.gz -C /opt/ && \
mv /opt/apache-maven-3.5.4 /opt/maven && \
rm -rf /opt/apache-maven-3.5.4 && \
rm -rf /opt/apache-maven-3.5.4.tar.gz && \
echo "export MAVEN_HOME=/opt/maven" >> /etc/profile && \
echo 'PATH=$PATH:$MAVEN_HOME/bin' >> /etc/profile && \
source /etc/profile
# 设置jdk,安装路径保持和jenkins-master中的一致.
RUN mkdir -p /usr/local/java && \
wget --no-cookies \
--no-check-certificate \
--header "Cookie: oraclelicense=accept-securebackup-cookie" \
http://download.oracle.com/otn-pub/java/jdk/8u171-b11/512cd62ec5174c3487ac17c61aaa89e8/jdk-8u171-linux-x64.tar.gz \
-O /usr/local/java/jdk1.8.tar.gz
RUN tar -xvf /usr/local/java/jdk1.8.tar.gz -C /usr/local/java && mv /usr/local/java/jdk1.8.0_171 /usr/local/java/jdk1.8 && echo 'export JAVA_HOME=/usr/local/java/jdk1.8' >> /etc/profile && echo 'PATH=$PATH:$JAVA_HOME/bin' && source /etc/profile
# 设置jenkins_home目录,保持和jenkins-master一致.(根据需要自己修改路径)
RUN mkdir -p /root/.jenkins && echo 'export JENKINS_HOME=/root/.jenkins/' >> /etc/profile && source /etc/profile
# 生成密钥
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
RUN ssh-keygen -q -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
RUN sed -i 's|session required pam_loginuid.so|session optional pam_loginuid.so|g' /etc/pam.d/sshd
RUN mkdir -p /root/.ssh && chown root.root /root && chmod 700 /root/.ssh

# 下载脚本.这里是因为脚本构建用到了许多脚本.
RUN git clone https://github.com/steven-ji/proxy-scripts.git /opt/scripts
RUN chown -R 1000:1000 /opt/scripts

#设置时区
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo 'Asia/Shanghai' >/etc/timezone

EXPOSE 22

CMD ip addr ls eth0 | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+';/usr/sbin/sshd -D