DevOps 环境部署实战：GitLab+Jenkins+Harbor 完整搭建

版本: V1.0 | 技术深度: 生产部署级 | 预计阅读时间: 65 分钟
质量目标: CSDN 评分>95 | 适用人群: DevOps 工程师、系统管理员、技术负责人

---

目录

• 1. 主机规划与网络拓扑
  • 1.1 主机角色分配
  • 1.2 网络架构设计
  • 1.3 资源配置标准
• [2. GitLab 服务器部署](#2-gitlab 服务器部署)
  • [2.1 Docker 安装 GitLab](#21-docker 安装-gitlab)
  • 2.2 配置优化
  • 2.3 高可用配置
• [3. Jenkins 服务器部署](#3-jenkins 服务器部署)
  • [3.1 Jenkins Master 部署](#31-jenkins-master 部署)
  • [3.2 Jenkins Agent 配置](#32-jenkins-agent 配置)
  • 3.3 插件管理
• [4. Harbor 镜像仓库部署](#4-harbor 镜像仓库部署)
  • [4.1 Harbor 离线安装](#41-harbor 离线安装)
  • 4.2 配置管理
  • 4.3 高可用集群
• [5. Web 服务器部署](#5-web 服务器部署)
  • [5.1 Nginx 反向代理](#51-nginx 反向代理)
  • [5.2 SSL 证书配置](#52-ssl 证书配置)
  • 5.3 负载均衡
• 6. 工具集成与配置
  • [6.1 GitLab-Jenkins 集成](#61-gitlab-jenkins 集成)
  • [6.2 Jenkins-Harbor 集成](#62-jenkins-harbor 集成)
  • 6.3 完整流水线测试
• 7. 监控与运维
  • 7.1 日志收集
  • 7.2 指标监控
  • 7.3 备份策略
• 8. 总结
• [附录 A：完整部署脚本](#附录-a 完整部署脚本)
• [附录 B：故障排查指南](#附录-b 故障排查指南)

---

1. 主机规划与网络拓扑

1.1 主机角色分配

1.1.1 生产环境主机规划（推荐配置）

主机名	IP 地址	配置	角色	运行服务
gitlab-01	192.168.1.11	8 核 16GB 500GB SSD	GitLab 主节点	gitlab-server, postgresql, redis
gitlab-02	192.168.1.12	8 核 16GB 500GB SSD	GitLab 从节点	gitlab-runner
jenkins-01	192.168.1.21	8 核 16GB 200GB SSD	Jenkins Master	jenkins-master
jenkins-02	192.168.1.22	8 核 16GB 200GB SSD	Jenkins Agent	jenkins-agent, docker
jenkins-03	192.168.1.23	8 核 16GB 200GB SSD	Jenkins Agent	jenkins-agent, docker
harbor-01	192.168.1.31	8 核 16GB 1TB SSD	Harbor 主节点	harbor-core, registry
harbor-02	192.168.1.32	8 核 16GB 1TB SSD	Harbor 从节点	harbor-core, registry
harbor-03	192.168.1.33	8 核 16GB 1TB SSD	Harbor 从节点	harbor-core, registry
web-01	192.168.1.41	4 核 8GB 100GB SSD	Web 服务器	nginx, tomcat
web-02	192.168.1.42	4 核 8GB 100GB SSD	Web 服务器	nginx, tomcat
monitor-01	192.168.1.51	4 核 8GB 500GB SSD	监控服务器	prometheus, grafana

1.2 网络架构设计

1.2.1 完整网络拓扑

1.3 资源配置标准

1.3.1 最小资源配置

组件	最小配置	推荐配置	生产配置
GitLab	4 核 8GB 200GB	8 核 16GB 500GB	16 核 32GB 1TB
Jenkins Master	4 核 4GB 100GB	8 核 16GB 200GB	16 核 32GB 500GB
Jenkins Agent	2 核 2GB 50GB	4 核 8GB 100GB	8 核 16GB 200GB
Harbor	4 核 8GB 500GB	8 核 16GB 1TB	16 核 32GB 2TB
Nginx	2 核 2GB 50GB	4 核 8GB 100GB	8 核 16GB 200GB

---

2. GitLab 服务器部署

2.1 Docker 安装 GitLab

2.1.1 完整安装脚本

#!/bin/bash
# install-gitlab.sh - Docker 安装 GitLab

set -euo pipefail

GITLAB_VERSION="16.5.0"
GITLAB_HOME="/opt/gitlab"
GITLAB_PORT="80"
GITLAB_SSH_PORT="2222"
GITLAB_ROOT_PASSWORD="Admin@123"

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "=== 开始安装 GitLab $GITLAB_VERSION ==="

# 1. 创建目录结构
log "[1/6] 创建目录结构..."
mkdir -p $GITLAB_HOME/{config,logs,data}

# 2. 准备配置文件
log "[2/6] 准备配置文件..."
cat > $GITLAB_HOME/config/gitlab.rb <<EOF
# GitLab 配置
external_url 'http://gitlab.example.com'

# SSH 配置
gitlab_rails['gitlab_shell_ssh_port'] = $GITLAB_SSH_PORT

# Nginx 配置
nginx['listen_port'] = $GITLAB_PORT

# PostgreSQL 配置
postgresql['max_connections'] = 1000

# Puma 配置
puma['worker_processes'] = 4
puma['min_threads'] = 5
puma['max_threads'] = 20

# Sidekiq 配置
sidekiq['max_concurrency'] = 25

# 存储配置
gitlab_rails['backup_path'] = '/var/opt/gitlab/backups'
gitlab_rails['backup_keep_time'] = 604800  # 7 天

# 邮件配置（可选）
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.example.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "gitlab"
gitlab_rails['smtp_password'] = "password"
gitlab_rails['smtp_domain'] = "example.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true

# LDAP 配置（可选）
# gitlab_rails['ldap_enabled'] = true
# gitlab_rails['ldap_servers'] = {
#   'main' => {
#     'label' => 'LDAP',
#     'host' => 'ldap.example.com',
#     'port' => 389,
#     'uid' => 'sAMAccountName',
#     'bind_dn' => 'CN=gitlab,OU=Users,DC=example,DC=com',
#     'password' => 'password',
#     'base' => 'DC=example,DC=com'
#   }
# }
EOF

# 3. 启动 GitLab 容器
log "[3/6] 启动 GitLab 容器..."
docker run -d \
    --name gitlab \
    --restart always \
    --hostname gitlab.example.com \
    --publish $GITLAB_PORT:80 \
    --publish $GITLAB_SSH_PORT:22 \
    --volume $GITLAB_HOME/config:/etc/gitlab \
    --volume $GITLAB_HOME/logs:/var/log/gitlab \
    --volume $GITLAB_HOME/data:/var/opt/gitlab \
    --shm-size 256m \
    --env GITLAB_ROOT_PASSWORD=$GITLAB_ROOT_PASSWORD \
    gitlab/gitlab-ce:$GITLAB_VERSION

# 4. 等待 GitLab 启动
log "[4/6] 等待 GitLab 启动（约 5 分钟）..."
for i in {1..60}; do
    if docker exec gitlab gitlab-rake gitlab:check >/dev/null 2>&1; then
        log "✓ GitLab 已就绪"
        break
    fi
    log "等待中... ($i/60)"
    sleep 10
done

# 5. 验证安装
log "[5/6] 验证安装..."
docker exec gitlab gitlab-rake gitlab:check

# 6. 显示访问信息
log "[6/6] 安装完成！"
log ""
log "访问地址：http://gitlab.example.com"
log "用户名：root"
log "密码：$GITLAB_ROOT_PASSWORD"
log ""
log "请首次登录后修改密码！"

2.1.2 Docker Compose 方式

# docker-compose-gitlab.yml
version: '3.8'

services:
  gitlab:
    image: gitlab/gitlab-ce:16.5.0
    container_name: gitlab
    hostname: gitlab.example.com
    restart: always
    ports:
      - "80:80"
      - "443:443"
      - "2222:22"
    volumes:
      - ./gitlab/config:/etc/gitlab
      - ./gitlab/logs:/var/log/gitlab
      - ./gitlab/data:/var/opt/gitlab
    environment:
      - GITLAB_OMNIBUS_CONFIG=|
          external_url 'http://gitlab.example.com'
          gitlab_rails['gitlab_shell_ssh_port'] = 2222
          nginx['listen_port'] = 80
          puma['worker_processes'] = 4
          puma['min_threads'] = 5
          puma['max_threads'] = 20
          sidekiq['max_concurrency'] = 25
          gitlab_rails['backup_path'] = '/var/opt/gitlab/backups'
          gitlab_rails['backup_keep_time'] = 604800
    deploy:
      resources:
        limits:
          cpus: '8.0'
          memory: 16G
        reservations:
          cpus: '4.0'
          memory: 8G
    healthcheck:
      test: ["CMD", "gitlab-rake", "gitlab:check"]
      interval: 5m
      timeout: 10s
      retries: 3
      start_period: 5m

2.2 配置优化

2.2.1 性能优化配置

# /opt/gitlab/config/gitlab.rb - 性能优化

# 1. 增加 Unicorn Worker
unicorn['worker_processes'] = 8
unicorn['worker_timeout'] = 60

# 2. 优化 PostgreSQL
postgresql['max_connections'] = 2000
postgresql['shared_buffers'] = "2GB"
postgresql['work_mem'] = "16MB"
postgresql['maintenance_work_mem'] = "512MB"
postgresql['effective_cache_size'] = "6GB"

# 3. 优化 Redis
redis['maxclients'] = 10000
redis['maxmemory'] = "4GB"
redis['maxmemory-policy'] = "noeviction"

# 4. 优化 Sidekiq
sidekiq['max_concurrency'] = 50
sidekiq['shutdown_timeout'] = 30

# 5. 优化 Gitaly
gitaly['ruby_max_rss'] = 300000000
gitaly['grpc_max_send_msg_size'] = 5368709120

# 6. 启用 Prometheus
prometheus_monitoring['enable'] = true

# 应用配置
# gitlab-ctl reconfigure

2.3 高可用配置

2.3.1 GitLab 高可用架构

---

3. Jenkins 服务器部署

3.1 Jenkins Master 部署

3.1.1 Docker 安装 Jenkins

#!/bin/bash
# install-jenkins.sh - Docker 安装 Jenkins

set -euo pipefail

JENKINS_VERSION="2.426.1-jdk17"
JENKINS_HOME="/opt/jenkins"
JENKINS_PORT="8080"
JENKINS_AGENT_PORT="50000"

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "=== 开始安装 Jenkins $JENKINS_VERSION ==="

# 1. 创建目录结构
log "[1/6] 创建目录结构..."
mkdir -p $JENKINS_HOME/{home,plugins,jobs}
chown -R 1000:1000 $JENKINS_HOME

# 2. 准备初始化脚本
log "[2/6] 准备初始化脚本..."
cat > $JENKINS_HOME/init.groovy.d/security.groovy <<'EOF'
import jenkins.model.*
import hudson.security.*

def instance = Jenkins.getInstance()

// 使用 Matrix 授权策略
def strategy = new GlobalMatrixAuthorizationStrategy()
strategy.add(Jenkins.ADMINISTER, "admin")
strategy.add(Jenkins.READ, "authenticated")
strategy.add(Item.READ, "authenticated")

instance.setAuthorizationStrategy(strategy)
instance.setSecurityRealm(new HudsonPrivateSecurityRealm(false))

// 创建 admin 用户（密码需要在首次登录时设置）
// def user = hudson.security.HudsonPrivateSecurityRealm.Details.fromPlainPassword("admin")
// instance.getSecurityRealm().createAccount("admin", user)

instance.save()
EOF

# 3. 启动 Jenkins 容器
log "[3/6] 启动 Jenkins 容器..."
docker run -d \
    --name jenkins \
    --restart always \
    --publish $JENKINS_PORT:8080 \
    --publish $JENKINS_AGENT_PORT:50000 \
    --volume $JENKINS_HOME/home:/var/jenkins_home \
    --volume $JENKINS_HOME/plugins:/var/jenkins_home/plugins \
    --volume $JENKINS_HOME/jobs:/var/jenkins_home/jobs \
    --volume /var/run/docker.sock:/var/run/docker.sock \
    --volume /usr/bin/docker:/usr/bin/docker \
    --env JAVA_OPTS="-Xmx4096m -Xms1024m" \
    jenkins/jenkins:$JENKINS_VERSION

# 4. 等待 Jenkins 启动
log "[4/6] 等待 Jenkins 启动..."
for i in {1..30}; do
    if curl -s http://localhost:$JENKINS_PORT/login >/dev/null; then
        log "✓ Jenkins 已就绪"
        break
    fi
    log "等待中... ($i/30)"
    sleep 10
done

# 5. 获取初始管理员密码
log "[5/6] 获取初始管理员密码..."
INITIAL_PASSWORD=$(docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword)
log "初始管理员密码：$INITIAL_PASSWORD"

# 6. 显示访问信息
log "[6/6] 安装完成！"
log ""
log "访问地址：http://jenkins.example.com:$JENKINS_PORT"
log "初始密码：$INITIAL_PASSWORD"
log "请在首次登录时修改密码！"

3.1.2 Docker Compose 配置

# docker-compose-jenkins.yml
version: '3.8'

services:
  jenkins:
    image: jenkins/jenkins:2.426.1-jdk17
    container_name: jenkins
    restart: always
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - ./jenkins/home:/var/jenkins_home
      - ./jenkins/plugins:/var/jenkins_home/plugins
      - ./jenkins/jobs:/var/jenkins_home/jobs
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    environment:
      - JAVA_OPTS=-Xmx4096m -Xms1024m
      - JENKINS_OPTS=--prefix=/jenkins
    deploy:
      resources:
        limits:
          cpus: '8.0'
          memory: 8G
        reservations:
          cpus: '4.0'
          memory: 4G
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/login"]
      interval: 5m
      timeout: 10s
      retries: 3

3.2 Jenkins Agent 配置

3.2.1 动态 Agent 配置

// Jenkinsfile - 动态 Agent 配置
pipeline {
    agent none
    
    stages {
        stage('Build') {
            agent {
                kubernetes {
                    yaml '''
apiVersion: v1
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - name: maven
    image: maven:3.8-openjdk-17
    command:
    - cat
    tty: true
    resources:
      limits:
        cpu: 2
        memory: 4Gi
      requests:
        cpu: 1
        memory: 2Gi
'''
                }
            }
            steps {
                sh 'mvn clean package'
            }
        }
        
        stage('Docker Build') {
            agent {
                docker {
                    image 'docker:24.0'
                    args '-v /var/run/docker.sock:/var/run/docker.sock'
                }
            }
            steps {
                sh 'docker build -t myapp:latest .'
            }
        }
    }
}

3.3 插件管理

3.3.1 必装插件清单

#!/bin/bash
# install-jenkins-plugins.sh - 安装 Jenkins 插件

set -euo pipefail

JENKINS_URL="http://localhost:8080"
JENKINS_USER="admin"
JENKINS_PASSWORD="admin"

# 必装插件列表
PLUGINS=(
    "git"                      # Git 集成
    "git-parameter"            # Git 参数
    "gitlab-plugin"            # GitLab 集成
    "docker-plugin"            # Docker 集成
    "kubernetes"               # K8s 集成
    "workflow-aggregator"      # Pipeline
    "pipeline-stage-view"      # Pipeline 视图
    "blueocean"                # Blue Ocean UI
    "configuration-as-code"    # 配置即代码
    "credentials"              # 凭证管理
    "credentials-binding"      # 凭证绑定
    "ssh-credentials"          # SSH 凭证
    "ssh-agent"                # SSH Agent
    "ssh-slaves"               # SSH Slave
    "matrix-auth"              # 矩阵授权
    "email-ext"                # 邮件扩展
    "slack"                    # Slack 通知
    "dingtalk"                 # 钉钉通知
    "wechat"                   # 企业微信
    "build-timeout"            # 构建超时
    "throttle-concurrents"     # 并发控制
    "lockable-resources"       # 资源锁
    "nexus-artifact-uploader"  # Nexus 上传
    "sonar"                    # SonarQube
    "cobertura"                # 代码覆盖率
    "junit"                    # JUnit 测试
    "htmlpublisher"            # HTML 发布
    "artifactdeployer"         # 构件部署
    "ssh-publish"              # SSH 发布
    "ansible"                  # Ansible
    "terraform"                # Terraform
)

log() {
    echo "[$(date +'%H:%M:%S')] $1"
}

log "=== 安装 Jenkins 插件 ==="

# 使用 Jenkins CLI 安装插件
for plugin in "${PLUGINS[@]}"; do
    log "安装插件：$plugin"
    java -jar jenkins-cli.jar \
        -s $JENKINS_URL \
        -auth $JENKINS_USER:$JENKINS_PASSWORD \
        install-plugin $plugin \
        -restart
    
    sleep 10
done

log "✓ 所有插件安装完成"

---

4. Harbor 镜像仓库部署

4.1 Harbor 离线安装

4.1.1 完整安装流程

#!/bin/bash
# install-harbor.sh - Harbor 离线安装

set -euo pipefail

HARBOR_VERSION="2.9.0"
HARBOR_HOME="/opt/harbor"
HARBOR_HOSTNAME="harbor.example.com"
HARBOR_PASSWORD="Harbor12345"

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "=== 开始安装 Harbor $HARBOR_VERSION ==="

# 1. 下载安装包
log "[1/8] 下载安装包..."
cd /tmp
wget https://github.com/goharbor/harbor/releases/download/v${HARBOR_VERSION}/harbor-offline-installer-v${HARBOR_VERSION}.tgz

# 2. 解压安装包
log "[2/8] 解压安装包..."
tar -xzf harbor-offline-installer-v${HARBOR_VERSION}.tgz -C $HARBOR_HOME
cd $HARBOR_HOME

# 3. 准备配置文件
log "[3/8] 准备配置文件..."
cat > harbor.yml <<EOF
# Harbor 配置
hostname: $HARBOR_HOSTNAME

# HTTP 端口
http:
  port: 80

# HTTPS 端口
https:
  port: 443
  certificate: /etc/harbor/ssl/server.crt
  private_key: /etc/harbor/ssl/server.key

# 管理员密码
harbor_admin_password: $HARBOR_PASSWORD

# 数据库配置
database:
  password: root123
  max_idle_conns: 50
  max_open_conns: 1000

# 数据卷配置
data_volume: /data

# Trivy 配置
trivy:
  ignore_unfixed: false
  skip_update: false
  offline_scan: false
  insecure: false

# 作业服务配置
jobservice:
  max_job_workers: 10

# 通知配置
notification:
  webhook_job_max_retry: 10

# 日志配置
log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /var/log/harbor

# 代理配置
# http_proxy: http://proxy.example.com:3128
# https_proxy: http://proxy.example.com:3128
# no_proxy: 127.0.0.1,localhost,.local,.internal

# 组件配置
proxy:
  nginx_image: nginx:1.24
  registry_controller_image: goharbor/harbor-registry-photon:v${HARBOR_VERSION}
  core_image: goharbor/harbor-core-photon:v${HARBOR_VERSION}
  job_service_image: goharbor/harbor-jobservice-photon:v${HARBOR_VERSION}
  db_image: goharbor/harbor-db-photon:v${HARBOR_VERSION}
  redis_image: redis:7
  exporter_image: goharbor/harbor-exporter-photon:v${HARBOR_VERSION}
  trivy_adapter_image: goharbor/trivy-adapter-photon:v${HARBOR_VERSION}
EOF

# 4. 创建 SSL 证书
log "[4/8] 创建 SSL 证书..."
mkdir -p /etc/harbor/ssl
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout /etc/harbor/ssl/server.key \
    -out /etc/harbor/ssl/server.crt \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=Harbor/CN=$HARBOR_HOSTNAME"

# 5. 运行安装脚本
log "[5/8] 运行安装脚本..."
./install.sh \
    --with-trivy \
    --with-notary \
    --with-chartmuseum

# 6. 等待 Harbor 启动
log "[6/8] 等待 Harbor 启动..."
for i in {1..60}; do
    if curl -sk https://$HARBOR_HOSTNAME/api/v2.0/ping >/dev/null; then
        log "✓ Harbor 已就绪"
        break
    fi
    log "等待中... ($i/60)"
    sleep 10
done

# 7. 验证安装
log "[7/8] 验证安装..."
curl -sk https://$HARBOR_HOSTNAME/api/v2.0/ping

# 8. 显示访问信息
log "[8/8] 安装完成！"
log ""
log "访问地址：https://$HARBOR_HOSTNAME"
log "用户名：admin"
log "密码：$HARBOR_PASSWORD"
log ""
log "请首次登录后修改密码！"

4.2 配置管理

4.2.1 Harbor 系统配置

# Harbor 系统配置模板
system-config:
  # 认证配置
  auth_mode: db_auth  # db_auth, ldap_auth, oidc_auth
  ldap_base_dn: DC=example,DC=com
  ldap_search_dn: CN=gitlab,OU=Users,DC=example,DC=com
  ldap_search_password: password
  ldap_filter: (objectClass=person)
  ldap_group_base_dn: OU=Groups,DC=example,DC=com
  ldap_group_search_filter: (objectClass=group)
  
  # 邮件配置
  email_host: smtp.example.com
  email_port: 587
  email_username: harbor@example.com
  email_password: password
  email_from: Harbor <harbor@example.com>
  email_ssl: true
  email_insecure: false
  
  # 项目配置
  project_creation_restriction: adminonly  # everyone, adminonly
  self_registration: false
  read_only: false
  
  # 复制配置
  replication_policy:
    - name: dev-to-prod
      source_registry: harbor-dev
      dest_registry: harbor-prod
      filters:
        - type: project
          value: library
      trigger:
        type: scheduled
        cron: "0 2 * * *"
  
  # 安全配置
  scan_all_policy:
    type: daily
    time: "00:00"
  
  # 配额配置
  quota:
    per_project:
      storage: -1  # 无限制
    per_user:
      storage: 10737418240  # 10GB

4.3 高可用集群

4.3.1 Harbor 集群配置

# docker-compose-harbor-cluster.yml
version: '3.8'

services:
  # PostgreSQL（主从复制）
  postgresql:
    image: goharbor/harbor-db-photon:v2.9.0
    container_name: harbor-db
    restart: always
    volumes:
      - /data/database:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=harbor
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.labels.harbor-db == "true"

  # Redis（Sentinel 模式）
  redis:
    image: redis:7-alpine
    container_name: harbor-redis
    restart: always
    command: redis-server --appendonly yes
    volumes:
      - /data/redis:/data
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s

  # Registry（多实例）
  registry:
    image: goharbor/harbor-registry-photon:v2.9.0
    container_name: harbor-registry
    restart: always
    volumes:
      - /data/registry:/storage
      - ./registry/config.yml:/etc/registry/config.yml:ro
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s

  # Core（多实例）
  core:
    image: goharbor/harbor-core-photon:v2.9.0
    container_name: harbor-core
    restart: always
    environment:
      - CORE_SECRET=your_secret
      - JOBSERVICE_SECRET=your_job_secret
    depends_on:
      - postgresql
      - registry
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s

  # Jobservice
  jobservice:
    image: goharbor/harbor-jobservice-photon:v2.9.0
    container_name: harbor-jobservice
    restart: always
    volumes:
      - /data/job-logs:/var/log/jobs
    depends_on:
      - core
    deploy:
      replicas: 2

  # Nginx（负载均衡）
  nginx:
    image: nginx:alpine
    container_name: harbor-nginx
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./ssl:/etc/nginx/ssl:ro
    depends_on:
      - core
      - registry
    deploy:
      replicas: 2
      placement:
        constraints:
          - node.role == manager

---

5. Web 服务器部署

5.1 Nginx 反向代理

5.1.1 Nginx 配置

# /etc/nginx/nginx.conf
user  nginx;
worker_processes  auto;
worker_rlimit_nofile 65535;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  65535;
    use epoll;
    multi_accept on;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    keepalive_timeout  65;
    types_hash_max_size 2048;

    # Gzip 压缩
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css text/xml application/json application/javascript;

    # 连接超时
    proxy_connect_timeout       60s;
    proxy_send_timeout          60s;
    proxy_read_timeout          60s;

    # 缓冲区
    proxy_buffer_size           128k;
    proxy_buffers               4 256k;
    proxy_busy_buffers_size     256k;

    # GitLab 上游
    upstream gitlab {
        least_conn;
        server gitlab-01:80;
        server gitlab-02:80;
        keepalive 32;
    }

    # Jenkins 上游
    upstream jenkins {
        least_conn;
        server jenkins-01:8080;
        keepalive 32;
    }

    # Harbor 上游
    upstream harbor {
        least_conn;
        server harbor-01:80;
        server harbor-02:80;
        server harbor-03:80;
        keepalive 32;
    }

    # GitLab 服务器
    server {
        listen 80;
        server_name gitlab.example.com;

        location / {
            proxy_pass http://gitlab;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /websocket {
            proxy_pass http://gitlab;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
    }

    # Jenkins 服务器
    server {
        listen 80;
        server_name jenkins.example.com;

        location / {
            proxy_pass http://jenkins;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

    # Harbor 服务器
    server {
        listen 80;
        server_name harbor.example.com;

        location / {
            proxy_pass http://harbor;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /v2/ {
            proxy_pass http://harbor;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_read_timeout 900;
        }
    }
}

5.2 SSL 证书配置

5.2.1 Let’s Encrypt 配置

#!/bin/bash
# install-ssl.sh - 安装 SSL 证书

set -euo pipefail

DOMAINS=(
    "gitlab.example.com"
    "jenkins.example.com"
    "harbor.example.com"
)

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "=== 安装 SSL 证书 ==="

# 1. 安装 Certbot
log "[1/4] 安装 Certbot..."
yum install -y certbot python3-certbot-nginx

# 2. 为每个域名申请证书
for domain in "${DOMAINS[@]}"; do
    log "[2/4] 申请证书：$domain"
    
    certbot certonly \
        --nginx \
        --agree-tos \
        --register-unsafely-without-email \
        --non-interactive \
        --redirect \
        -d $domain
    
    log "✓ 证书申请成功"
done

# 3. 配置自动续期
log "[3/4] 配置自动续期..."
(crontab -l 2>/dev/null; echo "0 3 * * * certbot renew --quiet") | crontab -

# 4. 验证证书
log "[4/4] 验证证书..."
for domain in "${DOMAINS[@]}"; do
    log "检查证书：$domain"
    openssl s_client -connect $domain:443 -servername $domain </dev/null 2>/dev/null | \
        openssl x509 -noout -dates
done

log "✓ SSL 证书安装完成"

---

6. 工具集成与配置

6.1 GitLab-Jenkins 集成

6.1.1 Webhook 配置

// Jenkins Pipeline - GitLab Webhook 触发
pipeline {
    agent any
    
    triggers {
        pollSCM('*/5 * * * *')  // 每 5 分钟轮询
        // 或使用 GitLab 插件
        // gitlab(triggerOnPush: true, triggerOnMergeRequest: true)
    }
    
    stages {
        stage('Checkout') {
            steps {
                git branch: 'main',
                    url: 'http://gitlab.example.com/group/project.git',
                    credentialsId: 'gitlab-credentials'
            }
        }
        
        stage('Build') {
            steps {
                sh 'mvn clean package -DskipTests'
            }
        }
    }
    
    post {
        success {
            // 通知 GitLab 构建成功
            gitlabCommitStatus name: 'build', state: 'success'
        }
        failure {
            // 通知 GitLab 构建失败
            gitlabCommitStatus name: 'build', state: 'failed'
        }
    }
}

6.2 Jenkins-Harbor 集成

6.2.1 Harbor 凭证配置

// Jenkins 配置 Harbor 凭证
import jenkins.model.*
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*

def domain = Domain.global()
def store = Jenkins.instance.getExtensionList(
    'com.cloudbees.plugins.credentials.SystemCredentialsProvider'
)[0].getStore()

// 添加 Harbor 凭证
def harborCredential = new UsernamePasswordCredentialsImpl(
    CredentialsScope.GLOBAL,
    'harbor-credentials',
    'Harbor Registry Credentials',
    'admin',           // 用户名
    'Harbor12345'      // 密码
)

store.addCredentials(domain, harborCredential)

---

7. 监控与运维

7.1 日志收集

7.1.1 ELK Stack 配置

# docker-compose-elk.yml
version: '3.8'

services:
  elasticsearch:
    image: elasticsearch:8.8.0
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    volumes:
      - es-data:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 8G

  logstash:
    image: logstash:8.8.0
    container_name: logstash
    volumes:
      - ./logstash/pipeline:/usr/share/logstash/pipeline:ro
    ports:
      - "5044:5044"
    depends_on:
      - elasticsearch
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4G

  kibana:
    image: kibana:8.8.0
    container_name: kibana
    ports:
      - "5601:5601"
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    depends_on:
      - elasticsearch
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 2G

  filebeat:
    image: elastic/filebeat:8.8.0
    container_name: filebeat
    volumes:
      - ./filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    depends_on:
      - logstash

volumes:
  es-data:

7.2 指标监控

7.2.1 Prometheus 监控配置

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'jenkins'
    static_configs:
      - targets: ['jenkins-01:8080']
    metrics_path: '/prometheus'
    
  - job_name: 'gitlab'
    static_configs:
      - targets: ['gitlab-01:9090']
    
  - job_name: 'harbor'
    static_configs:
      - targets: ['harbor-01:80']
    metrics_path: '/service/token'
    
  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']

7.3 备份策略

7.3.1 完整备份方案

#!/bin/bash
# backup-all.sh - 完整备份脚本

set -euo pipefail

BACKUP_DIR="/backup"
DATE=$(date +%Y%m%d-%H%M%S)

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}

log "=== 开始备份 ==="

# 1. 备份 GitLab
log "[1/4] 备份 GitLab..."
docker exec gitlab gitlab-backup create BACKUP=$DATE
cp /opt/gitlab/data/backups/${DATE}_gitlab_backup.tar $BACKUP_DIR/

# 2. 备份 Jenkins
log "[2/4] 备份 Jenkins..."
docker cp jenkins:/var/jenkins_home $BACKUP_DIR/jenkins-home-$DATE

# 3. 备份 Harbor
log "[3/4] 备份 Harbor..."
tar -czf $BACKUP_DIR/harbor-data-$DATE.tar.gz /data/harbor

# 4. 备份数据库
log "[4/4] 备份数据库..."
docker exec mysql mysqldump -u root -p --all-databases > $BACKUP_DIR/mysql-$DATE.sql

# 5. 清理旧备份（保留 7 天）
log "清理 7 天前的备份..."
find $BACKUP_DIR -name "*.tar*" -mtime +7 -delete
find $BACKUP_DIR -name "*.sql" -mtime +7 -delete

log "✓ 备份完成"

---

8. 总结

8.1 核心技术要点

1. 主机规划：合理分配资源、网络隔离
2. 工具部署：Docker 化部署、高可用配置
3. 集成配置：GitLab-Jenkins-Harbor 无缝集成
4. 监控运维：日志收集、指标监控、备份策略

8.2 最佳实践清单

✅ 部署规范：

• 使用 Docker Compose 统一管理
• 配置高可用集群
• 实施网络隔离
• 定期备份数据

✅ 安全加固：

• 配置 SSL/TLS
• 实施 RBAC 权限
• 定期更新补丁
• 安全扫描镜像

✅ 性能优化：

• 配置负载均衡
• 优化数据库连接
• 使用缓存加速
• 监控资源使用

---

附录 A：完整部署脚本

完整部署脚本已在上文各章节提供，包括：

• GitLab 安装脚本
• Jenkins 安装脚本
• Harbor 安装脚本
• SSL 证书安装脚本
• 备份脚本

附录 B：故障排查指南

# 常用故障排查命令

# 1. 检查服务状态
docker ps -a
docker logs <container>
docker stats

# 2. 网络检查
ping <host>
telnet <host> <port>
curl -v http://<host>:<port>

# 3. 日志检查
docker-compose logs -f
tail -f /var/log/nginx/error.log
journalctl -u docker

# 4. 性能检查
top
htop
iostat
netstat -an

---

文档版本: V1.0
最后更新: 2026-03-12
作者: AI 技术助手
许可协议: CC BY-SA 4.0