什么是CI/CD，什么是Iac，一篇文章带你从入门到实战自动化部署项目

大家好，欢迎来到IT知识分享网。

前言，很久没有写文章了。前端时间学习了Infrastructure as Code相关的知识，希望以此文总结一下所学的知识。

1. 概念

1.1 什么是CI/CD，为什么需要CI/CD？

当我们开发完一个功能模块想要部署上线的时候，如果没有持续集成/部署，那么你就需要自己运行测试，并打包部署到服务器。如果服务器只有一台，也许你还能轻松应对。但是当服务器有很多的时候，估计你就汗流浃背了。

这个时候，我们就需要使用CI/CD来实现新功能自动测试以及上线的功能。
例如，我们可以编写对应的单元测试和集成测试，在提交PR的时候，GitHub运行action来执行对应的测试也就是Continuous Integration。
对于Continuous Deployment这一部分，在PR合并到主分支后，可以运行Packer来将仓库代码生成对应的jar包，并部署到服务器上。后面会详细说明

1.2 什么是IaC，为什么IaC？

IaC是基础设施即代码的缩写，也是DevOps中必不可少的一部分。想必大家都有手动配置云服务器的经历吧，有时候配置简单的资源的时候还可以勉强应付。但是当需要配置复杂的资源以及配置海量资源时，手动配置就不是一个很明智的选择。毕竟人不是机器，有时候会忘掉配置防火墙，Vpc，又或者是配置的时候输入错了某些参数，又或者是需要你配置成百上千台服务器的时候，总不可能都用手动配置吧。

这个时候，我们就需要使用IaC来帮助我们一劳永逸。我们可以使用代码，来告诉云服务商AWS，GCP来为我创建什么资源，并且再创建以后，也可以一键销毁，大大提高了开发的效率。常见的配置和管理云基础架构和资源的服务商有Terraform

2. 自动化部署实战

对于一个简单的应用，我们需要一个服务器，一个数据库，数据库只能由服务器访问。我们只需要用terraform创建一个VM，一个DB，对应的防火墙和VPC即可。Terraform的代码在文章末尾。

这个时候，我们可以使用terraform apply来创建对应的云服务器资源。每次当我们添加了新的功能以后，在github merge了PR。我们就可以执行相应的github action来帮助我们给程序打包，并替换掉当前在线上的服务器，从而实现自动化部署的流程。对应的action文件如下：

# This workflow will compile a package using Maven and then publish it to GitHub packages when a release is created name: Build machine image on: pull_request: types: [closed] branches: - main jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up JDK 8 uses: actions/setup-java@v3 with: java-version: '8' distribution: 'temurin' - uses: shogo82148/actions-setup-mysql@v1 with: mysql-version: "8.0" root-password: ${ 
   { secrets.MYSQL_ROOT_PASSWORD }} - id: auth uses: google-github-actions/auth@v2 with: credentials_json: ${ 
   { secrets.GCP_ACCOUNT_JSON }} - name: Set up Cloud SDK uses: google-github-actions/setup-gcloud@v2 - name: Use gcloud CLI run: gcloud info - name: Run integration tests run: mvn -B clean test -Djasypt.encryptor.password=${ 
   { secrets.JASYPT_ENCRYPTION_KEY }} -Dlogback.log.path="." - name: Install Package run: mvn -B clean install -DskipTests -Djasypt.encryptor.password=${ 
   { secrets.JASYPT_ENCRYPTION_KEY }} -Dlogback.log.path="/var/log" - name: Setup Packer uses: hashicorp/setup-packer@main id: setup with: version: 1.10.1 - name: Run `packer init` id: init run: "packer init ./gcp-centos8.pkr.hcl" - name: Build Image run: | packer build -on-error=abort -var 'project_id='${ 
   { secrets.GCP_PROJECT_ID }} \ -var 'jasypt_encryption_key='${ 
   { secrets.JASYPT_ENCRYPTION_KEY }} \ -var 'zone='${ 
   { secrets.GCP_ZONE }} ./gcp-centos8.pkr.hcl | tee packer_output.txt - name: Extract Image ID id: extract_image_id run: | IMAGE_ID=$(tail -2 packer_output.txt | awk 'match($0, /packer-.*/) { print substr($0, RSTART, RLENGTH) }') echo "IMAGE_ID=${IMAGE_ID}" >> $GITHUB_ENV - name: Create Startup Scripts run: | echo "${ 
    { secrets.STARTUP_SCRIPT }}" | base64 -d > startup-script.sh # New Steps for Creating and Updating Instance Template - name: Create New Instance Template Version run: | gcloud compute instance-templates create "instance-template-${ 
    { github.run_id }}-${ 
    { github.run_attempt }}" \ --region=${ 
   { secrets.GCP_REGION }} \ --instance-template-region=${ 
   { secrets.GCP_REGION }} \ --machine-type=${ 
   { vars.MACHINE_TYPE }} \ --tags="webapp-${ 
    { vars.RANDOM_SUFFIX }}",allow-health-check \ --create-disk=auto-delete=true,size=${ 
   { vars.DISK_SIZE }},type=${ 
   { vars.DISK_TYPE }},image=${IMAGE_ID},boot=${ 
   { vars.IS_BOOT }},kms-key="projects/${ 
    { secrets.GCP_PROJECT_ID }}/locations/${ 
    { secrets.GCP_REGION }}/keyRings/webapp-key-ring-${ 
    { vars.RANDOM_SUFFIX }}/cryptoKeys/vm-key" \ --network-interface=network="vpc-network-${ 
    { vars.RANDOM_SUFFIX }}",subnet="webapp-subnet-${ 
    { vars.RANDOM_SUFFIX }}",no-address \ --metadata-from-file=startup-script=./startup-script.sh \ --service-account=${ 
   { secrets.VM_SERVICE_ACCOUNT }} \ --scopes=cloud-platform - name: Configure Managed Instance Group run: | gcloud compute instance-groups managed set-instance-template instance-group-manager \ --template="projects/${ 
    { secrets.GCP_PROJECT_ID }}/regions/${ 
    { secrets.GCP_REGION }}/instanceTemplates/instance-template-${ 
    { github.run_id }}-${ 
    { github.run_attempt }}" \ --region=${ 
   { secrets.GCP_REGION }} - name: Recreate Instances in Managed Instance Group, and wait for updates to complete run: | gcloud compute instance-groups managed rolling-action start-update instance-group-manager \ --region=${ 
   { secrets.GCP_REGION }} --version="template=projects/${ 
    { secrets.GCP_PROJECT_ID }}/regions/${ 
    { secrets.GCP_REGION }}/instanceTemplates/instance-template-${ 
    { github.run_id }}-${ 
    { github.run_attempt }}" while :; do STATUS=$(gcloud compute instance-groups managed describe instance-group-manager \ --region=${ 
    { 
     secrets.GCP_REGION }} --format="get(status.versionTarget.isReached)") echo "Current group status: $STATUS" if [ "$STATUS" = "True" ]; then echo "The Instance Group Manager has completed the refresh process" break else echo "The Instance Group Manager has not completed the refresh process, please wait for that......" sleep 10 fi done 

# Google Cloud Platform Provider provider "google" { 
    project = var.project_id region = var.region } resource "random_string" "name_suffix" { 
    length = 6 special = false lower = true upper = false } resource "google_compute_network" "vpc_network" { 
    count = var.vpc_count name = "vpc-network-${count.index + 1}-${random_string.name_suffix.result}" auto_create_subnetworks = false delete_default_routes_on_create = true routing_mode = var.vpc_routing_mode } resource "google_compute_subnetwork" "webapp_subnet" { 
    count = var.vpc_count name = "webapp-subnet-${count.index + 1}-${random_string.name_suffix.result}" ip_cidr_range = var.ip_cidr_ranges[count.index * 2] region = var.region network = google_compute_network.vpc_network[count.index].id private_ip_google_access = true } # Route for webapp resource "google_compute_route" "webapp_route" { 
    count = var.vpc_count name = "webapp-route-${count.index + 1}-${random_string.name_suffix.result}" network = google_compute_network.vpc_network[count.index].id dest_range = var.webapp_route_dest_range next_hop_gateway = "default-internet-gateway" priority = 1000 tags = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"] } resource "google_compute_firewall" "vpc_firewall_webapp_allow_rule" { 
    count = var.vpc_count name = "vpc-firewall-webapp-allow-rule-${count.index + 1}-${random_string.name_suffix.result}" network = google_compute_network.vpc_network[count.index].id priority = var.firewall.allow_rule_priority allow { 
    protocol = var.firewall.allow_rule_protocol ports = var.firewall.allow_rule_ports } source_ranges = var.firewall.allow_rule_source_ranges target_tags = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"] } // Explicit to deny all another rule as required resource "google_compute_firewall" "vpc_firewall_deny_rule" { 
    count = var.vpc_count name = "vpc-firewall-deny-rule-${count.index + 1}-${random_string.name_suffix.result}" network = google_compute_network.vpc_network[count.index].id priority = var.firewall.deny_rule_priority deny { 
    protocol = var.firewall.deny_rule_protocol } source_ranges = var.firewall.deny_rule_source_ranges target_tags = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"] } resource "google_compute_instance" "vm_instance" { 
    count = var.vpc_count name = "vm-instance-${count.index + 1}-${random_string.name_suffix.result}" machine_type = var.vm_machine_type zone = var.zone tags = ["webapp-${count.index + 1}-${random_string.name_suffix.result}"] boot_disk { 
    initialize_params { 
    image = var.vm_boot_disk_params.image size = var.vm_boot_disk_params.size type = var.vm_boot_disk_params.type } } network_interface { 
    network = google_compute_network.vpc_network[count.index].id subnetwork = google_compute_subnetwork.webapp_subnet[count.index].id access_config { 
   } } metadata_startup_script = <<-EOF #!/bin/bash sudo yum update -y echo "y" | sudo yum install -y mysql sudo cat <<EOT > /opt/webapp_repo/startup.sh #!/bin/bash DB_HOST="${var.psc_addrs[count.index]}" DB_USER="webapp" DB_PASSWORD=${random_password.mysql_password.result} java -jar /opt/webapp_repo/Health_Check-0.0.1-SNAPSHOT.jar --spring.datasource.username=\$DB_USER \ --spring.datasource.password=\$DB_PASSWORD \ --spring.datasource.url="jdbc:mysql://\$DB_HOST:3306/health_check?useUnicode=true&characterEncoding=utf-8&serverTimezone=America/New_York&createDatabaseIfNotExist=true" EOT sudo chmod +x /opt/webapp_repo/startup.sh sudo systemctl daemon-reload sudo systemctl start webapp EOF depends_on = [google_sql_database_instance.db_instance] } resource "google_compute_address" "psc_address" { 
    count = var.vpc_count project = var.project_id name = "psc-address-${count.index + 1}-${random_string.name_suffix.result}" region = var.region address_type = "INTERNAL" subnetwork = google_compute_subnetwork.webapp_subnet[count.index].id address = var.psc_addrs[count.index] } resource "google_compute_forwarding_rule" "psc_endpoint" { 
    count = var.vpc_count project = var.project_id name = "psc-endpoint-${count.index + 1}-${random_string.name_suffix.result}" region = var.region target = google_sql_database_instance.db_instance[count.index].psc_service_attachment_link network = google_compute_network.vpc_network[count.index].id ip_address = google_compute_address.psc_address[count.index].id load_balancing_scheme = "" allow_psc_global_access = true } resource "google_sql_database_instance" "db_instance" { 
    count = var.vpc_count name = "db-instance-${count.index + 1}-${random_string.name_suffix.result}" region = var.region database_version = var.database_instance_config.database_version deletion_protection = var.database_instance_config.deletion_protection settings { 
    tier = var.database_instance_config.settings.tier ip_configuration { 
    ipv4_enabled = false enable_private_path_for_google_cloud_services = true psc_config { 
    psc_enabled = true allowed_consumer_projects = [var.project_id] } } backup_configuration { 
    enabled = var.database_instance_config.settings.backup_configuration.enabled binary_log_enabled = var.database_instance_config.settings.backup_configuration.binary_log_enabled } availability_type = var.database_instance_config.settings.availability_type disk_type = var.database_instance_config.settings.disk_type disk_size = var.database_instance_config.settings.disk_size edition = var.database_instance_config.settings.edition } } resource "google_sql_database" "database" { 
    count = var.vpc_count name = "webapp-db-${count.index + 1}-${random_string.name_suffix.result}" instance = google_sql_database_instance.db_instance[count.index].name } resource "google_sql_user" "db_user" { 
    count = var.vpc_count name = var.db_username password = random_password.mysql_password.result instance = google_sql_database_instance.db_instance[count.index].name host = "%" } resource "random_password" "mysql_password" { 
    length = 16 special = false }