1. Pipeline Basics

Minimal .gitlab-ci.yml

# .gitlab-ci.yml — minimum viable pipeline
stages:
  - build
  - test
  - deploy

# Global defaults
default:
  image: node:20-alpine
  before_script:
    - echo "Pipeline $CI_PIPELINE_ID started"

build:
  stage: build
  script:
    - npm ci
    - npm run build
  after_script:
    - echo "Build finished with exit code $CI_JOB_STATUS"

test:
  stage: test
  script:
    - npm ci
    - npm test

deploy:
  stage: deploy
  script:
    - ./deploy.sh
  when: manual

Key Concepts

stages — ordered list; jobs in same stage run in parallel
image — Docker image for the job runner
script — shell commands (required per job)
before_script — runs before script (inherited)
after_script — runs after, even on failure
when — on_success | on_failure | always | manual | delayed

Job with Services

integration-test:
  stage: test
  image: node:20-alpine
  services:
    - name: postgres:16-alpine
      alias: db
    - name: redis:7-alpine
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: runner
    POSTGRES_PASSWORD: secret
    DATABASE_URL: "postgresql://runner:secret@db/testdb"
  script:
    - npm ci
    - npm run test:integration

2. Variables & Secrets

Defining Variables

# Global variables
variables:
  NODE_ENV: production
  APP_VERSION: "1.2.3"

# Job-level override
build:
  variables:
    NODE_ENV: development
  script:
    - echo $NODE_ENV  # → development

# Variable expansion
deploy:
  variables:
    TAG: "$CI_COMMIT_SHORT_SHA"
    IMAGE: "$CI_REGISTRY_IMAGE:$TAG"
  script:
    - docker push $IMAGE

Predefined Variables (Common)

Variable	Value
$CI_COMMIT_SHA	Full commit hash
$CI_COMMIT_SHORT_SHA	8-char short hash
$CI_COMMIT_REF_NAME	Branch or tag name
$CI_COMMIT_TAG	Tag name (if tagged)
$CI_PIPELINE_ID	Unique pipeline ID
$CI_PROJECT_NAME	Project name
$CI_REGISTRY_IMAGE	Container registry path
$CI_ENVIRONMENT_NAME	Deployment environment
$CI_DEFAULT_BRANCH	Default branch (main)
$CI_MERGE_REQUEST_IID	MR internal ID

🔒 Masked & Protected Variables

Set in Settings → CI/CD → Variables:

Masked — hidden in job logs (must meet base64 regex requirements)
Protected — only available on protected branches/tags
Environment scope — limit to specific environments (e.g., production)
File type — written to a temp file, var contains the path

3. Rules & Conditions

rules: (Modern — Use This)

deploy-staging:
  stage: deploy
  script: ./deploy.sh staging
  rules:
    # Run on default branch
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: on_success

    # Run on tags
    - if: $CI_COMMIT_TAG
      when: on_success

    # Run manually on merge requests
    - if: $CI_MERGE_REQUEST_IID
      when: manual
      allow_failure: true

    # Never run otherwise (implicit)

lint:
  script: npm run lint
  rules:
    # Only when source files change
    - changes:
        - "src/**/*"
        - "package.json"

    # Only when Dockerfile exists
    - exists:
        - Dockerfile

workflow:rules (Pipeline-Level)

# Control WHEN pipelines are created
workflow:
  rules:
    # Don't create pipelines for drafts
    - if: $CI_MERGE_REQUEST_TITLE =~ /^Draft:/
      when: never

    # MR pipelines
    - if: $CI_MERGE_REQUEST_IID

    # Default branch
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

    # Tags
    - if: $CI_COMMIT_TAG

only/except (Legacy)

# ⚠️ Deprecated — use rules: instead

deploy:
  script: ./deploy.sh
  only:
    - main
    - tags
  except:
    - schedules

# Equivalent with rules:
deploy:
  script: ./deploy.sh
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
    - if: $CI_COMMIT_TAG
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

4. Caching & Artifacts

Cache (Speed Up Installs)

build:
  stage: build
  cache:
    # Key based on lock file content
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
      - .npm/
    policy: pull-push  # pull | push | pull-push

  # Fallback keys
  cache:
    key:
      files:
        - package-lock.json
      prefix: $CI_COMMIT_REF_SLUG
    paths:
      - node_modules/
    fallback_keys:
      - npm-cache-main
      - npm-cache-default

Artifacts (Pass Between Jobs)

build:
  stage: build
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - .next/
      - node_modules/
    expire_in: 1 hour
    when: on_success  # on_success | on_failure | always

test:
  stage: test
  script:
    - npm test -- --coverage
  artifacts:
    # JUnit test report
    reports:
      junit: junit.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
    paths:
      - coverage/
    expire_in: 30 days

⚡ Cache vs Artifacts

Cache — best-effort, speeds up jobs, NOT guaranteed between pipelines
Artifacts — guaranteed, passed between stages, downloadable from UI
Use policy: pull on jobs that only read cache (e.g., test jobs)
Use cache:key:files to auto-invalidate when dependencies change

5. Docker & Registry

Docker-in-Docker (DinD)

build-image:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  before_script:
    - docker login -u $CI_REGISTRY_USER
        -p $CI_REGISTRY_PASSWORD
        $CI_REGISTRY
  script:
    - docker build
        -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
        -t $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE
        --all-tags

Kaniko (No Privileged Mode)

build-kaniko:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:v1.21.0-debug
    entrypoint: [""]
  script:
    - /kaniko/executor
      --context $CI_PROJECT_DIR
      --dockerfile $CI_PROJECT_DIR/Dockerfile
      --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
      --destination $CI_REGISTRY_IMAGE:latest
      --cache=true
      --cache-repo=$CI_REGISTRY_IMAGE/cache

Multi-Stage Build with Cache

# Dockerfile
FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

FROM node:20-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000
CMD ["node", "server.js"]

6. Environments & Deployments

deploy-staging:
  stage: deploy
  script:
    - kubectl set image deployment/app
        app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
        -n staging
  environment:
    name: staging
    url: https://staging.example.com
    on_stop: stop-staging
    auto_stop_in: 1 week
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

stop-staging:
  stage: deploy
  script:
    - kubectl delete namespace staging-$CI_COMMIT_REF_SLUG
  environment:
    name: staging
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

deploy-production:
  stage: deploy
  script:
    - kubectl set image deployment/app
        app=$CI_REGISTRY_IMAGE:$CI_COMMIT_TAG
        -n production
  environment:
    name: production
    url: https://example.com
    deployment_tier: production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
  allow_failure: false

Review Apps (Per-MR Environments)

deploy-review:
  stage: deploy
  script:
    - helm upgrade --install
        review-$CI_MERGE_REQUEST_IID
        ./chart
        --set image.tag=$CI_COMMIT_SHORT_SHA
        --set ingress.host=$CI_COMMIT_REF_SLUG.review.example.com
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.review.example.com
    on_stop: stop-review
    auto_stop_in: 2 days
  rules:
    - if: $CI_MERGE_REQUEST_IID

stop-review:
  stage: deploy
  script:
    - helm uninstall review-$CI_MERGE_REQUEST_IID
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  rules:
    - if: $CI_MERGE_REQUEST_IID
      when: manual

7. Templates & Includes

include: (4 Methods)

include:
  # Local file in same repo
  - local: "/.gitlab/ci/test.yml"

  # File from another project
  - project: "devops/pipeline-templates"
    ref: main
    file: "/templates/docker-build.yml"

  # GitLab-maintained templates
  - template: Auto-DevOps.gitlab-ci.yml
  - template: Security/SAST.gitlab-ci.yml

  # Remote URL
  - remote: "https://example.com/ci/lint.yml"

extends: & YAML Anchors

# Hidden job template (starts with .)
.node-job:
  image: node:20-alpine
  before_script:
    - npm ci
  cache:
    key:
      files: [package-lock.json]
    paths: [node_modules/]

lint:
  extends: .node-job
  script:
    - npm run lint

test:
  extends: .node-job
  script:
    - npm test

# YAML anchors (for data reuse)
.deploy-vars: &deploy-vars
  KUBE_NAMESPACE: production
  REPLICAS: "3"

deploy:
  variables:
    <<: *deploy-vars
    EXTRA: "value"

!reference Tag (Cherry-Pick Sections)

.setup:
  before_script:
    - echo "Setting up..."
    - npm ci
  after_script:
    - echo "Cleaning up..."

test:
  # Pull specific sections from other jobs
  before_script:
    - !reference [.setup, before_script]
  after_script:
    - !reference [.setup, after_script]
  script:
    - npm test

8. Parallel & DAG

parallel:matrix (Fan-Out)

test:
  stage: test
  image: node:${NODE_VERSION}-alpine
  parallel:
    matrix:
      - NODE_VERSION: ["18", "20", "22"]
        DATABASE: ["postgres", "mysql"]
  # Creates 6 jobs: 3 versions × 2 databases
  script:
    - npm ci
    - npm test

# Simple parallel (split by index)
test-chunks:
  stage: test
  parallel: 4
  script:
    - npm test -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

needs: (DAG — Skip Stage Order)

# Without needs: build → test → deploy
# With needs: jobs start as soon as deps finish

build-frontend:
  stage: build
  script: npm run build:frontend

build-backend:
  stage: build
  script: npm run build:backend

test-frontend:
  stage: test
  needs: ["build-frontend"]
  # Starts right after build-frontend,
  # doesn't wait for build-backend
  script: npm run test:frontend

test-backend:
  stage: test
  needs: ["build-backend"]
  script: npm run test:backend

deploy:
  stage: deploy
  needs: ["test-frontend", "test-backend"]
  script: ./deploy.sh

📊 needs: vs dependencies:

needs: — controls job ordering (DAG), downloads artifacts from listed jobs
dependencies: — only controls which artifacts are downloaded, NOT ordering
needs: [] — start immediately, no deps, no artifacts
dependencies: [] — run in stage order but download no artifacts

9. Security Scanning

# Include GitLab-maintained security templates
include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Security/Dependency-Scanning.gitlab-ci.yml
  - template: Security/Container-Scanning.gitlab-ci.yml
  - template: DAST.gitlab-ci.yml

variables:
  # SAST configuration
  SAST_EXCLUDED_PATHS: "spec,test,tests,tmp"
  SAST_EXCLUDED_ANALYZERS: "spotbugs"

  # Container scanning — scan your built image
  CS_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

  # DAST — dynamic application security testing
  DAST_WEBSITE: https://staging.example.com
  DAST_FULL_SCAN_ENABLED: "true"

  # Secret detection
  SECRET_DETECTION_HISTORIC_SCAN: "true"

Scanner	What It Checks	When to Use
SAST	Source code vulnerabilities	Every MR — static analysis
Secret Detection	Leaked credentials in code	Every push — catches secrets
Dependency Scanning	Known CVEs in packages	Every MR — npm audit equivalent
Container Scanning	OS-level CVEs in images	After docker build
DAST	Runtime web vulnerabilities	Against staging environment
License Scanning	License compliance	Before release — legal review

10. Production-Ready Example — Spring Boot on Cloud Foundry

Complete 4-stage pipeline: validate, test, build, deploy. Feature branches get auto-deployed to isolated routes for testing. Uses blue-green deployment for production.

# .gitlab-ci.yml — Spring Boot on Cloud Foundry (4 Stages)
# ========================================================
# Feature branches auto-deploy to isolated routes for testing
# Production uses blue-green deployment via cf-cli

stages:
  - validate
  - test
  - build
  - deploy

variables:
  JAVA_VERSION: "21"
  MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository"
  APP_NAME: "my-spring-app"
  CF_API: "https://api.cf.example.com"
  CF_ORG: "my-org"
  CF_SPACE_DEV: "development"
  CF_SPACE_STAGING: "staging"
  CF_SPACE_PROD: "production"
  CF_DOMAIN: "apps.example.com"

workflow:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/

# ─── Reusable Templates ──────────────────────────

.maven-base:
  image: eclipse-temurin:${JAVA_VERSION}-jdk-alpine
  cache:
    key:
      files: [pom.xml]
    paths: [.m2/repository/]
    policy: pull
  before_script:
    - chmod +x mvnw 2>/dev/null || true

.cf-base:
  image: governmentpaas/cf-cli:latest
  before_script:
    - cf api $CF_API
    - cf auth $CF_USERNAME $CF_PASSWORD

# ═══════════════════════════════════════════════════
# STAGE 1: VALIDATE
# ═══════════════════════════════════════════════════

checkstyle:
  extends: .maven-base
  stage: validate
  cache:
    policy: pull-push      # First job populates cache
  script:
    - ./mvnw checkstyle:check -B -q
    - ./mvnw spotbugs:check -B -q
  allow_failure: true       # Advisory, don't block

compile:
  extends: .maven-base
  stage: validate
  cache:
    policy: pull-push
  script:
    - ./mvnw compile -B -DskipTests
  artifacts:
    paths: [target/classes/]
    expire_in: 1 hour

# ═══════════════════════════════════════════════════
# STAGE 2: TEST
# ═══════════════════════════════════════════════════

unit-tests:
  extends: .maven-base
  stage: test
  needs: ["compile"]
  script:
    - ./mvnw verify -B
        -Dspring.profiles.active=test
  artifacts:
    when: always
    reports:
      junit: target/surefire-reports/TEST-*.xml
    paths:
      - target/site/jacoco/
    expire_in: 30 days
  coverage: '/Total.*?(\d+%)/'

integration-tests:
  extends: .maven-base
  stage: test
  needs: ["compile"]
  services:
    - name: postgres:16-alpine
      alias: testdb
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: test
    POSTGRES_PASSWORD: test
    SPRING_DATASOURCE_URL: "jdbc:postgresql://testdb:5432/testdb"
    SPRING_DATASOURCE_USERNAME: test
    SPRING_DATASOURCE_PASSWORD: test
  script:
    - ./mvnw verify -B
        -Dspring.profiles.active=integration-test
        -Dtest.groups=integration
  artifacts:
    when: always
    reports:
      junit: target/failsafe-reports/TEST-*.xml

# ═══════════════════════════════════════════════════
# STAGE 3: BUILD
# ═══════════════════════════════════════════════════

package:
  extends: .maven-base
  stage: build
  needs:
    - job: unit-tests
      artifacts: false
    - job: integration-tests
      artifacts: false
  script:
    - ./mvnw package -B -DskipTests
        -Dspring-boot.build-image.skip=true
    - mv target/*.jar target/app.jar
    - 'echo "BUILD_VERSION=$(./mvnw help:evaluate
        -Dexpression=project.version -q -DforceStdout)"
        >> build.env'
  artifacts:
    paths: [target/app.jar, manifest*.yml]
    expire_in: 7 days
    reports:
      dotenv: build.env

# ═══════════════════════════════════════════════════
# STAGE 4: DEPLOY
# ═══════════════════════════════════════════════════

# ─── Feature Branch → Isolated Route (Auto) ──────
# Every feature branch gets its own CF app + route
# Route pattern: feature-<branch-slug>.apps.example.com
# Auto-cleaned when branch is merged/deleted

deploy-feature:
  extends: .cf-base
  stage: deploy
  needs: ["package"]
  variables:
    FEATURE_APP: "${APP_NAME}-${CI_COMMIT_REF_SLUG}"
    FEATURE_ROUTE: "feature-${CI_COMMIT_REF_SLUG}"
  script:
    - cf target -o $CF_ORG -s $CF_SPACE_DEV
    # Push with unique name + route (no-route first, then map)
    - |
      cf push $FEATURE_APP \
        -p target/app.jar \
        -b java_buildpack_offline \
        -m 1G -k 512M -i 1 \
        --no-route \
        --no-start
    # Set feature-specific env vars
    - cf set-env $FEATURE_APP SPRING_PROFILES_ACTIVE dev
    - cf set-env $FEATURE_APP APP_VERSION $CI_COMMIT_SHORT_SHA
    # Map the isolated route
    - cf map-route $FEATURE_APP $CF_DOMAIN
        --hostname $FEATURE_ROUTE
    - cf start $FEATURE_APP
    # Health check
    - |
      for i in 1 2 3 4 5; do
        HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}"
          https://${FEATURE_ROUTE}.${CF_DOMAIN}/actuator/health)
        if [ "$HTTP_CODE" = "200" ]; then
          echo "✅ Health check passed"
          break
        fi
        echo "Waiting... (attempt $i)"
        sleep 10
      done
  environment:
    name: feature/$CI_COMMIT_REF_SLUG
    url: https://feature-$CI_COMMIT_REF_SLUG.$CF_DOMAIN
    on_stop: cleanup-feature
    auto_stop_in: 3 days
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      && $CI_COMMIT_BRANCH != "release"
      && $CI_COMMIT_BRANCH !~ /^hotfix/

# Auto-cleanup when feature branch env is stopped
cleanup-feature:
  extends: .cf-base
  stage: deploy
  variables:
    FEATURE_APP: "${APP_NAME}-${CI_COMMIT_REF_SLUG}"
  script:
    - cf target -o $CF_ORG -s $CF_SPACE_DEV
    - cf delete $FEATURE_APP -f -r || true
  environment:
    name: feature/$CI_COMMIT_REF_SLUG
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      when: manual
  allow_failure: true

# ─── Staging (Auto on main) ──────────────────────

deploy-staging:
  extends: .cf-base
  stage: deploy
  needs: ["package"]
  script:
    - cf target -o $CF_ORG -s $CF_SPACE_STAGING
    - |
      cf push $APP_NAME \
        -p target/app.jar \
        -b java_buildpack_offline \
        -m 2G -k 1G -i 2 \
        -f manifest-staging.yml
    - cf set-env $APP_NAME APP_VERSION $CI_COMMIT_SHORT_SHA
    - cf restage $APP_NAME
    # Smoke test
    - |
      HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}"
        https://staging-${APP_NAME}.${CF_DOMAIN}/actuator/health)
      if [ "$HTTP_CODE" != "200" ]; then
        echo "❌ Staging health check failed"
        exit 1
      fi
  environment:
    name: staging
    url: https://staging-$APP_NAME.$CF_DOMAIN
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# ─── Production — Blue-Green Deployment ──────────
# Zero-downtime: push to "green", remap route, stop "blue"

deploy-production:
  extends: .cf-base
  stage: deploy
  needs: ["package"]
  script:
    - cf target -o $CF_ORG -s $CF_SPACE_PROD
    # 1. Push new version as "green"
    - |
      cf push ${APP_NAME}-green \
        -p target/app.jar \
        -b java_buildpack_offline \
        -m 4G -k 2G -i 3 \
        -f manifest-production.yml \
        --no-route --no-start
    - cf set-env ${APP_NAME}-green APP_VERSION $CI_COMMIT_SHORT_SHA
    - cf set-env ${APP_NAME}-green SPRING_PROFILES_ACTIVE production
    - cf start ${APP_NAME}-green
    # 2. Health check green
    - |
      for i in $(seq 1 10); do
        HEALTH=$(cf curl /v2/apps/$(cf app ${APP_NAME}-green
          --guid)/stats 2>/dev/null | grep -c RUNNING || true)
        if [ "$HEALTH" -gt "0" ]; then break; fi
        sleep 5
      done
    # 3. Map production route to green
    - cf map-route ${APP_NAME}-green $CF_DOMAIN
        --hostname $APP_NAME
    # 4. Unmap route from blue (old version)
    - cf unmap-route $APP_NAME $CF_DOMAIN
        --hostname $APP_NAME || true
    # 5. Rename: blue→old, green→blue
    - cf rename $APP_NAME ${APP_NAME}-old || true
    - cf rename ${APP_NAME}-green $APP_NAME
    # 6. Stop old version (keep for rollback)
    - cf stop ${APP_NAME}-old || true
  environment:
    name: production
    url: https://$APP_NAME.$CF_DOMAIN
    deployment_tier: production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
  allow_failure: false

# ─── Rollback (swap back to old) ─────────────────

rollback-production:
  extends: .cf-base
  stage: deploy
  script:
    - cf target -o $CF_ORG -s $CF_SPACE_PROD
    # Start old version
    - cf start ${APP_NAME}-old
    # Remap route
    - cf map-route ${APP_NAME}-old $CF_DOMAIN
        --hostname $APP_NAME
    - cf unmap-route $APP_NAME $CF_DOMAIN
        --hostname $APP_NAME || true
    # Swap names back
    - cf rename $APP_NAME ${APP_NAME}-broken || true
    - cf rename ${APP_NAME}-old $APP_NAME
    - cf stop ${APP_NAME}-broken || true
    - echo "🔄 Rolled back to previous version"
  environment:
    name: production
    url: https://$APP_NAME.$CF_DOMAIN
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
  allow_failure: true

11. Production Example — Spring Boot + Gradle + 4 Environments

Enterprise pipeline with 4 environments (DEV → TEST → STAGING → PROD), semantic versioning, automatic release note generation, feature branch test deploys, and Gradle build.

Pipeline Overview

4 environments with promotion gates. Feature branches auto-deploy to DEV with isolated routes. Release tags trigger the full promotion chain.

# ┌─────────┐    ┌──────┐    ┌─────────┐    ┌──────────┐
# │   DEV   │───▶│ TEST │───▶│ STAGING │───▶│   PROD   │
# │  (auto) │    │(auto)│    │ (manual)│    │ (manual) │
# └─────────┘    └──────┘    └─────────┘    └──────────┘
#
# Feature branches → DEV only (isolated route)
# main branch      → DEV → TEST (auto)
# Release tags     → DEV → TEST → STAGING → PROD

Complete Pipeline

# .gitlab-ci.yml — Spring Boot + Gradle + Cloud Foundry
# 4 Environments | Semantic Versioning | Release Notes
# =====================================================

stages:
  - version
  - validate
  - test
  - build
  - deploy-dev
  - deploy-test
  - deploy-staging
  - deploy-prod
  - release

variables:
  GRADLE_OPTS: "-Dorg.gradle.daemon=false -Dorg.gradle.caching=true"
  GRADLE_USER_HOME: "$CI_PROJECT_DIR/.gradle"
  JAVA_VERSION: "21"
  APP_NAME: "my-spring-app"
  CF_API: "https://api.cf.example.com"
  CF_ORG: "my-org"
  CF_DOMAIN: "apps.example.com"

workflow:
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_BRANCH
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/

# ─── Reusable Templates ──────────────────────────

.gradle-base:
  image: eclipse-temurin:${JAVA_VERSION}-jdk-alpine
  cache:
    key:
      files: [build.gradle.kts, gradle/wrapper/gradle-wrapper.properties]
    paths:
      - .gradle/caches/
      - .gradle/wrapper/
      - build/
    policy: pull
  before_script:
    - chmod +x gradlew

.cf-base:
  image: governmentpaas/cf-cli:latest
  before_script:
    - cf api $CF_API
    - cf auth $CF_USERNAME $CF_PASSWORD

.cf-deploy:
  extends: .cf-base
  script:
    - cf target -o $CF_ORG -s $CF_SPACE
    - |
      cf push $DEPLOY_APP_NAME \
        -p build/libs/app.jar \
        -b java_buildpack_offline \
        -m $CF_MEMORY -k $CF_DISK -i $CF_INSTANCES \
        --no-route --no-start
    - cf set-env $DEPLOY_APP_NAME SPRING_PROFILES_ACTIVE $SPRING_PROFILE
    - cf set-env $DEPLOY_APP_NAME APP_VERSION $APP_VERSION
    - cf set-env $DEPLOY_APP_NAME BUILD_SHA $CI_COMMIT_SHORT_SHA
    - cf map-route $DEPLOY_APP_NAME $CF_DOMAIN
        --hostname $DEPLOY_HOSTNAME
    - cf start $DEPLOY_APP_NAME
    # Health check with retry
    - |
      for i in $(seq 1 12); do
        HTTP=$(curl -sf -o /dev/null -w "%{http_code}"
          "https://${DEPLOY_HOSTNAME}.${CF_DOMAIN}/actuator/health" || echo "000")
        if [ "$HTTP" = "200" ]; then
          echo "✅ Health check passed"; exit 0
        fi
        echo "⏳ Waiting... (attempt $i/12)"; sleep 10
      done
      echo "❌ Health check failed"; exit 1

# ═══════════════════════════════════════════════════
# VERSION — Semantic Versioning
# ═══════════════════════════════════════════════════

determine-version:
  stage: version
  image: alpine:3.19
  script:
    - apk add --no-cache git
    # Get latest tag or default to 0.0.0
    - |
      LATEST_TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "v0.0.0")
      MAJOR=$(echo $LATEST_TAG | sed 's/v//' | cut -d. -f1)
      MINOR=$(echo $LATEST_TAG | sed 's/v//' | cut -d. -f2)
      PATCH=$(echo $LATEST_TAG | sed 's/v//' | cut -d. -f3)
    # Auto-bump based on commit messages
    - |
      if git log $LATEST_TAG..HEAD --oneline | grep -qiE "^[a-f0-9]+ (feat|feature)"; then
        MINOR=$((MINOR + 1)); PATCH=0
        echo "📦 Minor bump (new feature detected)"
      elif git log $LATEST_TAG..HEAD --oneline | grep -qiE "BREAKING CHANGE|^[a-f0-9]+ (feat|fix)!"; then
        MAJOR=$((MAJOR + 1)); MINOR=0; PATCH=0
        echo "🚨 Major bump (breaking change detected)"
      else
        PATCH=$((PATCH + 1))
        echo "🔧 Patch bump"
      fi
    - |
      if [ "$CI_COMMIT_TAG" ]; then
        APP_VERSION="$CI_COMMIT_TAG"
      elif [ "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" ]; then
        APP_VERSION="v${MAJOR}.${MINOR}.${PATCH}-rc.${CI_PIPELINE_IID}"
      else
        APP_VERSION="v${MAJOR}.${MINOR}.${PATCH}-${CI_COMMIT_REF_SLUG}.${CI_PIPELINE_IID}"
      fi
    - echo "APP_VERSION=$APP_VERSION" >> version.env
    - echo "VERSION_MAJOR=$MAJOR" >> version.env
    - echo "VERSION_MINOR=$MINOR" >> version.env
    - echo "VERSION_PATCH=$PATCH" >> version.env
    - echo "📌 Version: $APP_VERSION"
  artifacts:
    reports:
      dotenv: version.env

# ═══════════════════════════════════════════════════
# VALIDATE
# ═══════════════════════════════════════════════════

lint:
  extends: .gradle-base
  stage: validate
  needs: ["determine-version"]
  cache:
    policy: pull-push
  script:
    - ./gradlew checkstyleMain checkstyleTest spotbugsMain -x test
  allow_failure: true

compile:
  extends: .gradle-base
  stage: validate
  needs: ["determine-version"]
  cache:
    policy: pull-push
  script:
    - ./gradlew compileJava compileTestJava

# ═══════════════════════════════════════════════════
# TEST
# ═══════════════════════════════════════════════════

unit-tests:
  extends: .gradle-base
  stage: test
  needs: ["compile"]
  script:
    - ./gradlew test jacocoTestReport
        -Dspring.profiles.active=test
  artifacts:
    when: always
    reports:
      junit: build/test-results/test/TEST-*.xml
    paths:
      - build/reports/jacoco/
    expire_in: 30 days
  coverage: '/Total.*?(\d+)%/'

integration-tests:
  extends: .gradle-base
  stage: test
  needs: ["compile"]
  services:
    - name: postgres:16-alpine
      alias: testdb
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: test
    POSTGRES_PASSWORD: test
    SPRING_DATASOURCE_URL: "jdbc:postgresql://testdb:5432/testdb"
  script:
    - ./gradlew integrationTest
        -Dspring.profiles.active=integration

# ═══════════════════════════════════════════════════
# BUILD
# ═══════════════════════════════════════════════════

build-jar:
  extends: .gradle-base
  stage: build
  needs:
    - determine-version
    - unit-tests
    - integration-tests
  script:
    - ./gradlew bootJar
        -Pversion=$APP_VERSION
        -x test
    - mv build/libs/*.jar build/libs/app.jar
    - echo "📦 Built app.jar ($APP_VERSION)"
  artifacts:
    paths:
      - build/libs/app.jar
    expire_in: 7 days

# ═══════════════════════════════════════════════════
# DEPLOY DEV — Feature branches get isolated routes
# ═══════════════════════════════════════════════════

deploy-dev-feature:
  extends: .cf-deploy
  stage: deploy-dev
  needs: ["build-jar", "determine-version"]
  variables:
    CF_SPACE: "development"
    CF_MEMORY: "1G"
    CF_DISK: "512M"
    CF_INSTANCES: "1"
    SPRING_PROFILE: "dev"
    DEPLOY_APP_NAME: "${APP_NAME}-${CI_COMMIT_REF_SLUG}"
    DEPLOY_HOSTNAME: "dev-${CI_COMMIT_REF_SLUG}-${APP_NAME}"
  environment:
    name: dev/$CI_COMMIT_REF_SLUG
    url: https://dev-$CI_COMMIT_REF_SLUG-$APP_NAME.$CF_DOMAIN
    on_stop: cleanup-dev-feature
    auto_stop_in: 5 days
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      && $CI_COMMIT_TAG == null

cleanup-dev-feature:
  extends: .cf-base
  stage: deploy-dev
  variables:
    FEATURE_APP: "${APP_NAME}-${CI_COMMIT_REF_SLUG}"
  script:
    - cf target -o $CF_ORG -s development
    - cf delete $FEATURE_APP -f -r || true
  environment:
    name: dev/$CI_COMMIT_REF_SLUG
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      when: manual
  allow_failure: true

deploy-dev-main:
  extends: .cf-deploy
  stage: deploy-dev
  needs: ["build-jar", "determine-version"]
  variables:
    CF_SPACE: "development"
    CF_MEMORY: "1G"
    CF_DISK: "512M"
    CF_INSTANCES: "1"
    SPRING_PROFILE: "dev"
    DEPLOY_APP_NAME: "$APP_NAME"
    DEPLOY_HOSTNAME: "dev-$APP_NAME"
  environment:
    name: dev
    url: https://dev-$APP_NAME.$CF_DOMAIN
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# ═══════════════════════════════════════════════════
# DEPLOY TEST — Auto on main
# ═══════════════════════════════════════════════════

deploy-test:
  extends: .cf-deploy
  stage: deploy-test
  needs: ["deploy-dev-main", "determine-version"]
  variables:
    CF_SPACE: "test"
    CF_MEMORY: "2G"
    CF_DISK: "1G"
    CF_INSTANCES: "2"
    SPRING_PROFILE: "test"
    DEPLOY_APP_NAME: "$APP_NAME"
    DEPLOY_HOSTNAME: "test-$APP_NAME"
  environment:
    name: test
    url: https://test-$APP_NAME.$CF_DOMAIN
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

# ═══════════════════════════════════════════════════
# DEPLOY STAGING — Manual gate
# ═══════════════════════════════════════════════════

deploy-staging:
  extends: .cf-deploy
  stage: deploy-staging
  needs: ["deploy-test", "determine-version"]
  variables:
    CF_SPACE: "staging"
    CF_MEMORY: "4G"
    CF_DISK: "2G"
    CF_INSTANCES: "2"
    SPRING_PROFILE: "staging"
    DEPLOY_APP_NAME: "$APP_NAME"
    DEPLOY_HOSTNAME: "staging-$APP_NAME"
  environment:
    name: staging
    url: https://staging-$APP_NAME.$CF_DOMAIN
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/

# ═══════════════════════════════════════════════════
# DEPLOY PROD — Blue-Green with manual gate
# ═══════════════════════════════════════════════════

deploy-prod:
  extends: .cf-base
  stage: deploy-prod
  needs: ["deploy-staging", "build-jar", "determine-version"]
  script:
    - cf target -o $CF_ORG -s production
    # Blue-green deployment
    - |
      cf push ${APP_NAME}-green \
        -p build/libs/app.jar \
        -b java_buildpack_offline \
        -m 4G -k 2G -i 3 \
        --no-route --no-start
    - cf set-env ${APP_NAME}-green SPRING_PROFILES_ACTIVE production
    - cf set-env ${APP_NAME}-green APP_VERSION $APP_VERSION
    - cf start ${APP_NAME}-green
    # Wait for healthy
    - sleep 15
    # Swap routes
    - cf map-route ${APP_NAME}-green $CF_DOMAIN --hostname $APP_NAME
    - cf unmap-route $APP_NAME $CF_DOMAIN --hostname $APP_NAME || true
    - cf rename $APP_NAME ${APP_NAME}-old || true
    - cf rename ${APP_NAME}-green $APP_NAME
    - cf stop ${APP_NAME}-old || true
    - echo "✅ Production deployed: $APP_VERSION"
  environment:
    name: production
    url: https://$APP_NAME.$CF_DOMAIN
    deployment_tier: production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
  allow_failure: false

rollback-prod:
  extends: .cf-base
  stage: deploy-prod
  script:
    - cf target -o $CF_ORG -s production
    - cf start ${APP_NAME}-old
    - cf map-route ${APP_NAME}-old $CF_DOMAIN --hostname $APP_NAME
    - cf unmap-route $APP_NAME $CF_DOMAIN --hostname $APP_NAME || true
    - cf rename $APP_NAME ${APP_NAME}-broken || true
    - cf rename ${APP_NAME}-old $APP_NAME
    - cf stop ${APP_NAME}-broken || true
    - echo "🔄 Rolled back to previous version"
  environment:
    name: production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual
  allow_failure: true

# ═══════════════════════════════════════════════════
# RELEASE — Tag, Changelog, GitLab Release
# ═══════════════════════════════════════════════════

create-release-tag:
  stage: release
  image: alpine/git:2.43.0
  needs: ["deploy-test", "determine-version"]
  script:
    - apk add --no-cache curl jq
    # Create annotated tag via GitLab API
    - |
      RELEASE_TAG="v${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}"
      echo "🏷️ Creating tag: $RELEASE_TAG"
    - |
      curl -s --fail --request POST \
        --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
        "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/tags" \
        --form "tag_name=$RELEASE_TAG" \
        --form "ref=$CI_COMMIT_SHA" \
        --form "message=Release $RELEASE_TAG"
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

generate-release-notes:
  stage: release
  image: alpine:3.19
  needs: ["determine-version"]
  script:
    - apk add --no-cache git curl jq
    - |
      PREV_TAG=$(git describe --tags --abbrev=0 HEAD^ 2>/dev/null || echo "")
      if [ -z "$PREV_TAG" ]; then
        RANGE="HEAD"
      else
        RANGE="${PREV_TAG}..HEAD"
      fi
    # Generate changelog from conventional commits
    - |
      echo "# Release $APP_VERSION" > RELEASE_NOTES.md
      echo "" >> RELEASE_NOTES.md
      echo "## 🚀 Features" >> RELEASE_NOTES.md
      git log $RANGE --oneline --grep="^feat" --format="- %s (%h)" >> RELEASE_NOTES.md || echo "- No new features" >> RELEASE_NOTES.md
      echo "" >> RELEASE_NOTES.md
      echo "## 🐛 Bug Fixes" >> RELEASE_NOTES.md
      git log $RANGE --oneline --grep="^fix" --format="- %s (%h)" >> RELEASE_NOTES.md || echo "- No bug fixes" >> RELEASE_NOTES.md
      echo "" >> RELEASE_NOTES.md
      echo "## 🔧 Other Changes" >> RELEASE_NOTES.md
      git log $RANGE --oneline --grep="^chore\|^refactor\|^docs\|^style\|^perf" --format="- %s (%h)" >> RELEASE_NOTES.md || true
      echo "" >> RELEASE_NOTES.md
      echo "---" >> RELEASE_NOTES.md
      echo "Deployed by pipeline #$CI_PIPELINE_IID" >> RELEASE_NOTES.md
    - cat RELEASE_NOTES.md
    # Create GitLab Release via API
    - |
      NOTES=$(cat RELEASE_NOTES.md | jq -Rs .)
      curl -s --fail --request POST \
        --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
        --header "Content-Type: application/json" \
        "$CI_API_V4_URL/projects/$CI_PROJECT_ID/releases" \
        --data "{
          \"tag_name\": \"$CI_COMMIT_TAG\",
          \"name\": \"Release $CI_COMMIT_TAG\",
          \"description\": $NOTES
        }"
  artifacts:
    paths: [RELEASE_NOTES.md]
    expire_in: 90 days
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/

Feature Branch Route Pattern

Each feature branch gets an isolated URL. Share with QA/PO for review before merging.

# Branch: feature/JIRA-1234-user-login
# → Route: dev-feature-jira-1234-user-login-my-spring-app.apps.example.com

# Branch: bugfix/fix-payment
# → Route: dev-bugfix-fix-payment-my-spring-app.apps.example.com

# main branch
# → Route: dev-my-spring-app.apps.example.com (always latest)

# Environments in GitLab UI:
# ┌──────────────────────────────────────────────┐
# │ dev/feature-jira-1234  → 🟢 active (3d left)│
# │ dev/bugfix-fix-payment → 🟢 active (5d left)│
# │ dev                    → 🟢 active           │
# │ test                   → 🟢 active           │
# │ staging                → 🟡 manual gate       │
# │ production             → 🔴 manual gate       │
# └──────────────────────────────────────────────┘

Gradle Build Configuration

// build.gradle.kts — Version from CI
plugins {
    java
    id("org.springframework.boot") version "3.3.0"
    id("io.spring.dependency-management") version "1.1.5"
    id("checkstyle")
    id("com.github.spotbugs") version "6.0.9"
    jacoco
}

version = project.findProperty("version") ?: "0.0.1-SNAPSHOT"
java { sourceCompatibility = JavaVersion.VERSION_21 }

tasks.named<org.springframework.boot.gradle.tasks.bundling.BootJar>("bootJar") {
    archiveFileName.set("app.jar")
}

tasks.jacocoTestReport {
    reports {
        xml.required.set(true)  // For GitLab coverage
        html.required.set(true)
    }
}

// Integration tests source set
sourceSets {
    create("integrationTest") {
        compileClasspath += sourceSets.main.get().output
        runtimeClasspath += sourceSets.main.get().output
    }
}

tasks.register<Test>("integrationTest") {
    testClassesDirs = sourceSets["integrationTest"].output.classesDirs
    classpath = sourceSets["integrationTest"].runtimeClasspath
    useJUnitPlatform()
}

Required CI/CD Variables

# Settings → CI/CD → Variables (protected + masked)
CF_USERNAME        # Cloud Foundry service account
CF_PASSWORD        # Cloud Foundry password (masked)
GITLAB_API_TOKEN   # For creating tags + releases (api scope)

# Optional
SLACK_WEBHOOK_URL  # For notifications