Build Images, Caching, Artifacts and Reproducible Delivery

Engineering scenario: A service Docker build takes 35 minutes and frequently fails when workers are scaled. Design faster container builds that remain reproducible, secure and traceable to tested source.

Distinguish the layers

A CI pipeline has at least four potential bottlenecks: checkout/download, dependency resolution, compile/test, and image/artifact creation. A Docker cache is not a general build cache. A faster image build that silently reuses stale dependencies or publishes an unreviewed tag is a correctness regression.

Cache architecture

Cache	Typical data	Invalidation / risk
SCM mirror	Git objects	Revision identity, permissions
Package manager	Downloaded dependencies	Lockfile + registry trust
Compilation	Object files/test result cache	Compiler flags, architecture, source hash
Docker layers/BuildKit	Layer snapshots/build contexts	COPY order, build arguments, secret leakage
Registry remote cache	Cache manifests and layers	Access policy, retention, trust domain
Persistent workspace	Checkout and intermediate files	Stale files and cross-build contamination

Cache hits should be observable; benchmark warm and cold builds separately. Avoid one writable shared cache spanning both untrusted pull requests and privileged release builders. A cache exporter can become a registry/storage cost center, so apply retention and cleanup.

BuildKit concept with controlled Dockerfile

# syntax=docker/dockerfile:1
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-input -r requirements.txt
COPY src/ ./src/
CMD ["python", "-m", "src.main"]

For true repeatability pin the base image digest and package hashes, vet index trust and define architecture. Secrets belong in ephemeral BuildKit secret mounts or a trusted identity mechanism, not ARG or copied into an image layer. This example is simplified and needs a real project plus a verified requirements.txt.

Jenkins publish flow

pipeline {
  agent { label 'trusted-builder' }
  stages {
    stage('Verify') {
      steps { checkout scm; sh './scripts/test.sh' }
    }
    stage('Build') {
      steps {
        sh '''
          set -eu
          docker buildx build --platform linux/amd64 \
            --tag registry.example.com/catalog:build-example \
            --load .
        '''
      }
    }
  }
}

The above intentionally does not push or authenticate. A production publish stage must resolve the exact built digest, authenticate using restricted release credentials, push once, attest the output where required, and hand a digest to deployment. --load generally loads an image into a local Docker image store and is not a universal solution for multi-platform manifests. Avoid mounting the host Docker daemon into untrusted jobs.

Multi-architecture builds

Architectures such as arm64 and amd64 may differ in dependency wheels, native compilation, test behavior and emulation performance. Buildx can build multi-platform manifests using suitable builders, but a build that succeeds under emulation doesn't establish runtime compatibility on native nodes. Test each target architecture when practical; record per-platform digest and resulting image index digest.

Failure scenario: build time regression

Cache hit rate falls after a dependency manifest moves behind a frequently changed COPY . .. The whole dependency layer is invalidated on every source edit. Reorder Dockerfile layers to copy lockfiles first, then application sources, and measure effect. If cold builds remain slow, inspect registry throttling, network throughput, ephemeral disk I/O and CPU before adding more concurrent builders.

docker buildx inspect
docker buildx du
docker image inspect IMAGE_ID
df -h
df -i
# On an approved agent: capture wall time and cache hit/miss evidence

Artifact identity and retention

Archived Jenkins artifacts are useful for short-lived test reports, but high-volume binaries usually belong in an external artifact repository or object store with immutability and retention. Releasing :latest makes rollback ambiguous. Persist source revision, dependency/lockfile identity, builder reference, artifact digest, test result and promotion approval.

Spoken explanation

“I'd profile checkout, dependencies, tests and container build separately. Then I'd fix the expensive layer using correct lockfile-based caching and trusted remote cache where justified, preserving immutable inputs. I would isolate build credentials from PR agents, build once, record the digest and promote the tested artifact through a separate authorization boundary. For multi-architecture images I'd validate each target runtime, not just a successful build command.”

References: Docker Build cache · Buildx · Jenkins Pipeline syntax