Most teams have a pipeline that builds the code and pushes it somewhere. That’s the easy part. The real test is whether the team deploys on a Friday afternoon without a knot in their stomach.
A trustworthy pipeline makes releases boring. Here is the checklist we use when we review one.
1. Build
- Builds are reproducible. The same commit produces the same artifact. Dependencies are pinned and lockfiles are committed.
- One artifact, promoted through environments. You build once and deploy that same artifact to staging and production, rather than rebuilding per environment.
- Builds are fast. If the feedback loop takes more than about ten minutes, people stop waiting for it.
2. Test
- Tests run on every change, and a failing test blocks the merge.
- Flaky tests are fixed or removed. A pipeline people re-run until it passes has stopped being a safety net.
- The test pyramid is sensible. Many fast unit tests, fewer integration tests, a small number of end-to-end checks on critical paths.
3. Security and Quality Gates
- Dependency and container image scanning runs automatically.
- Secrets never live in the repository, and a scanner catches the ones that slip in.
- Static analysis and type checks run before anything is deployed.
4. Deploy
- Infrastructure is code. Environments can be recreated from the repository.
- Deployments are automated and repeatable, with no manual steps and no “run this script by hand.”
- Configuration is separated from code, and each environment’s configuration is versioned and reviewable.
- Production deploys need an explicit, auditable approval where your risk profile requires one.
5. Release Safely
- Rollback is a tested, fast path. If you’ve never practised it, assume it doesn’t work.
- Risky changes ship behind feature flags, so deployment and release are separate decisions.
- Progressive rollout (canary or blue-green) limits the blast radius of a bad release.
- Database changes are backward compatible, so the old and new versions of the application can run side by side during a rollout.
6. Observe
- Every deployment is visible on dashboards, so a change can be correlated with a shift in errors or latency.
- Alerts are tied to user impact, not to raw resource metrics.
- Someone knows who’s on call and has a runbook for the common failure modes.
How to Use This
Score yourself honestly. Don’t try to fix everything at once. The highest-value gaps are almost always the same few: no tested rollback, rebuilding per environment, flaky tests, and no deployment visibility.
Fix those first. Each one removes a category of incident, and together they change how the team feels about shipping.
If you’d like an outside review of your delivery pipeline, our DevOps and automation work starts with exactly this kind of assessment.