GenAI-based development platform
Part 1: guardrails

generative AI   genAI development   harness engineering   deployment pipeline   testing   automation   development   devops  

---

---

This article is the first in a series about the GenAI-based development platform (aka. harness) that I’ve been developing to make GenAI-based coding agents like Claude Code more productive and less frustrating. The platform is a set of tools, constraints and feedback loops that guide their behavior, catch mistakes and prevent them from generating large amounts of poor-quality code that is often incomplete and untested.

Because coding agents cannot be relied on to consistently follow instructions, they must operate within deterministic guardrails that catch mistakes and enforce quality. Guardrails are therefore a central part of the platform. This article describes some of these guardrails.

Specifically, there are four guardrails that I want to cover:

• Pre-commit checklist skill
• Pre-commit Git hook
• GitHub Actions workflow jobs
• Automated pull request reviews

The first guardrail relies on the coding agent following instructions, while the others are deterministic mechanisms that enforce quality regardless of the agent’s behavior. Together, they provide defense in depth: if one guardrail is bypassed or ignored, the others still catch problems.

Let’s dive into each guardrail in turn, starting with the pre-commit checklist.

Pre-commit checklist skill

The first guardrail is the pre-commit checklist provided by the commit-guidelines skill that is part of the idea-to-code plugin in the Humans in the Loop workflow and tools repository.

This pre-commit checklist instructs the coding agent to run several quality and safety checks before creating a Git commit, including:

• Dead code detection (using vulture for Python)
• Linting with auto-fix (using ruff) to enforce coding standards
• Static type checking for Python (using pyright)
• Shell script linting for new or modified .sh files (using shellcheck)
• Code health analysis using CodeScene, requiring a score of 10 for all modified files
• Test coverage verification to ensure tests cover new or changed production code

If any quality gates fail, the agent must stop and fix the issues before committing.

CodeScene provides incredibly valuable feedback that has significantly improved the quality of the code generated by Claude Code. For example, requiring a CodeScene score of 10 for all modified files - in other words, the boy scout rule - has significantly improved the maintainability of the Idea To Code code base. One challenge, however, is that Claude Code often tries to justify not addressing code smells identified by CodeScene, such as claiming they are “pre-existing” or otherwise “acceptable”. This is one of the motivations for implementing the next guardrail.

Pre-commit Git hook

The second guardrail is a pre-commit Git hook that automatically runs quality and security checks before allowing commits to be made. It’s implemented using the pre-commit framework, which is configured by a .pre-commit-config.yaml file. See, for example, the .pre-commit-config.yaml files for the Idea to Code, and Eventuate RealGuardIO repositories.

In addition to running the tests, the pre-commit hook largely repeats the pre-commit checklist. The pre-commit hook does not run the dead code detection tool since its output needs to be interpreted and acted upon by the coding agent, which is not something a pre-commit hook can do. The hook also runs gitleaks to prevent secrets from being committed.

The pre-commit hook is essential because, unlike a Claude Code skill, it is deterministic. Its checks don’t rely on the GenAI-based coding agent following the checklist, which is often not the case. Fortunately, Claude Code hasn’t tried to bypass this pre-commit hook like I described previously. And even if it did, there are two more guardrails.

GitHub Actions workflow

The third guardrail is the set of checks performed by the GitHub Actions workflow that runs on every push or merge. A GitHub Actions workflow runs the tests. It also has one or more lint jobs.

For example, the Eventuate RealGuardIO repository’s workflow has a lint job that performs various quality and security checks on the codebase, including shell script linting and secret scanning using gitleaks. So does the Idea To Code repository’s GitHub Action’s workflow.

These checks will be performed even if the GenAI-based coding agent bypasses the pre-commit hook. In a later post, I will describe how to block a GenAI-based coding agent from modifying the GitHub Actions workflow files, which provides an additional layer of protection.

Automated GitHub status checks: CodeScene pull request review

The fourth and final guardrail is a set of automated checks that are performed when a pull request is created or a new commit is added. The failure of one of these GitHub status checks can prevent the pull request from being merged until the issues are addressed.

For example, I’ve configured CodeScene to automatically review all pull requests for the Eventuate RealGuardIO and Idea to Code repositories and provide feedback on code health. The GitHub status check will fail if the CodeScene score is below 10. It can also trigger a coding agent to respond to the CodeScene review and push a new commit that addresses the issues. However, these days CodeScene issues are almost always detected earlier by the Git pre-commit hook, and so the pull request reviews rarely fail.

What’s next

In future articles, I’ll describe other elements of the GenAI-based development (aka. Harness) platform that I’m developing.

Need help with modernizing your architecture?

I help organizations modernize safely and avoid creating a modern legacy system — a new architecture with the same old problems. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →