GitHub Copilot Code Review hit GA in April 2025, reached one million users within a month, grew 10x through the year, and as of early 2026 has processed 60 million code reviews. That is a meaningful dataset - enough to evaluate the tool honestly rather than from first-impressions or vendor documentation.
The honest evaluation is that it is genuinely useful for a specific category of finding, actively counterproductive if treated as a substitute for SAST, and properly effective when layered with the tools it complements rather than the tools it replaces.
What It Catches Well
Obvious logic errors and edge cases. Copilot Code Review is trained on a large code corpus and applies pattern-matching that a fatigued human reviewer might miss. Off-by-one errors in array iteration, null dereference paths that exist only on specific input combinations, resource handles that are opened but not closed in error paths - these are the review comments that developers report as genuinely useful in practice.
Dependency vulnerability detection. As of October 2025, Copilot Code Review integrates CodeQL and ESLint analysis and surfaces known-vulnerable dependency hashes before a PR opens. This is the tooling doing its job as a first-pass accelerator - a developer adding a dependency with a known CVE gets a review comment at PR creation rather than a separate Dependabot alert after merge.
Style and convention drift. For teams with established patterns, Copilot reads directory structure and existing source files for context and flags deviations from those patterns. This is the least security-relevant capability but the most frequently cited by developers as useful.
Secret scanning integration. Copilot Code Review’s coding agent now runs secret scanning as part of its autonomous workflow. A commit containing a hardcoded API key or credential pattern triggers a finding at the review stage rather than requiring a separate push protection alert.
What It Misses
An arXiv paper (2509.13650, September 2025) evaluated Copilot Code Review specifically for security vulnerabilities. The results are sobering:
SQL injection, XSS, and insecure deserialization are frequently missed. These are the OWASP Top 10 vulnerabilities that require understanding data flow from input to sink across multiple functions. Copilot Code Review evaluates context within a PR diff but does not perform full dataflow analysis across the codebase. A SQL injection that sources from a request parameter parsed in one file and concatenated into a query in another file - with three function calls in between - is invisible to a diff-level review.
Business logic vulnerabilities are invisible. Authorization bypass, insecure direct object reference, race conditions in multi-step operations - these require understanding what the code is supposed to do before you can identify where it fails to do it correctly. Copilot does not have access to requirements, threat models, or the business context that makes a particular flow suspicious.
The hardcoded credential problem. A separate study found 2,702 hardcoded API keys in Copilot-generated code in production. Copilot Code Review did not prevent Copilot from generating code with hardcoded credentials, and did not consistently catch those credentials in subsequent reviews. The model that writes code and the model that reviews code share training data that normalises the pattern.
The Alert Fatigue Problem
The Datadog DevSecOps 2026 Report surfaces a counterintuitive finding: 80% of “critical” dependency vulnerability alerts are not actually critical after context adjustment. Only 18% of flags that arrive as critical remain critical once the tool knows whether the vulnerable package is reachable from an internet-exposed path, whether the vulnerable code path is actually executed, and whether the deployment environment provides compensating controls.
Copilot Code Review at scale without context tuning increases the total volume of alerts that developers receive. Higher alert volume with a significant false-positive rate does not improve security - it trains developers to dismiss alerts. The tool produces high confidence on some categories (dependency versions, obvious patterns) and low confidence on others (dataflow vulnerabilities, business logic) but does not reliably distinguish between them in its output.
The Correct Layering
Copilot Code Review is correctly positioned as a first-pass review accelerator, not a security control. The layering that works in practice:
First pass - Copilot Code Review: Catches dependency issues, obvious logic errors, style deviations. Reduces the cognitive load on human reviewers for the easy categories, letting them focus on the hard ones.
Second pass - SAST with dataflow analysis: CodeQL for taint analysis (SQL injection, XSS, path traversal across function call graphs), Semgrep with custom rules for organisation-specific patterns. These are not redundant with Copilot Code Review - they perform fundamentally different analysis.
Third pass - human security review for high-risk changes: Authentication changes, cryptography implementations, privilege escalation paths, new external integrations. Copilot’s 60 million reviews are a useful complement to, not a substitute for, a security engineer reviewing changes to your OIDC implementation.
# Example pipeline layering
- name: Copilot security scan
# Runs automatically on PR open via GitHub integration
- name: CodeQL analysis
uses: github/codeql-action/analyze@v3
with:
languages: javascript, python
queries: security-extended
- name: Semgrep custom rules
run: semgrep --config=.semgrep/ --error
# Org-specific rules for patterns CodeQL doesn't coverGitHub’s 2026 production-context vulnerability filters help address the alert fatigue problem: you can now filter Copilot Code Review findings by has:deployment, restricting alerts to packages and patterns that are actually present in deployed production services. This does not fix the false-positive rate but it concentrates the signal on findings that matter if exploited.
What Changes If You Already Have Mature SAST
For teams already running CodeQL, Semgrep, and Dependabot with reasonable configuration, Copilot Code Review adds marginal security value. The dependency scanning is already covered by Dependabot. The pattern matching for common vulnerabilities is already covered by CodeQL and Semgrep. What Copilot Code Review adds is the natural language explanation of findings (useful for junior developers) and the PR-integrated UX (useful for review workflow).
The right question is not “should we adopt AI code review?” but “which part of our review workflow is the bottleneck?” If human reviewers are spending time on obvious issues that a first-pass tool could catch, Copilot Code Review addresses that. If the bottleneck is finding the dataflow vulnerabilities that human reviewers miss, Copilot Code Review does not address that - better CodeQL query coverage does.
The 60 million reviews milestone represents a genuinely useful product doing genuinely useful things. It is not, yet, a substitute for the security-specific tools it sits alongside.