OpenAI Open-Sourced Codex Security. Here’s How It Compares to Kira.

On July 29, OpenAI open-sourced Codex Security - an application security agent built on top of gpt-5.6-terra, released under Apache 2.0. It validates something we’ve believed at Kira since we started: AI belongs in application security.

But it also forces a question the industry has been dancing around: should AI replace the security analysis pipeline, or should it sit on top of one?

We spent time studying the Codex Security codebase, its documentation, and the early community feedback. Here’s what we found - and where we think both approaches land.

---

What Codex Security actually is

OpenAI calls Codex Security an “application security agent” rather than a scanner. The open-source repository (v0.1.1) is a TypeScript CLI and SDK that handles authentication, scan orchestration, output formatting, and artifact validation. The vulnerability reasoning happens entirely on OpenAI’s server-side Codex agent platform - the open-source CLI contains no AST parser, no taint propagation engine, no dataflow graph.

The scanning flow:

1. You point the CLI at a repository, a set of paths, or a diff between git refs.
2. The tool builds a codebase-specific threat model. You can feed it additional threat model documents (.md, .txt, .pdf) for more context.
3. According to its documentation, it scans “connected repositories commit-by-commit” and builds scan context from the repo.
4. The model reasons about vulnerabilities and streams findings back through the CLI.
5. High-signal findings are validated in an isolated sandbox environment.
6. Results are validated client-side (SHA-256 seals, schema checks) and formatted into SARIF, CSV, JSON, or Markdown.

The tool supports standard and deep scan modes, configurable cost limits (--max-cost), parallel workers, and discovery limits. For approved findings, it generates bounded patches - proposed fixes scoped to the specific vulnerability. Access requires a ChatGPT Enterprise, Edu, Business, or Pro account with Trusted Access for Cyber verification.

Notable features include a coverage.json that documents what was and wasn’t analyzed, and adaptive false positive feedback that incorporates user corrections into subsequent scans.

---

Why we chose a different architecture

When I started building Kira, I had this exact architectural choice in front of me. Go full LLM - send code to a model, get findings back, done. Or build a deterministic analysis engine and layer AI on top for the things structured analysis can’t handle.

I chose the hybrid path. Codex Security’s release is a useful case study for why.

Kira’s core engine performs structured analysis: AST parsing and dataflow graph construction. It produces deterministic, reproducible results with exact source-to-sink traces. AI is layered on top to reason about business logic and handle the long tail of vulnerability patterns that no rule set can cover.

This isn’t a philosophical preference. It’s a direct response to what breaks when analysis is purely LLM-driven.

Here’s a concrete example.

Consider this Express.js code:

app.get('/user', (req, res) => {
    const id = req.query.id;
    const filter = "SELECT * FROM users WHERE id = '" + id + "'";
    db.query(filter, (err, results) => { res.json(results); });
});

An LLM-driven tool like Codex Security would likely flag this as a SQL injection and generate a correct explanation. But because LLM inference is non-deterministic by nature, a different run could produce different wording, a different severity assessment, or a different conclusion about the finding.

Kira flags the same issue with a mechanically verified dataflow trace:

Source:  req.query.id → line 2, routes/user.js
↓ assignment to id
↓ string concatenation → line 3
↓ assignment to filter

Sink: db.query(filter) → line 4
CWE: CWE-89 (SQL Injection)

Status: CONFIRMED  -  no parameterization or sanitization detected

Every step is anchored to a specific code location. A developer can follow the trace and confirm the issue in seconds. The finding is deterministic - it will appear on every scan, unchanged, until the code is fixed.

Now make it harder:

// routes/user.js
app.get('/user', (req, res) => {
    const id = req.query.id;
    const result = userService.findById(id);
    res.json(result);
});

// userService.js
function findById(userId) {
    return db.query("SELECT * FROM users WHERE id = '" + userId + "'");
}

The vulnerability now spans two files and a function boundary. Kira resolves cross-file function calls, follows the taint through parameter passing, and confirms the sink is reached without sanitization.

An LLM might catch this for a two-file example. But real-world codebases have taint chains that cross many files through middleware stacks, service layers, ORM abstractions, and framework-specific routing. As the number of files and indirections grows, the reliability of LLM-based tracing decreases - while structured taint analysis remains systematic.

This gap widens with patterns like async/await chains, middleware pipelines where request properties are mutated across handlers, and conditional sanitizers that neutralize XSS but not SQL injection. Structured taint analysis handles these systematically because sources, sinks, and sanitizers are encoded as explicit, testable rules. LLM-based analysis handles them based on training data coverage - which is not auditable or guaranteed.

---

The comparison that matters

---

Where the LLM-only approach has advantages

Novel and logic vulnerabilities. An LLM can reason about business logic flaws, race conditions, and unusual anti-patterns that no rule set covers. Structured analysis generally won’t catch application-specific issues like a flawed token refresh flow or an authorization bypass that depends on specific ordering of API calls.

Instant language coverage. Adding a new language to Kira requires building a parser and writing framework-specific analyzers. Codex Security can scan any language the model can reason about immediately, with zero setup.

Patch generation. Generating scoped fixes for approved findings reduces the gap between “vulnerability found” and “vulnerability fixed.” Kira surfaces remediation guidance but doesn’t yet generate ready-to-apply patches. This is on our roadmap.

---

What Kira has actually found

Benchmarks are useful, but the real test of a security tool is whether it finds vulnerabilities that matter - in production code, in projects people depend on. Kira has been used to responsibly disclose dozens of vulnerabilities across dozens of open-source projects. All were patched. Here are some of them.

Hoppscotch - CVE-2026-50160, CVSS 10.0. Kira identified a mass assignment vulnerability in the self-hosted backend’s onboarding endpoint. An unauthenticated attacker could inject JWT_SECRET and SESSION_SECRET through the setup API, enabling complete server takeover. Missing whitelist validation on the ValidationPipe allowed arbitrary configuration keys to pass through unchecked. Patched in v2026.5.0. Advisory: GHSA-j542-4rch-8hwf ↗

Onyx AI - CVSS 8.7, CWE-798. Kira traced a multi-step attack chain: the SUPER_CLOUD_API_KEY defaulted to the literal string "api_key" in the open-source repository. This credential protected the /tenants/impersonate endpoint, which performs a global user lookup across all tenants and returns a valid session token for any email address. Silent impersonation with access to every connected data source - Google Drive, Confluence, Slack, GitHub, Jira, Notion. No notification to the victim, no audit entry. Fixed with the default removed, comparison switched to secrets.compare_digest, and an IMPERSONATION_ENABLED feature flag added (default off).

LiteLLM - CVSS 9.0. An org_admin could elevate any user on the platform to proxy_admin via bulk update, with access spanning all tenants. Two compounding authorization failures: Pydantic silently dropped the organization_id field before the handler ran, and user_role passed through with no elevation check. Patched in v1.83.7.

Microsoft VibeVoice - CVSS 7.8. Checkpoint conversion scripts called torch.load() on attacker-supplied file paths without weights_only=True, allowing arbitrary code execution during deserialization. Microsoft patched across the repository and credited Offgrid Security.

Ghost CMS - CVE-2026-53945, CVSS 5.5. Webhook delivery used an unprotected HTTP client while every other external request path in Ghost used the hardened version with private-range blocking. An admin could register webhooks targeting AWS/GCP/Azure metadata endpoints. Single-line fix: swap to the hardened client. Patched in v6.21.1.

Cognithor - CVSS 9.8. The bootstrap endpoint returned the application’s master bearer token to any caller with zero authentication, with the API server binding to 0.0.0.0 by default. One GET request exposed 14 API keys including OpenAI, Anthropic, and PostgreSQL credentials. Fixed in v0.78.2.

Redash - CVSS 6.8. Three query runners (Elasticsearch, Graphite, Prometheus) passed user-supplied URLs directly to requests.get() with no scheme check or private-range blocklist. The Elasticsearch runner embedded full response bodies from internal targets in API responses - including, on EC2 with IMDSv1, IAM credentials in a single API call.

All responsibly disclosed and patched.

These aren’t benchmark test cases. They’re real vulnerabilities in production open-source software - the kind that end up in breach reports when they go undetected. Every one of them was found by Kira’s analysis engine, validated with proof-of-concept exploits, and responsibly disclosed to the maintainers.

---

The bigger picture

The security tooling industry is splitting into two camps.

Camp 1: AI replaces the analysis pipeline. Send code to a model, get findings back. The bet is that models will keep getting better until they close the gaps. Codex Security falls in this camp.

Camp 2: AI enhances a deterministic foundation. Build structured analysis - taint tracking, data flow, framework-aware rules - and use AI for what it’s genuinely better at: business logic, novel patterns, contextual validation. Kira is in this camp.

I think the hybrid approach is right - not because LLMs aren’t powerful, but because security demands properties that pure LLM systems can’t yet guarantee. Determinism, so your CI pipeline gives consistent results. Traceability, so your findings hold up in a SOC 2 audit. Evidence, so a developer can verify a vulnerability without trusting a model’s reasoning.

The community feedback on Codex Security already shows this tension. People are excited about the tool’s reach, but immediately asking for the structured evidence and reproducibility that comes from deterministic analysis. That’s a signal about what production security workflows actually require.

AI makes a strong scanner stronger. It doesn’t yet replace the need for one.

Kira finds security vulnerabilities across web applications, APIs, and infrastructure - from injection flaws and exposed secrets to misconfigurations and business logic issues. Try it on your own codebase ↗