Vibe Coder #5.3: Security Audit – Hunting Hidden Mines in AI Code
Three happy-path tests pass while the class carries three mines: input embedded in SQL, eval() and a password in the code. We read agent-written code the way OWASP would, then see what a token-based PHP gate, Semgrep taint tracking and the AI reviewers of four ecosystems catch and miss.
This post is also available in Turkish →
We closed the previous chapter on this line: the tests are green and the seal holds, but green doesn't mean safe; a query wide open to SQL injection will happily pass its own test. This chapter starts by proving it. The topic is AI code security: reading agent-written code the way OWASP would, finding the mines buried in it, and catching the next ones at the gate.
Last month, in the PHP2Shell post, the fire was in PHP itself, and all we could do was shut a door and wait for the patch. These mines are different. PHP is fine and the server is up to date; the mines are laid by code that entered our repository with our approval. Nobody buried them on purpose. The agent took the shortest path, and the shortest path ran straight through the minefield.
That's the metaphor for this chapter: a minefield. We'll carry two tools. One is a metal detector: fast, beeps a lot, blind to plastic mines. The other is a tracker: it follows input from where it enters to where it ends up, but only along paths it knows. By the end you'll see, in numbers, what each one finds and what each one misses.
The code keeps the series' Turkish identifiers and messages; the comments next to each block explain what matters.
Why does AI-written code need its own security review?#
Because it works. That's the problem. Everyone notices code that doesn't run; only someone who looks notices code that runs and leaves a door open.
Veracode's 2025 GenAI Code Security report (30 July 2025) gave 80 coding tasks to more than 100 language models. Result: a security flaw in 45% of cases. Java failed more than 70% of the time; Python, C# and JavaScript between 38% and 45%. Models failed to defend against cross-site scripting (CWE-80) in 86% of cases and against log injection (CWE-117) in 88%. PHP isn't in the report, so I have no number for our language. We'll run a small PHP measurement of our own in a minute.
OWASP Top 10:2025 tells us what to look for. In the eighth edition of the list, A05 Injection dropped two places to fifth, but it is still one of the most tested categories, with 38 CWEs under it. This chapter's mines land in three categories: input embedded in SQL and eval() under A05, the password in the code somewhere between A04 Cryptographic Failures and A07 Authentication Failures, and installing a package that doesn't exist under A03 Software Supply Chain Failures, new to the list in this edition.
Three mines, three green tests#
The scenario: product search plus a campaign discount. Picture an agent being told "search products by name and apply the campaign formula from the admin panel (something like fiyat * 0.9, where fiyat is price) to the price". I wrote the class below myself, not an agent, but I deliberately put in three patterns we see often in agent output. What I wanted to measure isn't the agent; it's what tests and reviews can and can't see.
<?php
declare(strict_types=1);
final class UrunArama
{
public function __construct(private PDO $db)
{
}
public static function baglan(): self
{
return new self(new PDO('mysql:host=127.0.0.1;dbname=magaza', 'magaza', 'Magaza-2026!')); // Mayın 1: kodda şifre
}
/**
* Ürünleri ada göre arar, kampanya formülünü fiyata uygular ("fiyat * 0.9" gibi).
*
* @return list<array{ad: string, fiyat: float}>
*/
public function ara(string $kelime, string $formul = 'fiyat'): array
{
$sql = "SELECT ad, fiyat FROM urunler WHERE ad LIKE '%$kelime%' ORDER BY ad"; // Mayın 2: SQL'e gömülü girdi
$ifade = str_replace('fiyat', '$fiyat', $formul);
$sonuc = [];
foreach ($this->db->query($sql)->fetchAll(PDO::FETCH_ASSOC) as $satir) {
$fiyat = (float) $satir['fiyat'];
$sonuc[] = ['ad' => (string) $satir['ad'], 'fiyat' => (float) eval("return $ifade;")]; // Mayın 3: eval
}
return $sonuc;
}
}
The tests follow the happy path too: finds by name, applies the formula, returns empty when nothing matches. I ran them with an in-memory SQLite database on PHPUnit 9.6.17 (8 November 2026, PHP 8.3.6 CLI):
$ phpunit
OK (3 tests, 3 assertions)
Three tests, three green, three mines. Now let's ask the same class something nasty:
$ php deneme.php
1) Normal: [{"ad":"Kırmızı Kupa","fiyat":108},{"ad":"Mavi Kupa","fiyat":90}]
2) SQL: [{"ad":"yonetici@ornek.test","fiyat":0}]
3) eval: [{"ad":"Defter","fiyat":41}]
On the second line I typed x' UNION SELECT eposta, sifre_ozeti FROM kullanicilar -- into the search box (eposta is the e-mail column, kullanicilar the users table), and the admin's e-mail address showed up in the product list. On the third line I sent strlen(php_uname()) instead of a formula; a function that reads the server's operating system details ran, and the length of its output was listed as a price. Any other function could have gone in place of strlen. I didn't put one there.
Hard truth: the tests aren't wrong. They asked what we wanted, not what we didn't want. The wish was "find the products"; "and nothing else" was never part of it.
Mine 1: A password in the code#
The quietest mine, because no test ever runs it. baglan() (connect) lands in the repository with the first commit, and one day the repository is opened to another team, a contractor, or by accident to everyone. According to GitGuardian's 2025 report, 23.8 million secrets leaked on public GitHub repositories in 2024, and 70% of those leaked in 2022 were still valid when the report was written. Deleting the line in a new commit isn't enough; git history doesn't forget. A leaked password gets rotated, not deleted.
Technical detail: the agent isn't being malicious. You said "set up the connection", and the shortest working connection is the line with the password in it. Reading .env and failing with a clear error when it's missing is extra work.
Mine 2: Input embedded in SQL#
Old as the hills and still alive. In "... LIKE '%$kelime%'" the input becomes part of the command instead of part of the data. The cure hasn't changed in years: a prepared statement and a bound parameter. One subtlety: even with a parameter, % and _ stay wildcards inside LIKE. Returning the whole table when someone searches for % isn't injection, but it isn't what you meant either.
Mine 3: The pull of eval()#
"Let the formula come from the panel" leads an agent straight to eval(), because the shortest way to run a formula is to run it as code. The PHP manual is blunt: eval() is "very dangerous" because it allows execution of arbitrary PHP code; its use is discouraged, and user-provided data must never reach it without validation. For its cousin unserialize(), the manual's warning reads: don't pass untrusted input to it, whatever allowed_classes says; use JSON.
Bonus mine: a package that doesn't exist#
The fourth one isn't in the code but in a composer require line. Spracklen and colleagues' study of package hallucinations found, across 576,000 code samples, that commercial models suggest a non-existent package at least 5.2% of the time on average and open models 21.7%, for 205,474 distinct made-up names in total. An attacker only has to register one of those names. composer audit checks known security advisories; it won't recognise a malicious package published yesterday. When an agent proposes a new package, open its Packagist page, its maintainer and its age yourself.
Clearing the mines#
The clean version does three things: it reads the password from the environment, builds the query with a parameter, and turns the formula from code into data. What comes from the panel is no longer an expression to run but a discount type (Indirim: none, percentage, fixed amount) and a number.
<?php
declare(strict_types=1);
enum Indirim: string
{
case Yok = 'yok';
case Yuzde = 'yuzde';
case Tutar = 'tutar';
}
final class UrunArama
{
public function __construct(private PDO $db)
{
}
public static function baglan(): self
{
$dsn = getenv('DB_DSN');
if ($dsn === false || $dsn === '') {
throw new RuntimeException('DB_DSN tanımlı değil.');
}
return new self(new PDO($dsn, getenv('DB_KULLANICI') ?: null, getenv('DB_SIFRE') ?: null));
}
/** @return list<array{ad: string, fiyat: float}> */
public function ara(string $kelime, Indirim $indirim = Indirim::Yok, float $deger = 0.0): array
{
$kalip = '%' . addcslashes($kelime, '%_\\') . '%';
$sorgu = $this->db->prepare("SELECT ad, fiyat FROM urunler WHERE ad LIKE ? ESCAPE '\\' ORDER BY ad");
$sorgu->execute([$kalip]);
$sonuc = [];
foreach ($sorgu->fetchAll(PDO::FETCH_ASSOC) as $satir) {
$fiyat = (float) $satir['fiyat'];
$sonuc[] = ['ad' => (string) $satir['ad'], 'fiyat' => match ($indirim) {
Indirim::Yok => $fiyat,
Indirim::Yuzde => round($fiyat * (1 - min(max($deger, 0.0), 90.0) / 100), 2),
Indirim::Tutar => max(round($fiyat - $deger, 2), 0.0),
}];
}
return $sonuc;
}
}
Reality: "make the formula data" costs some flexibility. The panel can no longer write fiyat * 0.9 + 5. If you truly need free-form formulas, the answer is not eval() but a small parser that knows only numbers, fiyat and the four operators. Most campaigns fit into three types, though; try that first.
Following the logic of 5.2, I added two more wishes to the test class, this time spelling out what we don't want:
/** Kötü niyetli dilek: girdi sorgunun parçası olmamalı. */
public function testSqlGirdisiVeriOlarakKalir(): void
{
self::assertSame([], $this->arama->ara("x' UNION SELECT eposta, sifre_ozeti FROM kullanicilar --"));
}
public function testJokerKarakterlerDuzMetindir(): void
{
self::assertSame(['%50 Kupon'], array_column($this->arama->ara('%'), 'ad'));
}
The clean version gives OK (5 tests, 5 assertions). Run the same two tests against the mined version and both go red: Tests: 2, Assertions: 2, Failures: 2. A security test is a wish too. If you don't write it, the genie has no reason to grant it.
The metal detector: looking for mine patterns in changed files#
You'd need a separate test for every mine, and you don't know which mine you forgot. The second line of defence is a gate that looks for known patterns in every PHP file the agent changed. The script below works with PHP's own tokenizer (token_get_all()) rather than regex, so it doesn't beep at the word eval in a comment or a string. It looks for three things: eval() and functions that run commands or build objects from input, query()/exec()/mysqli_query() calls whose argument contains a variable or concatenation, and passwords sitting in the code. If it finds any, it exits with 2; as you may remember from 4.2 and 5.2, agent hooks read 2 as "go back and fix it".
<?php
declare(strict_types=1);
/**
* Mayın kapısı: değişen PHP dosyalarında üç mayın türünü arar.
* php mayin-kapisi.php [taban] taban verilmezse HEAD; izlenmeyen yeni dosyalar da taranır
* Çıkış: 0 temiz, 2 mayın var (ajan kancaları 2'yi "geri dön, düzelt" diye anlar), 1 kullanım hatası.
* Bu bir kapı, denetçi değil: yalnızca kalıp arar, veri akışını izlemez.
*/
const TEHLIKELI_FONKSIYONLAR = [
'exec', 'shell_exec', 'system', 'passthru', 'proc_open', 'popen',
'unserialize', 'create_function', 'assert',
];
const SORGU_METOTLARI = ['query', 'exec', 'mysqli_query', 'mysqli_multi_query'];
const SIR_ADI = '/(pass(word)?|passwd|sifre|parola|secret|token|api_?key|anahtar)/i';
/** @return list<string> */
function degisenDosyalar(string $taban): array
{
$komutlar = [
'git diff --name-only --diff-filter=ACMR ' . escapeshellarg($taban),
'git ls-files --others --exclude-standard',
];
$dosyalar = [];
foreach ($komutlar as $komut) {
exec($komut . ' 2>/dev/null', $cikti, $kod);
if ($kod !== 0) {
fwrite(STDERR, "Git komutu başarısız: $komut\n");
exit(1);
}
$dosyalar = array_merge($dosyalar, $cikti);
$cikti = [];
}
return array_values(array_unique(array_filter(
$dosyalar,
static fn (string $yol): bool => str_ends_with($yol, '.php') && is_file($yol) && $yol !== 'mayin-kapisi.php'
)));
}
/**
* Açılan parantezden kapanana kadar argümanları üst düzey virgüllerden böler.
*
* @param list<mixed> $jetonlar
* @return list<list<mixed>>
*/
function argumanlar(array $jetonlar, int $acilis): array
{
$derinlik = 0;
$liste = [[]];
for ($i = $acilis, $n = count($jetonlar); $i < $n; $i++) {
$j = $jetonlar[$i];
if ($j === '(' || $j === '[') {
if ($derinlik++ === 0) {
continue;
}
} elseif ($j === ')' || $j === ']') {
if (--$derinlik === 0) {
break;
}
} elseif ($j === ',' && $derinlik === 1) {
$liste[] = [];
continue;
}
if (!is_array($j) || !in_array($j[0], [T_WHITESPACE, T_COMMENT, T_DOC_COMMENT], true)) {
$liste[count($liste) - 1][] = $j;
}
}
return $liste;
}
function sonrakiAnlamli(array $jetonlar, int $i): int
{
while (isset($jetonlar[++$i]) && is_array($jetonlar[$i]) && $jetonlar[$i][0] === T_WHITESPACE) {
}
return $i;
}
/** @return list<string> "satır: açıklama" */
function tara(string $yol): array
{
$jetonlar = token_get_all((string) file_get_contents($yol));
$bulgular = [];
$satir = 1;
$tersTirnak = false;
foreach ($jetonlar as $i => $j) {
if (!is_array($j)) {
if ($j === '`' && ($tersTirnak = !$tersTirnak)) {
$bulgular[] = "$satir: ters tırnakla kabuk komutu";
}
continue;
}
[$tur, $metin, $satir] = $j;
if ($tur === T_EVAL) {
$bulgular[] = "$satir: eval() — formülü veri yap, kod yapma";
continue;
}
if ($tur !== T_STRING) {
continue;
}
$sonraki = sonrakiAnlamli($jetonlar, $i);
if (($jetonlar[$sonraki] ?? null) !== '(') {
continue;
}
$isim = strtolower($metin);
$once = $jetonlar[$i - 1] ?? null;
$metotMu = is_array($once) && in_array($once[0], [T_OBJECT_OPERATOR, T_NULLSAFE_OBJECT_OPERATOR, T_DOUBLE_COLON], true);
$tanimMi = is_array($once) && $once[0] === T_WHITESPACE
&& is_array($jetonlar[$i - 2] ?? null) && $jetonlar[$i - 2][0] === T_FUNCTION;
if (!$metotMu && !$tanimMi && in_array($isim, TEHLIKELI_FONKSIYONLAR, true)) {
$bulgular[] = "$satir: $metin() — girdiyle buluşursa komut/nesne çalıştırır";
}
if (in_array($isim, SORGU_METOTLARI, true) && !$tanimMi) {
$ilk = argumanlar($jetonlar, $sonraki)[$isim === 'mysqli_query' || $isim === 'mysqli_multi_query' ? 1 : 0] ?? [];
foreach ($ilk as $parca) {
if ($parca === '.' || $parca === '"' || (is_array($parca) && in_array($parca[0], [T_VARIABLE, T_CURLY_OPEN, T_DOLLAR_OPEN_CURLY_BRACES], true))) {
$bulgular[] = "$satir: $metin() değişkenle kurulan sorgu — prepare() ve parametre kullan";
break;
}
}
}
if ($isim === 'pdo' && is_array($once) && ($once[0] === T_WHITESPACE) && ($jetonlar[$i - 2][0] ?? null) === T_NEW) {
$ucuncu = argumanlar($jetonlar, $sonraki)[2] ?? [];
if (isset($ucuncu[0]) && is_array($ucuncu[0]) && $ucuncu[0][0] === T_CONSTANT_ENCAPSED_STRING && strlen($ucuncu[0][1]) > 2) {
$bulgular[] = "$satir: new PDO() şifresi kodda — ortam değişkeninden oku";
}
}
}
// Adı sır gibi duran bir şeye en az sekiz karakterlik sabit metin atanıyor mu?
foreach (file($yol) ?: [] as $no => $satirMetni) {
if (preg_match('/([A-Za-z_]\w*)[\'"]?\s*(=>|=)\s*[\'"]([^\'"\s]{8,})[\'"]/', $satirMetni, $m) === 1
&& preg_match(SIR_ADI, $m[1]) === 1) {
$bulgular[] = ($no + 1) . ": {$m[1]} sabit metin — sırrı koda değil ortama koy";
}
}
return $bulgular;
}
if (!is_dir('.git')) {
fwrite(STDERR, "Git deposunun kökünden çalıştırın.\n");
exit(1);
}
$toplam = 0;
foreach (degisenDosyalar($argv[1] ?? 'HEAD') as $yol) {
foreach (tara($yol) as $bulgu) {
fwrite(STDERR, "$yol:$bulgu\n");
$toplam++;
}
}
if ($toplam > 0) {
fwrite(STDERR, "$toplam mayın. Kapı kapalı.\n");
exit(2);
}
echo "Değişen dosyalarda bilinen mayın kalıbı yok.\n";
exit(0);
With the agent's files not yet committed, I ran it against the skeleton commit:
$ php mayin-kapisi.php
src/UrunArama.php:13: new PDO() şifresi kodda — ortam değişkeninden oku
src/UrunArama.php:26: query() değişkenle kurulan sorgu — prepare() ve parametre kullan
src/UrunArama.php:28: eval() — formülü veri yap, kod yapma
3 mayın. Kapı kapalı.
$ echo $?
2
On the clean version: Değişen dosyalarda bilinen mayın kalıbı yok. ("no known mine pattern in changed files"), exit 0. I also tried it on a small edge-case file: exec() built by concatenation, unserialize(), backticks, mysqli_query() with a variable, a hard-coded ODEME_API_ANAHTARI (payment API key) and a 'password' array key were all caught. A constant query('SELECT 1'), prepare()->execute(), json_decode(), getenv('DB_SIFRE') and a password-less new PDO('sqlite::memory:') went through quietly.
Hard truth: the detector beeps a lot. $sql = "SELECT COUNT(*) FROM urunler"; $db->query($sql); is perfectly harmless, but the gate stops it anyway; it doesn't know what's in the variable. I left it that way on purpose. Writing prepare() costs the agent nothing; missing a mine doesn't. The other direction has a limit too: it can't see plastic mines. Code that first puts $kelime into an array and then feeds it into the query from there passes if it doesn't match the pattern.
The tracker: following the input's path#
The detector looks at shape. The tracker asks a different question: does data from outside reach a dangerous place without being cleaned? That's taint analysis. Semgrep can do it in its Community Edition with a rule file; it's developed by Semgrep, Inc. and licensed LGPL-2.1. I installed version 1.184.0 in a cloud sandbox and wrote this rule:
rules:
- id: girdi-sorguya-akiyor
mode: taint
languages: [php]
severity: ERROR
message: Dış girdi SQL sorgusuna ya da eval'e akıyor. prepare() ve parametre kullanın.
pattern-sources:
- pattern: $_GET
- pattern: $_POST
- pattern: $_REQUEST
- pattern: $_COOKIE
pattern-sinks:
- pattern: $DB->query(...)
- pattern: $DB->exec(...)
- pattern: mysqli_query($C, ...)
- pattern: eval(...)
I tried three cases: a function that concatenates $_GET into a query inside the same function, a function that uses the same data through prepare(), and a call that passes $_GET as a parameter to another function that embeds it in a query. Plus the mined UrunArama class.
$ semgrep --config mayin.yml --metrics=off --no-git-ignore akis.php eski/UrunArama.php
10┆ return $db->query($sql)->fetchAll();
• Findings: 1 (1 blocking)
One finding, and the right one: the flow inside a single function. It stayed silent on the prepare() function, which is good. But it missed two mines. It didn't see the flow crossing a function boundary because, per Semgrep's documentation, interprocedural taint analysis is a Pro feature. And the UrunArama class has no $_GET at all; input arrives as a parameter, so the tracker has no source to follow.
Result: the detector found all three mines in the class but also beeped at a harmless line; the tracker never beeped wrongly but found nothing in the class. They aren't two tools for one question; they answer two different questions. Put both at the gate.
Security reviewers across four ecosystems#
Model-based reviewers now exist in every ecosystem. They don't match patterns; they read code, so they can catch what the two tools above can't (authorisation bypass, access to the wrong object). I read the docs on 8 November 2026 and ran none of them for this post.
| Tool | What it does | Lock or eye? |
|---|---|---|
Claude Code /security-review |
Reviews the diff between your branch and origin's default branch for injection, auth issues and data exposure (commands) | Eye; runs when you ask |
| Claude Code security-guidance plugin | Matches patterns like eval(, pickle, .innerHTML = on every edit, has a separate model review the turn's diff, looks deeper on commit (docs) |
Eye; in the docs' own words, none of the layers block writes or commits |
| OpenAI Codex Security | Formerly Aardvark; builds a project-specific threat model, validates findings in a sandbox where possible, proposes patches (announcement) | Eye; the cloud version is in research preview |
| Gemini CLI security extension | /security:analyze reviews the local git diff for hard-coded secrets, injection and access control; /security:scan-deps scans dependencies with OSV-Scanner (repo, Apache-2.0) |
Eye |
Semgrep CE, composer audit |
Rule-based scanning, known advisories | Can be a gate; with an exit code in CI |
The last column is the point of this chapter. A model reviewer is a second pair of eyes; it hands findings back to the agent and the agent fixes them. But a model can miss things, and none of these lock the door. A lock is a command that stops with an exit code. Adding the mine gate to the Claude Code Stop hook from 5.2 takes one line:
{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "cd \"$CLAUDE_PROJECT_DIR\" && php test-muhru.php kontrol && php mayin-kapisi.php origin/main"
}
]
}
]
}
}
In Aider the same chain goes into the test command; in Codex and Cursor, the Stop/stop hooks from 5.2 can call the same command:
aider src/UrunArama.php --read tests/UrunAramaTest.php \
--test-cmd "php test-muhru.php kontrol && php mayin-kapisi.php origin/main" --auto-test
When isn't it enough?#
The detector, the tracker and the model reviewers all look for known kinds of mines. A01 Broken Access Control, number one on the OWASP list, is usually not a pattern but a missing line: a query that never asks "does this order belong to this customer?" No scanner knows your business rules. In the multi-store post we asked "whose data is this?" for every table; that question gets asked by hand here too. It's also why Codex Security starting with a threat model makes sense: first the map of the field, then the detector.
There's an ordering question as well. None of these gates see the commands an agent runs on a server; they look at code. What an agent may touch in production is a job for permission settings, as we covered in Urban Renewal #1.
On the code side we now have the seal and the mine gate, but both only run inside the agent's own session. If the agent skips the gate, disables the hook, or you commit by hand, nobody is checking. The next chapter, Vibe Coder #6.1: CI/CD, the Vibe Way, takes these gates out of the agent's session and into the pipeline every commit has to pass. The full series and the list of published chapters are on the Vibe Coder's Handbook page (in Turkish).
When the detector beeps, stop. When it doesn't, still don't run without a map. Stay with technology.
Comments
0 comments
Leave a comment