SkillSpector: Inspect an Agent Skill Before You Install It – Fail on the Defect, Not the Score

In last week's piece on Moli, an empty table came back with exit code 0. This week's zero is more serious: a report with a critical finding in it, also exiting 0. On the radar is SkillSpector, NVIDIA's open source scanner that inspects the skill packages you give your AI agents before you install them.

A skill is a SKILL.md instruction file, often with a few scripts next to it. Agents such as Claude Code, Codex CLI and Gemini CLI load them and trust what they say. The January 2026 study Agent Skills in the Wild (Yi Liu et al.) collected 42,447 skills from two marketplaces and analysed 31,132 of them: 26.1% had at least one vulnerability, 5.2% showed signs of malicious intent. Skills that bundle scripts were 2.12 times more likely to be vulnerable.

The metaphor for this piece is a roadworthiness inspection. A car does not fail its inspection on points. It fails on a defect: one major defect is enough, and a clean record everywhere else does not save it. SkillSpector adds up points. The gap between those two ideas is what this article is about.

What is SkillSpector?#

SkillSpector is a command line tool that inspects a skill without ever executing it, using regex, Python AST analysis, YARA rules and an optional LLM review, then hands back a risk score from 0 to 100 and an install / caution / do-not-install recommendation. Its README lists 71 patterns across 17 categories: prompt injection, data exfiltration, privilege escalation, supply chain, excessive agency, MCP tool poisoning and more. Declared dependencies are also checked against known vulnerabilities through OSV.dev.

Installing it and running a first scan#

It needs Python 3.12 to 3.14. The documented path:

# CLI only
uv tool install git+https://github.com/NVIDIA/skillspector.git
​
# Static analysis only, no model calls at all
skillspector scan ./my-skill/ --no-llm
​
# Machine-readable report for CI
skillspector scan ./my-skill/ --no-llm --format json --output report.json
​
# MCP server so the agent itself can ask before installing (needs the mcp extra)
uv tool install 'skillspector[mcp] @ git+https://github.com/NVIDIA/skillspector.git'
claude mcp add skillspector -- skillspector mcp

Up front: I did not install or run it myself; the environment I worked in did not allow third-party package installs. Everything below comes from reading the source and documentation at the v2.13.1 tag. I rebuilt the scoring maths from the code's own formula.

How is the inspection score calculated?#

The score is summed by a formula in report.py: CRITICAL 50, HIGH 25, MEDIUM 10, LOW 5. Each finding is multiplied by its confidence (0 to 1); a second hit on the same rule counts half, a third counts a quarter. Findings in an executable script get a 1.3 multiplier; findings in documentation files such as Markdown do not. 0-20 is SAFE, 21-50 CAUTION, 51 and up DO NOT INSTALL.

The default exit code: 0 if the score is 50 or below, 1 above that. The README says so plainly; SAFE and CAUTION collapse into the same zero. I recomputed the formula in PHP:

The hard truth: all four rows go green in CI. The project knows this, which is why the code has a few exceptions: an executable file that ships compiled code or was left out of analysis pushes the score straight to 51. In August a max_issue_severity field was added to the report; the commit message says it exists so "a normalized verdict cannot hide a HIGH finding". The scale weighs correctly. Just don't let the scale open the gate.

Gating on the defect#

That is what the docs recommend as well: read the JSON fields rather than the exit code, or use --fail-on-findings to stop on any active finding. I wanted something in between, where LOW and MEDIUM pass and HIGH or worse stops. Framework-free plain PHP:

<?php
​
declare(strict_types=1);
​
/**
 * Reads a SkillSpector JSON report and makes the install decision
 * on the worst finding instead of the score.
 * Usage: php skill-gate.php report.json
 * Exit: 0 pass, 1 stop, 2 report unreadable.
 */
​
$path = $argv[1] ?? '';
$report = json_decode((string) @file_get_contents($path), true);
​
if (!is_array($report) || !isset($report['risk_assessment'])) {
    fwrite(STDERR, "Cannot read report: {$path}\n");
    exit(2);
}
​
$rank = ['NONE' => 0, 'LOW' => 1, 'MEDIUM' => 2, 'HIGH' => 3, 'CRITICAL' => 4];
$risk = $report['risk_assessment'];
​
// Older reports have no max_issue_severity; derive it from the findings.
$worst = $risk['max_issue_severity'] ?? 'NONE';
if (!isset($risk['max_issue_severity'])) {
    foreach ($report['issues'] ?? [] as $issue) {
        $s = strtoupper((string) ($issue['severity'] ?? 'LOW'));
        if (($rank[$s] ?? 0) > $rank[$worst]) {
            $worst = $s;
        }
    }
}
​
$reasons = [];
​
if (($rank[$worst] ?? 0) >= $rank['HIGH']) {
    $reasons[] = "worst finding {$worst} (score {$risk['score']}, recommendation {$risk['recommendation']})";
}
​
if ($risk['recommendation'] === 'DO_NOT_INSTALL') {
    $reasons[] = 'recommendation DO_NOT_INSTALL';
}
​
// LLM analysis was requested but did not run: the result is static only, don't call it clean.
if (isset($report['metadata']['llm_error'])) {
    $reasons[] = 'LLM analysis unavailable, result is static only';
}
​
if ($reasons === []) {
    echo "Pass: {$risk['recommendation']}, score {$risk['score']}, worst finding {$worst}\n";
    exit(0);
}
​
echo 'Stop: ' . implode('; ', array_unique($reasons)) . PHP_EOL;
exit(1);

php -l is clean. I tested it against four sample reports I wrote by hand from the JSON schema in the README: it stopped the 50-point CAUTION report with "worst finding CRITICAL", caught the HIGH finding inside the 20-point SAFE report, refused to call the report with a failed LLM run clean, and let the empty report through. The sample reports are mine; it has not been run against real SkillSpector output.

Where does the report go?#

Hidden gem: the README's "Trust model and data egress" section. LLM analysis is on by default, and file contents go to the configured provider. If you have not set one, the default is nv_build, NVIDIA's build.nvidia.com endpoint. Without a key, the scan falls back to static analysis and records a note in the report's metadata.llm_error field. If the skill touches client code, use --no-llm or your own Ollama server. Even with --no-llm, dependency names and versions are sent to OSV.dev: coordinates, not contents.

Two more details. The codex_cli provider, which would use Codex CLI for the analysis, is disabled; the stated reason is that its read-only sandbox permits reads of host files. In Urban Renewal #1 we said "read-only" means something different in every tool, and NVIDIA reached the same conclusion. Second, the README's limitations section says patterns may be missed in non-English content. If your team writes skills in another language, do not assume the static layer will catch an "ignore previous instructions" written in it.

When does it help, and when doesn't it?#

It helps: at the pull request step when a skill from outside is added to a team repository; in CI with --format sarif; when you want the agent itself to call the scan_skill tool before installing. The baseline feature mutes accepted findings and shows only new ones.

It doesn't: it is not a sandbox, and the README says as much. It does not isolate a skill you decide to install, does not see runtime behaviour, and cannot read text inside images or encrypted content. The HTTP MCP server has no authentication; it binds only to 127.0.0.1 and ::1, and remote access needs an authenticating reverse proxy with mTLS in front of it.

Maintenance and who built it#

• Licence: Apache-2.0.
• Latest release: v2.13.1, 23 October 2026; 19 release tags since v2.5.0 on 27 July.
• Activity: first commit 21 March 2026; 1,698 commits on main, 1,391 of them in the last 90 days; 122 distinct non-bot author names.
• Whose work it is: the repository sits in the NVIDIA organisation and is the scanning step of NVIDIA's verified skills pipeline. The most non-bot commits and the recent merges come from Narendran Raghavan, followed by yashrajbasav, Mohit Gupta and Keshav Pradeep.
• Security reports: to NVIDIA PSIRT, not GitHub.
• Loose end: the README still points to issue #199 for an stdio MCP initialise hang; the issue itself was closed with a note that the cause was the local runtime environment.

A quick comparison with the alternatives#

Which one for which job: to look at a single skill locally, with nothing leaving the machine, SkillSpector with --no-llm; to inventory every agent setup on developer machines, Agent Scan; to stand up a red-teaming platform, AI-Infra-Guard. Running the skill's scripts through Semgrep or PHPStan as well does no harm; as we said in the Open Code Review piece, these sit next to each other, not in place of each other.

Where it fits in day-to-day work#

In an agent-assisted PHP project, skills live in folders like .claude/skills/, in the same repository as your rule files. Any pull request that touches that folder needs two lines: skillspector scan writes the JSON, php skill-gate.php makes the call. On projects close to client code, keep --no-llm fixed; if you want the LLM layer, run Ollama on your own server.

An inspection station turns a car away on a defect, not on a total. SkillSpector gives you the full defect list; reducing that list to one number and a green light is your choice. Tie the decision at the gate to the worst defect, not to the sum on the scale. Stay tuned to technology.