Skip to content
Report library
Purpose / Data analysis

Skill Creator Skill Security Audit

What the author says it does (original text)

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Independent security check

Do not install or run it yet

This check is incomplete. Only available results are shown below.

Files checked
18
Risks found
6
Could it run dangerous commands?Looks for programs run straight after downloading, remote control of your computer, and hidden commands.Risks found: 1
Medium risk

Starting the reviewer terminates any process using its target port

Source references: 3
What we found

Before binding its default port, the reviewer runs `lsof -ti :<port>` and sends SIGTERM to every returned PID without checking whether the process belongs to this Skill.

Why this matters

If port 3117 is used by a development server, database proxy, or other work, that process can be stopped unexpectedly, interrupting sessions or losing unsaved state.

Starting the non-static viewer first finds every process listening on the target port and sends each PID SIGTERM without checking ownership. If another local application uses the default port 3117, it could be unexpectedly terminated and lose unsaved work. The user can use `--static`, or ask the author to select a free port without killing processes.

eval-viewer/generate_review.py:288In the codeOpen original file
def _kill_port(port: int) -> None:    """Kill any process listening on the given port."""    try:        result = subprocess.run(            ["lsof", "-ti", f":{port}"],            capture_output=True, text=True, timeout=5,        )        for pid_str in result.stdout.strip().split("\n"):            if pid_str.strip():                try:                    os.kill(int(pid_str.strip()), signal.SIGTERM)                except (ProcessLookupError, ValueError):                    pass        if result.stdout.strip():            time.sleep(0.5)    except subprocess.TimeoutExpired:
Show 2 other places
eval-viewer/generate_review.py:387In the codeOpen original file
def main() -> None:    parser = argparse.ArgumentParser(description="Generate and serve eval review")    parser.add_argument("workspace", type=Path, help="Path to workspace directory")    parser.add_argument("--port", "-p", type=int, default=3117, help="Server port (default: 3117)")    parser.add_argument("--skill-name", "-n", type=str, default=None, help="Skill name for header")    parser.add_argument(
eval-viewer/generate_review.py:438In the codeOpen original file
    # Kill any existing process on the target port    port = args.port    _kill_port(port)    handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path)
Could it expose your files or keys?Looks for uploads of files containing passwords or keys, and keys written directly in the code.Risks found: 2
High risk

A malicious test output can inject script into the reviewer and read other embedded content

Source references: 3
What we found

The reviewer reads output files, serializes the data with `json.dumps`, and directly substitutes it inside an HTML `<script>` block without script-context encoding for sequences such as `</script>`. Test-produced files are untrusted input.

Why this matters

When the user opens the reviewer, a crafted output can execute as browser script, read other embedded test outputs, prompts, grades, and feedback, and potentially transmit them or alter review results.

The viewer reads test outputs and inserts their JSON directly into a `<script>`. Python's `json.dumps` does not normally escape `</script>`, so an untrusted output containing a closing tag and script code could execute when the review page opens and access other prompts, outputs, or feedback embedded in that page. Ask the author to use script-context-safe encoding or store data in a non-executable element and parse it.

eval-viewer/generate_review.py:154In the codeOpen original file
    if ext in TEXT_EXTENSIONS:        try:            content = path.read_text(errors="replace")        except OSError:            content = "(Error reading file)"        return {            "name": path.name,            "type": "text",            "content": content,        }    elif ext in IMAGE_EXTENSIONS:
Show 2 other places
eval-viewer/generate_review.py:270In the codeOpen original file
    embedded = {        "skill_name": skill_name,        "runs": runs,        "previous_feedback": previous_feedback,        "previous_outputs": previous_outputs,    }    if benchmark:        embedded["benchmark"] = benchmark    data_json = json.dumps(embedded)    return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};")
eval-viewer/viewer.html:647In the instructionsOpen original file
  <script>    // ---- Embedded data (injected by generate_review.py) ----    /*__EMBEDDED_DATA__*/    // ---- State ----
Medium risk

Packaging can publish credentials and private files stored in the skill directory

Source references: 2
What we found

The packager recursively adds every file, while its exclusion list covers only caches, node_modules, pyc, DS_Store, and root-level evals. It does not exclude `.env`, private keys, token files, version-control metadata, or other secrets, and does not explicitly reject symbolic links.

Why this matters

If the user or generated workflow places credentials, customer data, or internal material in the skill directory, the resulting `.skill` archive may disclose it to installers or distribution recipients.

The packager recursively adds every file under the selected skill directory, while exclusions cover only a few caches, build files, and root-level `evals`. If that directory contains `.env` files, keys, tokens, private samples, or `.git` data, they may enter the distributable `.skill` archive; `is_file()` also does not explicitly reject symlinks. Ask for a pre-package manifest, symlink rejection, and default exclusions for secrets and VCS data.

scripts/package_skill.py:19In the codeOpen original file
# Patterns to exclude when packaging skills.EXCLUDE_DIRS = {"__pycache__", "node_modules"}EXCLUDE_GLOBS = {"*.pyc"}EXCLUDE_FILES = {".DS_Store"}# Directories excluded only at the skill root (not when nested deeper).ROOT_EXCLUDE_DIRS = {"evals"}
Show 1 other places
scripts/package_skill.py:89In the codeOpen original file
    # Create the .skill file (zip format)    try:        with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf:            # Walk through the skill directory, excluding build artifacts            for file_path in skill_path.rglob('*'):                if not file_path.is_file():                    continue                arcname = file_path.relative_to(skill_path.parent)                if should_exclude(arcname):                    print(f"  Skipped: {arcname}")                    continue                zipf.write(file_path, arcname)                print(f"  Added: {arcname}")
Could it delete files or keep running?Looks for broad file deletion, disk overwrites, and programs set to start automatically.Risks found: 1
Low risk

Trigger tests create `.claude/commands` inside the real project

Source references: 3
What we found

The evaluator walks upward from the current directory to find the real project root, then creates `.claude/commands` and a temporary command file there. The file is removed in `finally`, but newly created directories are not removed.

Why this matters

Testing changes the user's project structure and may leave an empty Claude configuration directory. A crash or forced termination may also leave a temporary command file that affects later Claude Code skill discovery.

Trigger testing walks upward to find the real `.claude` project root, then creates `.claude/commands` and a temporary command file there. The file is deleted in `finally`, but no matching cleanup removes newly created directories, so testing can permanently change the project tree. The impact is usually small but may dirty version control or violate expectations that the project remains unchanged. Ask for an isolated temporary project or cleanup of empty directories.

scripts/run_eval.py:22In the codeOpen original file
def find_project_root() -> Path:    """Find the project root by walking up from cwd looking for .claude/.    Mimics how Claude Code discovers its project root, so the command file    we create ends up where claude -p will look for it.    """    current = Path.cwd()    for parent in [current, *current.parents]:        if (parent / ".claude").is_dir():            return parent    return current
Show 2 other places
scripts/run_eval.py:51In the codeOpen original file
    """    unique_id = uuid.uuid4().hex[:8]    clean_name = f"{skill_name}-skill-{unique_id}"    project_commands_dir = Path(project_root) / ".claude" / "commands"    command_file = project_commands_dir / f"{clean_name}.md"    try:        project_commands_dir.mkdir(parents=True, exist_ok=True)        # Use YAML block scalar to avoid breaking on quotes in description        indented_desc = "\n  ".join(skill_description.split("\n"))        command_content = (            f"---\n"            f"description: |\n"            f"  {indented_desc}\n"            f"---\n\n"            f"# {skill_name}\n\n"            f"This skill handles: {skill_description}\n"        )        command_file.write_text(command_content)
scripts/run_eval.py:179In the codeOpen original file
        return triggered    finally:        if command_file.exists():            command_file.unlink()
Could it bypass safety checks?Looks for skipped website security checks, excessive file access, or actions that skip your approval.Risks found: 1
Medium risk

Any website can overwrite feedback.json while the local review server is running

Source references: 3
What we found

The local POST endpoint checks only the path and whether parsed JSON contains `reviews`; it performs no Origin, CSRF-token, or requester validation before writing the feedback file.

Why this matters

A malicious page visited during review could send a request to localhost and erase or forge feedback, causing later skill changes to be based on manipulated user decisions.

The service is localhost-only, but its POST handler checks neither Origin, a CSRF token, nor Content-Type. It only requires JSON containing `reviews`, then overwrites the workspace's `feedback.json`. While the service runs, a malicious webpage could send JSON text using a request that avoids preflight; it need not read the response to corrupt feedback used for later skill decisions. Ask for a random token, Origin validation, and strict content-type checks.

eval-viewer/generate_review.py:361In the codeOpen original file
    def do_POST(self) -> None:        if self.path == "/api/feedback":            length = int(self.headers.get("Content-Length", 0))            body = self.rfile.read(length)            try:                data = json.loads(body)                if not isinstance(data, dict) or "reviews" not in data:                    raise ValueError("Expected JSON object with 'reviews' key")                self.feedback_path.write_text(json.dumps(data, indent=2) + "\n")                resp = b'{"ok":true}'                self.send_response(200)            except (json.JSONDecodeError, OSError, ValueError) as e:                resp = json.dumps({"error": str(e)}).encode()                self.send_response(500)            self.send_header("Content-Type", "application/json")            self.send_header("Content-Length", str(len(resp)))            self.end_headers()            self.wfile.write(resp)        else:
Show 2 other places
eval-viewer/generate_review.py:441In the codeOpen original file
    _kill_port(port)    handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path)    try:        server = HTTPServer(("127.0.0.1", port), handler)    except OSError:
eval-viewer/generate_review.py:438In the codeOpen original file
    # Kill any existing process on the target port    port = args.port    _kill_port(port)    handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path)    try:        server = HTTPServer(("127.0.0.1", port), handler)    except OSError:        # Port still in use after kill attempt — find a free one        server = HTTPServer(("127.0.0.1", 0), handler)        port = server.server_address[1]
Could it mislead the AI or hide text?Checks the skill instructions for requests to ignore you, influence the report, or hide text in invisible characters.No risks found
Could it change links or payment recipients without asking?Looks for forced referral or payment changes combined with instructions to hide the change.Risks found: 1
Medium risk

The optimizer bypasses the nested-session guard and heavily uses the current account's Claude authentication

Source references: 5
What we found

The code explicitly removes `CLAUDECODE` to permit nested `claude -p` processes and relies on existing session authentication. Defaults run every query three times for up to five iterations; with the suggested roughly 20 queries, one optimization can make hundreds of account calls.

Why this matters

This can rapidly consume paid usage, rate limits, or organization quotas, while submitting test queries and full skill content to the model service.

The optimizer explicitly reuses the current Claude Code authentication and removes the `CLAUDECODE` guard before launching nested `claude -p`. With the documented 20 queries and defaults of three runs per query and five iterations, evaluation alone can reach about 300 model calls, plus rewrite calls. This serves the stated optimization purpose but may consume account quota, money, and time. Ask for an explicit call budget or reduce queries, iterations, or repetitions.

scripts/improve_description.py:4In the codeOpen original file
Takes eval results (from run_eval.py) and generates an improved descriptionby calling `claude -p` as a subprocess (same auth pattern as run_eval.py —uses the session's Claude Code auth, no separate ANTHROPIC_API_KEY needed)."""
Show 4 other places
scripts/run_eval.py:80In the codeOpen original file
        # Remove CLAUDECODE env var to allow nesting claude -p inside a        # Claude Code session. The guard is for interactive terminal conflicts;        # programmatic subprocess usage is safe.        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}        process = subprocess.Popen(            cmd,            stdout=subprocess.PIPE,            stderr=subprocess.DEVNULL,            cwd=project_root,            env=env,        )
scripts/run_loop.py:249In the codeOpen original file
    parser.add_argument("--description", default=None, help="Override starting description")    parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")    parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")    parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations")    parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")    parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")    parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)")    parser.add_argument("--model", required=True, help="Model for improvement")    parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
SKILL.md:339In the instructionsOpen original file
Create 20 eval queries — a mix of should-trigger and should-not-trigger. Save as JSON:```json[  {"query": "the user prompt", "should_trigger": true},  {"query": "another prompt", "should_trigger": false}]```
scripts/improve_description.py:26In the codeOpen original file
    """    cmd = ["claude", "-p", "--output-format", "text"]    if model:        cmd.extend(["--model", model])    # Remove CLAUDECODE env var to allow nesting claude -p inside a    # Claude Code session. The guard is for interactive terminal conflicts;    # programmatic subprocess usage is safe. Same pattern as run_eval.py.    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}    result = subprocess.run(        cmd,        input=prompt,        capture_output=True,        text=True,        env=env,        timeout=timeout,

Inside this skill

8 instruction sections

The Skill creates or modifies skills, generates test cases, compares with-skill runs against baselines, and can package the completed skill directory.

View source
SKILL.md:12In the instructionsOpen original file
- Decide what you want the skill to do and roughly how it should do it- Write a draft of the skill- Create a few test prompts and run claude-with-access-to-the-skill on them- Help the user evaluate the results both qualitatively and quantitatively  - While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)  - Use the `eval-viewer/generate_review.py` script to show the user the results for them to look at, and also let them look at the quantitative metrics- Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)- Repeat until you're satisfied- Expand the test set and try again at larger scale
SKILL.md:408In the instructionsOpen original file
### Package and Present (only if `present_files` tool is available)Check whether you have access to the `present_files` tool. If you don't, skip this step. If you do, package the skill and present the .skill file to the user:```bashpython -m scripts.package_skill <path/to/skill-folder>```After packaging, direct the user to the resulting `.skill` file path so they can install it.

The reviewer recursively finds run directories containing `outputs/`, embeds their text and binary outputs into an HTML page, and normally displays it through a local HTTP server.

View source
eval-viewer/generate_review.py:60In the codeOpen original file
def find_runs(workspace: Path) -> list[dict]:    """Recursively find directories that contain an outputs/ subdirectory."""    runs: list[dict] = []    _find_runs_recursive(workspace, workspace, runs)    runs.sort(key=lambda r: (r.get("eval_id", float("inf")), r["id"]))    return runs
eval-viewer/generate_review.py:149In the codeOpen original file
def embed_file(path: Path) -> dict:    """Read a file and return an embedded representation."""    ext = path.suffix.lower()    mime = get_mime_type(path)    if ext in TEXT_EXTENSIONS:        try:            content = path.read_text(errors="replace")        except OSError:            content = "(Error reading file)"        return {            "name": path.name,            "type": "text",            "content": content,        }    elif ext in IMAGE_EXTENSIONS:
eval-viewer/generate_review.py:443In the codeOpen original file
    try:        server = HTTPServer(("127.0.0.1", port), handler)    except OSError:        # Port still in use after kill attempt — find a free one        server = HTTPServer(("127.0.0.1", 0), handler)        port = server.server_address[1]    url = f"http://localhost:{port}"    print(f"\n  Eval Viewer")

The packager walks the selected skill directory and writes every file except a small fixed set of exclusions into a `.skill` ZIP archive.

View source
scripts/package_skill.py:19In the codeOpen original file
# Patterns to exclude when packaging skills.EXCLUDE_DIRS = {"__pycache__", "node_modules"}EXCLUDE_GLOBS = {"*.pyc"}EXCLUDE_FILES = {".DS_Store"}# Directories excluded only at the skill root (not when nested deeper).ROOT_EXCLUDE_DIRS = {"evals"}
scripts/package_skill.py:89In the codeOpen original file
    # Create the .skill file (zip format)    try:        with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf:            # Walk through the skill directory, excluding build artifacts            for file_path in skill_path.rglob('*'):                if not file_path.is_file():                    continue                arcname = file_path.relative_to(skill_path.parent)                if should_exclude(arcname):                    print(f"  Skipped: {arcname}")                    continue                zipf.write(file_path, arcname)                print(f"  Added: {arcname}")
Start here · InstructionsSKILL.md
skill-creator
Lines connect the instruction file to its sections, not an observed execution order. Select a section to read the source. 1 more sections are available in the original file.

File reference map

References: 3
Files making referencesReferenced content
Lines show actual file references, not execution order. Select a node to highlight its connections and inspect the files and source locations. Dashed lines include files that still need locating.
Files and check records18 files

Coverage and gaps

  • Some results did not pass evidence validation or finish processing. This report does not represent a complete check.
Content covered in each file

These are the source ranges included in this check, not a guarantee that every issue has been resolved.

  • SKILL.mdFull text included
  • scripts/__init__.pyFull text included
  • scripts/aggregate_benchmark.pyFull text included
  • scripts/generate_report.pyFull text included
  • scripts/improve_description.pyFull text included
  • scripts/package_skill.pyFull text included
  • scripts/quick_validate.pyFull text included
  • scripts/run_eval.pyFull text included
  • scripts/run_loop.pyFull text included
  • scripts/utils.pyFull text included
  • assets/eval_review.htmlFull text included
  • references/schemas.mdFull text included
  • eval-viewer/generate_review.pyFull text included
  • agents/analyzer.mdFull text included
  • agents/comparator.mdFull text included
  • agents/grader.mdFull text included
  • eval-viewer/viewer.htmlFull text included
  • LICENSE.txtFull text included

This report is for the version above. We read the available code and instructions without running the skill or checking extra packages it installs. This is not a promise of safety: a different version or setup may behave differently.

  • LICENSE.txtLicense
  • SKILL.mdInstructions
  • agents/analyzer.mdSupporting file
  • agents/comparator.mdSupporting file
  • agents/grader.mdSupporting file
  • assets/eval_review.htmlSupporting file
  • eval-viewer/generate_review.pyScript
  • eval-viewer/viewer.htmlSupporting file
  • references/schemas.mdSupporting file
  • scripts/__init__.pyScript
  • scripts/aggregate_benchmark.pyScript
  • scripts/generate_report.pyScript
  • scripts/improve_description.pyScript
  • scripts/package_skill.pyScript
  • scripts/quick_validate.pyScript
  • scripts/run_eval.pyScript
  • scripts/run_loop.pyScript
  • scripts/utils.pyScript

Operations mentioned in code and instructions

Connect to websites
eval-viewer/generate_review.py:449In the codeOpen original file
    url = f"http://localhost:{port}"    print(f"\n  Eval Viewer")
scripts/generate_report.py:39In the codeOpen original file
""" + refresh_tag + """    <title>""" + title_prefix + """Skill Description Optimization</title>    <link rel="preconnect" href="https://fonts.googleapis.com">    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
scripts/generate_report.py:40In the codeOpen original file
    <link rel="preconnect" href="https://fonts.googleapis.com">    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>    <link href="https://fonts.googleapis.com/css2?family=Poppins:wght@500;600&family=Lora:wght@400;500&display=swap" rel="stylesheet">
Run commands
eval-viewer/generate_review.py:22In the codeOpen original file
import signalimport subprocessimport sys
eval-viewer/generate_review.py:291In the codeOpen original file
    try:        result = subprocess.run(            ["lsof", "-ti", f":{port}"],
eval-viewer/generate_review.py:303In the codeOpen original file
            time.sleep(0.5)    except subprocess.TimeoutExpired:        pass
Read files
eval-viewer/generate_review.py:94In the codeOpen original file
            try:                metadata = json.loads(candidate.read_text())                prompt = metadata.get("prompt", "")
eval-viewer/generate_review.py:107In the codeOpen original file
                try:                    text = candidate.read_text()                    match = re.search(r"## Eval Prompt\n\n([\s\S]*?)(?=\n##|$)", text)
eval-viewer/generate_review.py:134In the codeOpen original file
            try:                grading = json.loads(candidate.read_text())            except (json.JSONDecodeError, OSError):
Change files
eval-viewer/generate_review.py:348In the codeOpen original file
            self.end_headers()            self.wfile.write(content)        elif self.path == "/api/feedback":
eval-viewer/generate_review.py:357In the codeOpen original file
            self.end_headers()            self.wfile.write(data)        else:
eval-viewer/generate_review.py:369In the codeOpen original file
                    raise ValueError("Expected JSON object with 'reviews' key")                self.feedback_path.write_text(json.dumps(data, indent=2) + "\n")                resp = b'{"ok":true}'
Read keys or account settings
scripts/improve_description.py:6In the codeOpen original file
by calling `claude -p` as a subprocess (same auth pattern as run_eval.py —uses the session's Claude Code auth, no separate ANTHROPIC_API_KEY needed)."""
scripts/improve_description.py:33In the codeOpen original file
    # programmatic subprocess usage is safe. Same pattern as run_eval.py.    env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
scripts/run_eval.py:83In the codeOpen original file
        # programmatic subprocess usage is safe.        env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
Lines read
5,672
File checksum (to compare versions)
25871608bc6d1204fd6c5e73d619ba73f736ec0ff8ec145536ff375a697c5f39