Skip to content
skillsafebeta
← Report library
Purpose / Browser automation

Clerk Testing Skill Security Audit

What the author says it does (original text)

E2E testing for Clerk apps. Use with Playwright or Cypress for auth flow

Independent security check

Do not install or run it yet

Files checked
1
Risks found
5
Could it run dangerous commands?Looks for programs run straight after downloading, remote control of your computer, and hidden commands.Risks found: 1
High risk

Executes an unpinned remote Clerk CLI package

Source references: 1
What we found

The workflow directs execution of `npx clerk@latest init`. `npx` can download and immediately run whichever package is tagged latest, without this Skill pinning or verifying a specific version; the executed code can change over time.

Why this matters

If the upstream package, publishing account, or dependency chain is compromised, code could run with the agent's permissions and access project files, environment variables, or available credentials. Ordinary upgrades could also introduce unreviewed behavior changes.

`npx clerk@latest init` can fetch and execute the CLI currently referenced by the latest tag, with no pinned version or verification procedure in the source. Its stated purpose—creating a development instance for testing—is relevant to the Skill, but the code a user runs can change as package releases change. Users can restrict this to a reviewed, fixed version.

SKILL.md:35In the instructionsOpen original file
3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
Could it expose your files or keys?Looks for uploads of files containing passwords or keys, and keys written directly in the code.Risks found: 1
Medium risk

Persisted authentication state can leave a reusable session

Source references: 3
What we found

The Skill recommends saving authentication state with Playwright `storageState` for reuse, but does not require protecting, ignoring, or cleaning up the generated state file. Such a file commonly represents an authenticated browser session.

Why this matters

If the state file enters source control, CI artifacts, or storage readable by others, a holder may impersonate the test user while the session remains valid. Accidentally targeting production could extend the impact to a real account.

The source explicitly recommends persisting and reusing authentication state with `storageState`, but gives no storage location, access-control, version-control exclusion, or cleanup requirement. If written to a file, that state may contain reusable browser authentication material that could be committed accidentally or read locally. Users can require a restricted, ignored temporary path and post-test cleanup.

SKILL.md:25In the instructionsOpen original file
Test auth = isolated session state. Each test needs fresh auth context.- `clerkSetup()` initializes test environment- `setupClerkTestingToken()` bypasses bot detection- `storageState` persists auth between tests for speed
Show 2 other places
SKILL.md:39In the instructionsOpen original file
- Use `setupClerkTestingToken()` before navigating to auth pages- Use test API keys: `pk_test_xxx`, `sk_test_xxx`- Save auth state with `storageState` for faster tests- Use `page.waitForSelector('[data-clerk-component]')` for Clerk UI
SKILL.md:41In the instructionsOpen original file
- Use test API keys: `pk_test_xxx`, `sk_test_xxx`- Save auth state with `storageState` for faster tests- Use `page.waitForSelector('[data-clerk-component]')` for Clerk UI
Could it delete files or keep running?Looks for broad file deletion, disk overwrites, and programs set to start automatically.Risks found: 1
Medium risk

CLI creates a Clerk application and writes secret keys into an environment file

Source references: 2
What we found

The instruction goes beyond local test configuration: it says the CLI creates a Clerk application and writes development-instance keys to an environment file. It does not identify the target file, handling of existing content, or a confirmation step.

Why this matters

This changes an external Clerk resource and leaves secrets in the project directory. If the file is tracked, logged, packaged, or readable by others, test secrets could be exposed; automatic file selection could also overwrite or mix with existing configuration.

The source says the CLI creates a Clerk application and writes development keys to “the env file,” without identifying that file, explaining treatment of existing values, or requiring confirmation. It is conditional on having no keys and requires test keys, which reduces production-credential risk, but it still changes local files and creates an external application. Users can ask for the exact target, merge behavior, and a pre-creation confirmation.

SKILL.md:10In the instructionsOpen original file
  version: 1.2.1compatibility: Requires a publishable key and either CLERK_SECRET_KEY or CLERK_TESTING_TOKEN. Keys can be auto-generated by `clerk init` (temporary development keys, no Clerk account required), or pulled from the Clerk Dashboard.---
Show 1 other places
SKILL.md:35In the instructionsOpen original file
3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
Could it bypass safety checks?Looks for skipped website security checks, excessive file access, or actions that skip your approval.Risks found: 1
Low risk

The install command does not pin a dependency version

Source references: 1
What we found

The installation command does not specify dependency versions. The same command may download different code later, so what you install can differ from what was checked.

Why this matters

A later install may download different code even though the command and this report have not changed.

The workflow explicitly uses `clerk@latest` rather than a fixed version. At execution time, it resolves to whatever release is then tagged latest, so the downloaded CLI code can change. The command is conditional on the project lacking keys and is relevant to test setup, but version drift remains a real risk. A user can ask the author to pin and verify a specific CLI version.

SKILL.md:35In the instructionsOpen original file
3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
Could it mislead the AI or hide text?Checks the skill instructions for requests to ignore you, influence the report, or hide text in invisible characters.Risks found: 1
Medium risk

Directly follows mutable instructions fetched at runtime

Source references: 3
What we found

The Skill allows WebFetch and explicitly says to fetch Clerk webpages and “Follow official setup instructions.” That content is absent from the supplied source and can change after publication, so its commands, permission requests, and data handling cannot be confirmed by this static audit.

Why this matters

If a page, redirect, or content supply chain changes, the agent could follow newly added commands, credential requests, or file modifications while the user assumes those actions were reviewed with the Skill.

The Skill permits WebFetch and directs the agent to fetch a listed page and follow its setup instructions. Those pages are absent from the supplied source and can change later, so this static audit cannot verify the commands, permissions, or data handling they may request at runtime. The links are presented as Clerk documentation and fit the testing purpose, but users can require fetched commands, versions, permissions, and file changes to be shown for approval before execution.

SKILL.md:5In the instructionsOpen original file
  tests.allowed-tools: WebFetchlicense: MIT
Show 2 other places
SKILL.md:17In the instructionsOpen original file
| Framework | Documentation ||-----------|---------------|| Overview | https://clerk.com/docs/guides/development/testing/overview || Playwright | https://clerk.com/docs/guides/development/testing/playwright/overview || Cypress | https://clerk.com/docs/guides/development/testing/cypress/overview |
SKILL.md:32In the instructionsOpen original file
1. Identify test framework (Playwright or Cypress)2. WebFetch the appropriate URL from decision tree above3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
Could it change links or payment recipients without asking?Looks for forced referral or payment changes combined with instructions to hide the change.No risks found

Inside this skill

7 instruction sections

This Skill targets end-to-end authentication testing for Clerk applications with Playwright or Cypress and calls for isolated session state for each test.

View source
SKILL.md:2In the instructionsOpen original file
---name: clerk-testingdescription: E2E testing for Clerk apps. Use with Playwright or Cypress for auth flow  tests.allowed-tools: WebFetch
SKILL.md:23In the instructionsOpen original file
## Mental ModelTest auth = isolated session state. Each test needs fresh auth context.- `clerkSetup()` initializes test environment- `setupClerkTestingToken()` bypasses bot detection- `storageState` persists auth between tests for speed

It directs the agent to fetch current instructions from predefined Clerk documentation URLs and then follow those setup steps, so the complete behavior is not contained in this file.

View source
SKILL.md:17In the instructionsOpen original file
| Framework | Documentation ||-----------|---------------|| Overview | https://clerk.com/docs/guides/development/testing/overview || Playwright | https://clerk.com/docs/guides/development/testing/playwright/overview || Cypress | https://clerk.com/docs/guides/development/testing/cypress/overview |
SKILL.md:32In the instructionsOpen original file
1. Identify test framework (Playwright or Cypress)2. WebFetch the appropriate URL from decision tree above3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.

The Skill explicitly limits testing to development-instance keys and identifies using production keys in tests as a security risk.

View source
SKILL.md:35In the instructionsOpen original file
3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
SKILL.md:46In the instructionsOpen original file
| Pattern | Problem | Fix ||---------|---------|-----|| Production keys in tests | Security risk | Use `pk_test_*` keys || No `setupClerkTestingToken()` | Auth fails | Call before navigation || UI-based sign-in every test | Slow tests | Use `storageState` |
Start here · InstructionsSKILL.md
clerk-testing
Lines connect the instruction file to its sections, not an observed execution order. Select a section to read the source.
Files and check records1 files

Coverage and gaps

Content covered in each file

These are the source ranges included in this check, not a guarantee that every issue has been resolved.

  • SKILL.mdFull text included

This report is for the version above. We read the available code and instructions without running the skill or checking extra packages it installs. This is not a promise of safety: a different version or setup may behave differently.

  • SKILL.mdInstructions

Operations mentioned in code and instructions

Connect to websites
SKILL.md:19In the instructionsOpen original file
|-----------|---------------|| Overview | https://clerk.com/docs/guides/development/testing/overview || Playwright | https://clerk.com/docs/guides/development/testing/playwright/overview |
SKILL.md:20In the instructionsOpen original file
| Overview | https://clerk.com/docs/guides/development/testing/overview || Playwright | https://clerk.com/docs/guides/development/testing/playwright/overview || Cypress | https://clerk.com/docs/guides/development/testing/cypress/overview |
SKILL.md:21In the instructionsOpen original file
| Playwright | https://clerk.com/docs/guides/development/testing/playwright/overview || Cypress | https://clerk.com/docs/guides/development/testing/cypress/overview |
Install extra software packages
SKILL.md:35In the instructionsOpen original file
3. Follow official setup instructions4. Use development-instance keys only (`pk_test_*` / `sk_test_*`). If the project has no keys yet, `npx clerk@latest init` (the Clerk CLI; see `clerk-setup`) creates a Clerk application and writes development-instance keys to the env file. No Clerk account or Dashboard visit is needed.
Lines read
62
File checksum (to compare versions)
7145f9bf9b6a9afbf2659a872bab7b1cc2184eb35805c05fe69247c0016734e1