Repository-supplied test commands may execute arbitrary code without a required review or sandbox
Source references: 5The Skill explicitly prefers checked-in wrappers or repository scripts and requires the full suite before completion, but does not require inspecting the command contents, obtaining user confirmation, or using an isolated environment. Test scripts and build wrappers can execute arbitrary programs, and the document also permits large tests to use external services.
In an untrusted repository or dependency set, tests could read accessible files or environment variables, alter the workspace, initiate downloads, or contact external services. If cloud or third-party credentials are available, integration tests could also change real account data or incur charges.
Running tests is normal and relevant to this Skill’s stated purpose. However, its live instructions prefer repository-provided wrappers or scripts and require a full-suite run before completion. If the repository is untrusted or its test configuration is compromised, those commands can run programs with the agent’s current permissions; “large” tests may also contact external services. No prior script review, isolation, or confirmation of external side effects is required. This supports a conditional risk to files, credentials, and network access, but does not show that malicious code exists or ran. Users can ask for script review, sandboxing, a credential-free environment, and network-deny defaults.
- **Language and build system** — `package.json`, `pom.xml`/`build.gradle`, `pyproject.toml`, `go.mod`, `Cargo.toml`, `Gemfile`, a `Makefile`- **Checked-in wrappers** — prefer `./gradlew`, `./mvnw`, `make test`, or a repo script over globally installed tools- **Test framework and configuration** — and how it runs a single focused test vs the full suite- **Existing conventions** — where tests live, how files are named, what patterns neighboring tests follow- **Documented commands** — README, CONTRIBUTING, and CI workflows show the commands that actually gate mergesRun the repository's focused-test command during the loop and its full-suite command before completion. Never assume a default like `npm test` — a Gradle, Cargo, or pytest project has its own equivalent.Show 4 other places
| Size | Constraints | Speed | Example ||------|------------|-------|---------|| **Small** | Single process, no I/O, no network, no database | Milliseconds | Pure function tests, data transforms || **Medium** | Multi-process OK, localhost only, no external services | Seconds | API tests with test DB, component tests || **Large** | Multi-machine OK, external services allowed | Minutes | E2E tests, performance benchmarks, staging integration |After completing any implementation:- [ ] Every new behavior has a corresponding test- [ ] The full suite passes, run with the repository's own test command (`npm test`, `./gradlew test`, `pytest`, `go test ./...`, ...)- [ ] Bug fixes include a reproduction test that failed before the fixThe TDD cycle is universal; the commands are not. Before writing the first test, discover how *this* repository tests, and use its commands for every RED, GREEN, and verification step:- **Language and build system** — `package.json`, `pom.xml`/`build.gradle`, `pyproject.toml`, `go.mod`, `Cargo.toml`, `Gemfile`, a `Makefile`- **Checked-in wrappers** — prefer `./gradlew`, `./mvnw`, `make test`, or a repo script over globally installed tools- **Test framework and configuration** — and how it runs a single focused test vs the full suite- **Existing conventions** — where tests live, how files are named, what patterns neighboring tests followRun the repository's focused-test command during the loop and its full-suite command before completion. Never assume a default like `npm test` — a Gradle, Cargo, or pytest project has its own equivalent.