Project Knowledge

View as Markdown

Shiplight can preserve useful facts learned while working with you so you do not need to provide the same information again in later Shiplight sessions. It stores those facts as ordinary Markdown files under knowledge/ in the root of your Shiplight test project.

For example, you might tell the agent that the staging environment delays confirmation emails, that viewer accounts must never be used to create records, or that a particular cleanup endpoint is the reliable way to reset test data. Those are durable operational facts that belong in project knowledge.

Project knowledge is not a transcript of your conversation. It is a small, curated set of current facts that Shiplight can use in later sessions.

How /shiplight uses knowledge/

When you start a /shiplight session, the skill tells your agent to read the knowledge files relevant to the work it is about to do. It looks for notes about:

• The application area or feature being tested
• The target URL or deployment environment
• Authentication and account roles
• Test-data setup and cleanup
• Tooling constraints and known failure modes

The agent uses this knowledge as context when it plans, creates, verifies, or repairs tests. For example, a note that staging emails can take 30 seconds prevents Shiplight from treating normal delivery latency as a failure in a later session.

Knowledge does not override your current instruction or the project's sources of truth. When information conflicts, Shiplight follows this order:

1. Your current instruction
2. Feature and test specifications
3. Existing test intent and assertions
4. Current application behavior
5. specs/context.md and knowledge/

If stored knowledge conflicts with intended product behavior, the agent reports the mismatch instead of silently changing a test to match an old note.

How /shiplight updates knowledge/

Before an interactive /shiplight session ends, the skill tells the agent to reflect on what it learned. The agent writes a note only when the information is durable, useful in later testing sessions, and not easily recovered from code, specs, or maintained guides.

The agent may record:

• Application quirks that consistently affect testing
• Reliable setup, reset, and cleanup procedures
• Known failure modes and how to distinguish them
• Tooling or environment constraints
• Stable account roles and non-secret data constraints
• Corrections to assumptions recorded during an earlier session
• Testing preferences you want Shiplight to follow in later sessions

The agent does not simply append a new file after every conversation. It first checks whether the fact already exists and then:

• Updates the existing note when it covers the same topic
• Merges duplicate notes
• Corrects incomplete or inaccurate guidance
• Removes obsolete guidance that could mislead Shiplight in a later session
• Creates a new Markdown file only when the information does not belong in an existing note

The agent also decides whether knowledge/ is the correct destination:

Location	Use it for
knowledge/	Operational facts discovered while working, such as quirks, reliable procedures, and tooling constraints
specs/context.md	Project-wide context, such as the app and risk profile, target deployments, roles, data strategy, and broad scope decisions
specs/tests/*.md	Feature intent, expected behavior, journeys, assertions, and coverage decisions for specific E2E tests

The knowledge/ directory is created only when there is something useful to store. Shiplight does not create an empty directory during project setup.

How you can add knowledge

You can add project knowledge yourself or ask Shiplight to record it.

Tell Shiplight what to remember

During an interactive /shiplight session, state the durable fact and ask the agent to remember it for later Shiplight sessions. For example:

/shiplight create-yaml-tests for sign-up

Remember for future tests: confirmation emails in staging can take up to
30 seconds, and sign-up tests must delete the account they create.

The agent should place each fact in knowledge/, specs/context.md, or the relevant test spec according to its scope. You can ask it to show you the resulting change before the session ends.

Add or edit a Markdown file

Create a descriptive Markdown file inside knowledge/ at the test project root. There is no required filename or schema; group related facts so agents can find and maintain them easily.

your-test-project/
├── knowledge/
│   └── staging-email.md
├── specs/
└── tests/

Write each note so it stands on its own. State the current fact directly, include enough context for an agent to act on it, and avoid references such as “this task” or “what we discussed.”

For example, knowledge/staging-email.md could contain:

# Staging email delivery

Confirmation emails in staging normally arrive within 30 seconds. Poll the
test inbox for up to 45 seconds before treating delivery as failed.

Use a unique `+e2e-<timestamp>` address for each sign-up. Delete the created
account through the test-data cleanup endpoint after the test.

Do not duplicate a fact in project knowledge when it is already clear in a spec, test, code comment, or maintained guide. When the expected product behavior changes, update the relevant spec rather than treating knowledge/ as a substitute for the source of truth.

Do not add secrets

Never put passwords, API tokens, session cookies, private keys, one-time codes, or other secret values in knowledge/.

It is safe to record the names and purpose of required environment variables, account roles, and authentication patterns. Keep the values themselves in your project's secret-management system or local .env file. For example, record that administrator tests require ADMIN_EMAIL and ADMIN_PASSWORD; do not record their values.

Build project knowledge together

Treat knowledge/ as a shared knowledge base that every team member can contribute to and maintain. Because the notes are ordinary project files, your team can review changes with Git and commit updates so everyone—and later Shiplight sessions—uses the same current context. Maintain the notes like code: keep them focused, correct inaccurate information, and remove guidance that no longer describes the project.