claudiomaradonna/agent-foundry
claudiomaradonna/agent-foundry
Template
1
0
Fork 0
Code Issues 5 Pull requests Activity

Keep evals/ in the source when upgrading and initializing #146

New issue
Open
opened 2026-10-19 17:22:31 +02:00 by motoko · 0 comments
motoko commented 2026-10-19 17:22:31 +02:00
Collaborator
Copy link

Seen at d7036ed199.

Location

  • UPGRADING.md step 3, lines 20–21: "LICENSE, README.md,
    UPGRADING.md, and the root docs/, scripts/, and tests/ stay in
    the source; for every other path: added: copy it in".
  • docs/INSTALL.md lines 19–21, the New repository case, which deletes
    the same list after a forge's template copy.

Evidence

evals/ is the source's own maintenance suite, but neither list names
it. The other documents treat it as staying in the source:

  • README Adoption step 2 (lines 88–90) copies only AGENTS.md,
    .agents/, .claude/, .codex/, .opencode/, .gitignore, and
    .prettierrc.
  • evals/install-policy.sh line 24 archives that same set.

Scenario

A repository upgrades across e8c5e3f, which added evals/. Its agent follows step
3 to the letter and copies the eval runner, cases, and fixture into the
repository. After a template copy, a new repository keeps them in the
same way. An agent that notices the gap has to decide on its own.

Smallest fix

Add evals/ to the stay-in-the-source list in UPGRADING.md step 3 and
in docs/INSTALL.md's New repository case.

Seen at d7036ed199ca23491159479231c377c3e17de02d. ## Location - `UPGRADING.md` step 3, lines 20–21: "`LICENSE`, `README.md`, `UPGRADING.md`, and the root `docs/`, `scripts/`, and `tests/` stay in the source; for every other path: added: copy it in". - `docs/INSTALL.md` lines 19–21, the New repository case, which deletes the same list after a forge's template copy. ## Evidence `evals/` is the source's own maintenance suite, but neither list names it. The other documents treat it as staying in the source: - README Adoption step 2 (lines 88–90) copies only `AGENTS.md`, `.agents/`, `.claude/`, `.codex/`, `.opencode/`, `.gitignore`, and `.prettierrc`. - `evals/install-policy.sh` line 24 archives that same set. ## Scenario A repository upgrades across `e8c5e3f`, which added `evals/`. Its agent follows step 3 to the letter and copies the eval runner, cases, and fixture into the repository. After a template copy, a new repository keeps them in the same way. An agent that notices the gap has to decide on its own. ## Smallest fix Add `evals/` to the stay-in-the-source list in `UPGRADING.md` step 3 and in `docs/INSTALL.md`'s New repository case.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
perf/codex-reviewer-context
refactor/checklist-skills-2
docs/deliver-gaps
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
claudiomaradonna/agent-foundry#146
No description provided.