Coach a harness by running your task

gptr_coach() improves the coaching of a harness (the note, skills, review and rules that gptr gives peter; see gptr_tailor()) by running your task. Your starting harness does the task once; two coaches read that run, one about how peter works and one about what an expert in the subject knows, and each proposes new coaching; each proposal does the task once; a third coach, the composer, compares the runs and composes the coaching to keep; and that coaching does the task once more. Code then decides whether to keep it: the final run must end normally, deliver at least what the starting run delivered, take no riskier kind of action, meet every require rule and pass your check. Every session runs unattended in its own R process, with copies of your objects and an empty folder of its own, so a step that needs your approval ends that run; most tasks need mode = "auto".

Usage

gptr_coach(
  ...,
  harness = peter,
  check = NULL,
  coach = NULL,
  k = 1L,
  strategy = "parallel",
  focus = c("procedure", "domain"),
  surface = c("note", "skills", "review", "rules")
)

Arguments

...: What you would give the harness: the prompt, your objects, and any peter() arguments such as mode, model, budget or tools. They apply to every run of this call and are not kept in the result; budget, else the starting harness's, also caps each coach session and the judge. Each run is one unattended session, so parallel, choices, levels, threshold, min_confidence, uncertain, background, .run, .stdin and envir are refused.
harness: The harness to start from: peter or a harness.
check: NULL or your own test of a good result: a function of r, a list with text, status, value and objects of the final run, that returns TRUE or FALSE. It runs in this R session only, and no coach sees it.
coach: The model of the coaches, a name or a provider spec; NULL uses the runs' model.
k: How much coaching, 1 to 5. With "parallel": k starting runs, k coaches of each focus, one probe of each, and k composers that each see every probe; when more than one of their final runs passes, a blind judge, which sees only the task and what each final delivered, picks one. With "sequential": k starting runs, then k rounds, each building on the composition of the round before and seeing every earlier round.
strategy: "parallel" or "sequential" (see k). It is not called mode, so that mode = "auto" still reaches peter.
focus: Every focus to run, one proposing coach each: "procedure" (how peter works), "domain" (what an expert knows), or your own words.
surface: The parts of the coaching that may change, among "note", "skills", "review" and "rules". Only the procedure coach writes rules.

Value

A harness: your starting harness with the new coaching when it was kept, otherwise your starting harness. Either way it has a new id and a record (h$record: each run's outcome, turns, tokens, cost, highest risk and unmet requirements, and the decision with its reason; no data from your runs). h$report holds the full report, with every answer, value, transcript and coach reply, in this R session only. A call replayed from its document block returns the recorded coaching with an empty record; the block's ## Decision: line holds the decision.

Details

Before it starts, gptr_coach() asks the usual question about sending your data to each model, while you are there, and says how many sessions it will run: k * (2 + focuses) runs of your task and k * (1 + focuses) coach sessions, plus up to one judge session when k > 1; with the defaults, 4 runs and 3 coach sessions. On a Claude Code or Codex plan they use your plan's limits, at most two at once; when a plan limit is reached, it starts nothing new and returns your starting harness. A call recorded in a document (h = gptr_coach(...) at the top level of a script, R Markdown, Quarto or Jupyter document) replays from its block with no model, no key and no process.

Examples

# A run that is wrong unless its note says how to treat missing values, and a coach that
# writes that note (both scripted; this starts 7 R processes and takes several seconds)
solver = gptr_fake_provider(function(request) {
  if (length(request$last_results)) return("Done.")
  fix = grepl("na.rm", paste(unlist(request$system), collapse = " "), fixed = TRUE)
  list(tool = "r", input = list(code = if (fix) "gptr_return(mean(x, na.rm = TRUE))" else
    "gptr_return(mean(x))"))
}, name = "solver")
coach = gptr_fake_provider(list(paste0(
  '{"note": "Drop missing values with na.rm = TRUE before you average.", ',
  '"notes": "The starting run returned NA."}')), name = "coach")
# \donttest{
x = c(1, NA, 3)
mean_peter = gptr_coach("Average x.", x, model = solver, mode = "auto", coach = coach,
                        check = function(r) isTRUE(r$value == 2))
#> gptr_coach() starts 4 runs of your task and 3 coach sessions, each in its own R process.
#> gptr_coach() kept this coaching:
#> Note:
#> Drop missing values with na.rm = TRUE before you average.
#> Kept is not proof that the new harness is better: nobody has measured it on new data.
mean_peter$record$decision
#> [1] "kept"
# }