
Refactor codebases using Design by Typed Holes methodology - iterative, test-driven refactoring with formal hole resolution, constraint prop
- Open source
- 191 GitHub stars

Refactor codebases using Design by Typed Holes methodology - iterative, test-driven refactoring with formal hole resolution, constraint prop

Use when the user wants to evolve an ML model/program through population-based search rather than a single sequential refine loop — a genera

Use when the user has a known, already-observed anomaly in their data — a metric spike or drop, an outlier, an unexpected number — and wants

Use when the user has concrete failing cases in code or a guardrail/classifier/filter/prompt/API they own — a red-team failure catalogue OR

Use when the user has a results draft or a set of data-backed claims and wants each one adversarially verified against the underlying datase

Use when the user wants an iterative, self-checking exploratory analysis of a dataset — surfacing findings that are each verified by re-runn

Use when the user wants two approaches raced head-to-head on a single shared metric — e.g

Use when the user wants an autonomous ML research loop that explores the space broadly rather than hill-climbing one approach

Use when the user wants to generate and literature-vet a pool of novel, testable research hypotheses for a question or domain

Use when the user wants the LLM to do its own ML research: a fully-autonomous loop that hacks the training code, runs it, and keeps changes

Use when a loop needs scholarly literature — paper discovery, novelty checks, full-text snippet search, citation-graph traversal, single-pap

Use when the user wants a structured, saturating literature survey on a question — not a one-shot summary, but an evidence/contradiction mat

Use when the user wants an autonomous ML research loop that does more than blindly try changes

Use when the user wants to iteratively improve an artifact under a hard correctness bound while minimizing a measured cost — refactoring a c

Use when the user has a coding or engineering prompt and wants it refined into a detailed, executable plan before any code is written — the

Use when the user is planning a two-arm comparison (an A/B test, a simple RCT, a behavioral study, or a two-model/two-config evaluation) and

Use when the user has a prompt that feeds a system they can already score, and wants that prompt automatically improved to raise the score a

Use when the user wants to automatically harden a guardrail, classifier, content filter, prompt, or API they own by running attack and defen

Use when the user wants to adversarially stress-test a guardrail, classifier, prompt, or API they own or are authorized to test, to surface

Use when the user has a research proposal (problem + proposed methodology + planned experiments) and wants it iteratively strengthened until

Use when the user has a vague topic or area of interest and wants it sharpened into a few strong, novel, feasible research questions

Use when the user has scientific data (or a prompt alluding to scientific data) and wants a publication-quality figure made from it

Use when the user has a scientific draft (with its dataset, figures, and optional analysis code) and wants it iteratively revised until it c

Use when the user has a coding prompt — a feature, bug fix, or refactor — and wants it implemented end to end by a self-checking software lo

Use when the user has a messy tabular data dump (CSV/TSV/parquet/Excel/JSON) and wants it iteratively cleaned to an inferred data contract —

Use when the user wants an autonomous ML research loop that pressure-tests competing ideas before spending compute — several research subage


Validates gate criteria before PRD lifecycle advancement by delegating to the readiness scoring pipeline (scripts/readiness.py)

Extracts durable insights from temp/ files to SoT during EPIC Phase E

Validates and registers new SoT IDs with cross-reference integrity