internal/conformance
github.com/ensera-ai/taisce/internal/conformance · 5 files · 1122 lines · 13 tests · source
Package conformance is the one suite every adapter runs against a live deployment. It decides three things and nothing else: the cases, which are language-neutral and live in conformance/cases.json; the protocol by which a driver, the adapter under test, is told a turn to run and reports what it did; and the form of the memory message an adapter injects, because a suite that cannot parse the message cannot say whether it carries the watermark and the plan.
What is real and what is not
The deployment is real: the runner seeds it through the public contract and reads it back the same way, and the driver reaches it through a proxy the runner stands up. The proxy records every request the adapter makes and, for the cases that need it, refuses one operation, because the rules under test are about what an adapter does when recall or the store is unavailable, and the only honest way to make a real store unavailable is to stand between the adapter and it. The model is the one thing stubbed, inside the driver: the suite is about what the adapter hands the model and what it stores afterwards, and a model that answers a fixed reply is what makes those observable. A mock of the deployment would prove that the mock matches the assertion, and there is none.
Why the memory message has a fixed shape
Every adapter injects memory as one user message marked untrusted. If each chose its own wording, the suite could count messages but not check contents, and the watermark, which is the one number that lets an agent know how far behind its memory is, would be present in some languages and not others. So the content is a prefix line and a JSON document, readable by a model and parseable by this suite, and every adapter renders the same one.
Where it sits
Imports: none in this repository
Imported by: cmd/taisce
Files
| File | Lines | Declarations | What it is for |
|---|---|---|---|
| protocol.go | 239 | 19 | Carries the package documentation. |
| proxy.go | 89 | 7 | Recorded is one request the adapter made through the proxy. |
| reference.go | 266 | 8 | Reference is the smallest adapter that keeps every rule, in this process, over the public contract. |
| runner.go | 410 | 13 | Driver is the adapter under test, however it is reached: in this process for the reference, over a subprocess for an adapter in another language. |
| subprocess.go | 118 | 5 | Subprocess reaches a driver written in any language: the runner starts the command once and exchanges one JSON line per turn, an Instruction in and a Report out, on its stdin and stdout. |
13 tests hold this package's behaviour; the list is named for what each one proves.