internal/infra/inference/extractor.go
internal/infra/inference · 317 lines · 13 declarations · source
Declarations
type Model
type Model struct {
config Config
client *http.Client
}
Model proposes claims by asking a chat model.
It proposes and nothing more. It does not decide whether a relation is admitted, does not locate a quote, and does not build a byte span — all of that is in the extractor, deliberately, because those are the decisions with a correctness argument and they must not vary with the vendor.
func NewModel
func NewModel(config Config) *Model
NewModel builds a proposer against an OpenAI-compatible endpoint.
method Model.Propose
func (m *Model) Propose(ctx context.Context, message domain.Message, vocabulary domain.Ontology) ([]extract.Proposal, error)
Propose asks the model what one message asserts.
No retry here
A rate limit and a transient failure are both real and both are the caller's to handle: the caller is the one that knows whether this message is being formed in a batch that can be paused or on a path a customer is waiting on. A retry buried here would make the two indistinguishable and would hold a connection open through a backoff nobody chose.
func parseProposals
func parseProposals(content string) ([]extract.Proposal, error)
parseProposals reads the model's JSON.
A malformed body is an ERROR rather than an empty result. An empty result means "this message asserts nothing", which is the ordinary case for most messages — so returning it for a broken response would make a misconfigured model indistinguishable from a quiet conversation, and memory would simply stop forming with nothing reporting a problem.
func userPrompt
func userPrompt(message domain.Message) string
userPrompt carries the message and its speaker, with the message fenced.
The speaker is included because who is talking changes what a sentence asserts: "you moved to Dublin last month" from an assistant is a claim about the assistant's belief, not the user's history. What is done with the speaker afterwards is the role policy's business, on the road to the fact table — this is only so the model is not reading the sentence blind.
Why the message is fenced with a value the message could not contain
Content extracted from a web page, a document or a tool result can carry wording aimed at the model rather than at the reader — an instruction to record a particular fact, or to disregard what came before it. A claim produced that way passes every check this system makes: it uses an admitted relation, and its quote really is in the message, so the span verifies exactly.
A fixed delimiter is one an attacker writes into their own content to close the block early and continue outside it. The identifier here is random per call and chosen AFTER the content is in hand, so it cannot appear in text that was written before it existed — and it is regenerated in the impossible case that it does.
This is a weak defence and it is worth having anyway. Prompt-level measures do not survive a determined attacker, so what actually bounds the damage is what a non-principal message is permitted to assert at all. That is a decision rather than a patch. This removes the trivial version of the attack while it is being taken.
func newFence
func newFence(content string) string
newFence returns a random identifier that does not occur in the content it will delimit.
type claimEnvelope
type claimEnvelope struct {
Claims *[]struct {
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Object string `json:"object"`
Statement string `json:"statement"`
Confidence float32 `json:"confidence"`
Quote string `json:"quote"`
SubjectType string `json:"subject_type"`
ObjectType string `json:"object_type"`
Polarity string `json:"polarity"`
Tense string `json:"tense"`
} `json:"claims"`
}
claimEnvelope is the reply contract.
Claims is a POINTER so that an object WITHOUT the key is distinguishable from one whose list is empty. That difference is what lets the locator below tell our document from some other object that happened to parse, and getting it wrong would turn a misread reply into "this message asserts nothing" — silently, on the write path.
func decodeEnvelope
func decodeEnvelope(content string) (claimEnvelope, error)
decodeEnvelope finds the reply inside whatever the model actually sent.
Why this is not simply json.Unmarshal over the body
Measured, against a local model in JSON mode, on the fixture `I moved to Dublin last month.`:
claims":{"claims": [{"subject": "I", "predicate": "lives_in", ...}]}
A fragment of the prompt's own shape, emitted before a perfectly formed document. Unmarshalling the whole body failed on the first byte, so a correct extraction was discarded, the attempt was counted, and the turn would eventually have been parked. Models do this. A strict parse over the whole body makes our tolerance for it zero and makes the failure indistinguishable from a model that cannot extract at all.
Why being tolerant here loosens no guarantee
This locates the document. It does not relax what happens to it: the JSON is still decoded strictly, every relation is still checked against the closed vocabulary, and every quote is still located in the message by byte span. A claim arriving behind a prefix faces exactly the checks a claim arriving cleanly faces.
Why it insists the claims key is present
Decoding into a struct ignores unknown fields, so a single claim emitted without its envelope — `{"subject": "I", "predicate": "lives_in", ...}` — would decode as an envelope holding no claims and be read as an empty result. Requiring the key means an object that is not the envelope is skipped and the search continues, and a body with no envelope anywhere is an ERROR rather than a quiet nothing.
func truncateForError
func truncateForError(content string) string
truncateForError bounds what a model's reply contributes to an error message.
The reply can contain the message that was sent to it, which is a customer's words — and this error is stored on the observation, where erasure reaches it. Bounded so that a remote system cannot decide how much of that column it uses.
type chatRequest
type chatRequest struct {
Model string `json:"model"`
Temperature float32 `json:"temperature"`
Messages []chatMessage `json:"messages"`
ResponseFormat *responseFormat `json:"response_format,omitempty"`
}
type chatMessage
type chatMessage struct {
Role string `json:"role"`
Content string `json:"content"`
}
type responseFormat
type responseFormat struct {
Type string `json:"type"`
}
type chatResponse
type chatResponse struct {
Choices []struct {
Message chatMessage `json:"message"`
} `json:"choices"`
}