Skip to main content

internal/infra/inference/embedder.go

internal/infra/inference · 229 lines · 7 declarations · source

Declarations​

const MaxEmbeddingInputs, MaxEmbeddingInputBytes, MaxEmbeddingBatchBytes and 2 more​

const (
	MaxEmbeddingInputs     = 64
	MaxEmbeddingInputBytes = 64 << 10
	MaxEmbeddingBatchBytes = 1 << 20
	MaxEmbeddingDimensions = 16000
	MaxEmbeddingReplyBytes = 64 << 20
)

Defensive ceilings bound serialization, response allocation and numeric work. They are not a claim about provider throughput or the dimension/index contract of a persisted corpus.

source

type Embedder​

type Embedder struct {
	config Config
	client *http.Client
}

Embedder turns text into a vector over the OpenAI-compatible embeddings interface.

It embeds and nothing more. What text stands for an entity, how much of it, and what is done with the distance between two vectors are decisions elsewhere — this is the hop to a model, and the reason it is a separate type from the extractor's is that the two are different jobs: extraction is a generation that runs once per message, embedding is a forward pass that runs once per entity and once per question.

source

func NewEmbedder​

func NewEmbedder(config Config) *Embedder

NewEmbedder builds an embedder against an OpenAI-compatible endpoint.

source

method Embedder.Embed​

func (e *Embedder) Embed(ctx context.Context, inputs []string) ([][]float32, error)

Embed returns one vector per input, in the order the inputs were given.

Why a batch​

Embedding is dominated by the round trip on short inputs, and the callers here have batches by nature: every entity in a scope, every community's report. One call per item would make a rebuild after an erasure a few thousand round trips.

Why the order is a promise​

The interface returns an index with each vector and nothing requires them to arrive in order. A caller matching by position against a reply that came back sorted differently would attach every entity's vector to a different entity — silently, and in a way that looks like poor retrieval rather than like a defect. So the index is honoured here and the promise is made once.

source

func Similarity​

func Similarity(a, b []float32) (float64, error)

Similarity is the cosine of the angle between two vectors, in [-1, 1].

Here rather than in a caller because the failure it prevents is arithmetic: a dot product used without normalising rewards long vectors, and whether a model returns normalised vectors is a property of the model rather than of the interface. Normalising costs one pass and removes the question.

Two vectors of different lengths are not comparable, and returning zero would read as "unrelated" rather than as "this comparison is meaningless".

source

type embeddingRequest​

type embeddingRequest struct {
	Model string   `json:"model"`
	Input []string `json:"input"`
}

source

type embeddingResponse​

type embeddingResponse struct {
	Data []struct {
		Index     int       `json:"index"`
		Embedding []float32 `json:"embedding"`
	} `json:"data"`
}

source