Skip to main content

Tests: internal/ingest

internal/ingest · 6 tests

Each test is named for the property it holds, and runs against a real deployment: there is no mock of the database and no arm that skips when it is absent. The sentence is the test's name read back; the name is what go test -run takes.

ingest_test.go​

  • Segments are the document and quote positions round trip — TestSegmentsAreTheDocumentAndQuotePositionsRoundTrip. The segments are the document: their concatenation is the text, none is empty, none exceeds the ceiling, and every cut is on a code-point boundary.
  • The cut prefers paragraph then sentence then space then code point — TestTheCutPrefersParagraphThenSentenceThenSpaceThenCodePoint. The cut prefers a paragraph, then a sentence, then whitespace, then the last whole code point; none of them may fall inside a multibyte space or code point.
  • Unsegmentable text and bad ceilings are refused — TestUnsegmentableTextAndBadCeilingsAreRefused. A segment that would carry no words is refused with the whole document, and so is a ceiling outside the range a message can hold.
  • A document is refused whole before anything is sent — TestADocumentIsRefusedWholeBeforeAnythingIsSent. Every refusal happens before anything could be sent: the size ceiling from the file's size, not its content; text that is not UTF-8 or carries NUL; an empty file; an extension this contract does not read; JSON that is malformed, has trailing content, or names a field the shape does not have.
  • A read document carries its text time and digest — TestAReadDocumentCarriesItsTextTimeAndDigest. A JSON document is its title, a blank line and its text, with the time it says it happened; a text file is its bytes trimmed of trailing whitespace.
  • A segment key is stable and sensitive to speaker and subject — TestASegmentKeyIsStableAndSensitiveToSpeakerAndSubject. A segment's key is a function of the document, its position, the speaker and the subject: the same document sent twice replays, and the same words as another speaker or under another subject are another observation.