Index-Time Redaction

Index-time redaction operates on a record while it is being prepared for retrieval, replacing identifiers, account numbers, clinical details or other regulated values before any vector representation is produced. The alternative — redacting output — leaves the original text embedded, which creates two exposures that are easy to overlook. The first is retrieval: an embedding derived from unredacted text carries semantic signal from the sensitive content, so a query about that content can surface the record even when the displayed result is masked. The second is durability: anything embedded persists until the index is rebuilt, so a redaction rule introduced later does not retroactively clean what came before. Redacting at ingestion avoids both, at the cost of requiring the rules to be right before the content is loaded — which is why the rule set, rather than the redaction mechanism, is where the real work sits.

Data Cleaner applies redaction as part of the transformation that prepares each record, so a Social Security number or equivalent identifier is removed before the record reaches the Vector Index rather than suppressed on its way out. Because the rules are authored as records themselves, they carry version history: what was being redacted in a given quarter can be established rather than recalled. Data Transfer imports the organization's specific terms, laws and policy references into that dictionary, so the redaction reflects the regulatory environment the organization actually operates in rather than a generic pattern list.