Skip to content

improvement(knowledge): serve date tag filters from an index - #8348

Merged
waleedlatif1 merged 1 commit into
stagingfrom
improvement/sargable-date-tag-filters
Sep 26, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
improvement/sargable-date-tag-filters

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Date tag filters compiled to col::date <op> d::date, which no btree on the timestamp column can serve. Every date filter walked past its slot index, so improvement(db): drop unusable date slot and redundant prefix indexes #8332 dropped those never-used indexes. This makes date filtering index-backed instead
  • Date filters now compile to half-open ranges on the raw timestamp column, e.g. eq d → col >= d AND col < d + 1 day. Each range selects exactly the rows the calendar-day comparison did: the slots have no time zone, and a NULL tag still matches nothing
  • One builder, buildDateTagCondition, is shared by search tag filters and the document list filter. Each keeps its existing fallback for unknown operators
  • New migration re-creates {emb,doc}_date1_idx / date2_idx concurrently

Type of Change

  • Improvement

Testing

  • Real-Postgres equivalence test: every operator, every day boundary (23:59:59.999999, midnight), NULL, and equal-endpoint and multi-day between, under three session time zones, compared against the old ::date predicate evaluated on the same rows. Five boundary mutations of the builder each turn it red
  • Isolated EXPLAIN test: selective eq, between, gte, and lt date filters use the date index from both builders. It fails on the pre-change builders
  • Knowledge search A/B on a synthetic 732k-chunk fixture, with identical results in every shape:
    • a single-day filter no longer seq-scans embedding in the vector leg's short-result rescue probe (55–61 ms → under 1 ms)
    • vector and hybrid date searches run 19–25% faster end to end
    • hybrid keyword ranking with a date filter drops from ~30 → ~10 ms
    • tags-only single-day filters add ~1 ms
  • kb-block-search, filtered-search, workspace-kb-document-access, search-index-policy, type-check, lint, check:audits, and check:migrations origin/staging pass
  • Migration number is renumbered at merge time, since other open PRs also add migrations

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 26, 2026 10:18pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 10 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Rebuilds date filtering logic and adds database indexes.

The PR appears safe to merge; no outstanding findings remain.

Summary

The PR changes date-tag comparisons to index-usable half-open timestamp ranges, shares the predicate builder between document listing and search, restores the date-slot indexes, and adds PostgreSQL equivalence and query-plan tests.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Document-list date filter] --> C[Shared date-range builder]
  B[Search date filter] --> C
  C --> D[Timestamp range predicate]
  D --> E[Document and embedding date-slot indexes]
Loading

Reviews (2) · Last reviewed commit: "improvement(knowledge): serve date tag f..."

Comment thread packages/db/migrations/0387_date_tag_range_indexes.sql
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
waleedlatif1 merged commit c479a5a into staging Sep 26, 2026
32 checks passed
@waleedlatif1
waleedlatif1 deleted the improvement/sargable-date-tag-filters branch September 26, 2026 23:24

This branch was successfully deployed

1 active deployment
Preview — 5feba82a Deployed Sep 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant