Skip to content

fix: Read all items when a tombstone has no key - #443

Open
jsonbailey wants to merge 3 commits into
mainfrom
jb/sdk-2995/tombstone-store-keys
Open

jsonbailey wants to merge 3 commits into
mainfrom
jb/sdk-2995/tombstone-store-keys

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Symptom

With Consul or DynamoDB, a single deleted item whose stored JSON carries no key
breaks the entire all-flags read. all_flags_state returns
{"$flagsState":{},"$valid":false}, so every flag falls back to its default.
Individual variation calls keep working — only the all-items read breaks.

Root cause

Both stores rebuilt the all-items map from the key inside the record body,
discarding the store key they had just read:

  • consul_impl.rb:76items_out[item[:key].to_sym] = item
  • dynamodb_impl.rb:116items_out[item_out[:key].to_sym] = item_out

For a keyless tombstone item[:key] is nil, and nil.to_sym raises
NoMethodError, which propagates out of the store and fails the whole read.

Keyless tombstones are the norm, not a corruption. .NET, Java, Node (Redis
upsert path) and Haskell all write {"version":N,"deleted":true}, and any SDK
can be pointed at a store another SDK wrote. The inner key is redundant anyway:
the store already addresses the record by key — the Consul KV path, the DynamoDB
sort key.

Redis was never affected because it already keys the map by the outer hash
field. Segments were unaffected on all stores because that path goes through
get(kind, key), which receives the key as a parameter.

Fix

Key the output map by the store's own key, which both implementations already
have in hand:

  • Consul — recover the item key from the KV path in result[:key], stripping
    the kind_key(kind) prefix. Entries outside that prefix are skipped rather
    than mis-keyed.
  • DynamoDB — read the sort key attribute from the raw item before
    unmarshal_item, and skip an item that unmarshals to nil.

No change was needed in the model layer: Ruby's FeatureFlag and Segment
constructors read data[:key] without requiring it, so a keyless tombstone
already decoded. (This is where the equivalent Python fix also had to change the
model.)

Tests

spec/feature_store_spec_base.rb gains a shared example, run for every
persistent store under all four caching/prefix permutations: a keyless tombstone
written straight to the database alongside a valid item must not spoil the
all-items read, and must read back as absent. Each store tester gains a
write_raw_item helper to set up data in a shape the store itself never writes.

Verified against the tombstone contract tests in
launchdarkly/sdk-test-harness#438, run locally with Redis, Consul and
DynamoDB-local:

before after
persistent data store (v2, -enable-persistence-tests) 2 failures 103 ran, 0 failures

The two pre-fix failures were exactly
consul/daemon mode/tombstones/flags/body has no key and
dynamodb/daemon mode/tombstones/flags/body has no key.

New rspec examples reproduce the same split: 8 of 12 fail without the fix
(Consul and DynamoDB), Redis's 4 pass either way.

Full suite: rspec spec 1354 examples / 0 failures, rubocop clean.

Related


Note

Overview
Fixes Consul and DynamoDB bulk reads (get_all_internal) so the returned map is keyed by each record’s storage key (Consul KV path suffix, DynamoDB sort key) instead of the optional key field inside the JSON. Keyless deletion tombstones ({version, deleted}) from other SDKs no longer trigger nil.to_sym and abort the whole all / all-flags read.

Adds shared persistent-store coverage that writes a raw keyless tombstone beside a valid item and asserts all still returns only live data, plus write_raw_item on Consul/DynamoDB/Redis testers and model deserialization examples for tombstones with and without a key.

Reviewed by Cursor Bugbot for commit 8010c8f. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey
jsonbailey added this pull request to stack #445 September 18, 2026 16:06
@jsonbailey
jsonbailey marked this pull request as ready for review September 18, 2026 16:14
@jsonbailey
jsonbailey requested a review from a team as a code owner September 18, 2026 16:14
The keyless-tombstone spec already requires the store to key its all-items
map by the key each item is stored under, because such an item carries no key
of its own. A record whose body key names a different item is a shape no SDK
writes, so the spec only covered data that cannot occur.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant