diff --git a/.changeset/21840-contractless-datasource-credentials.md b/.changeset/21840-contractless-datasource-credentials.md new file mode 100644 index 00000000000..86d05dc48ca --- /dev/null +++ b/.changeset/21840-contractless-datasource-credentials.md @@ -0,0 +1,236 @@ +--- +'@objectstack/spec': minor +'@objectstack/service-datasource': patch +'@objectstack/metadata-protocol': patch +--- + +fix(spec)!: a datasource whose driver the platform ships no config contract for refuses inline credential material at publish, and every read door withholds it by name — `apiKey`, `client_secret`, `secretAccessKey`, `privateKey`, `accessToken` and a `Password=` connection-string segment included (#21840) + +Clause-②: yes (narrowing) + + + +**BREAKING**: a datasource config that published before can now be refused. For a +driver the platform ships no config contract for (a plugin-contributed driver such as +`com.vendor.warehouse`), `config` used to be validated against nothing, so credential +material other than a key literally named `password` / `authToken` (and their former +aliases) was accepted, stored in cleartext in `sys_metadata` and its history, and +served back on administrator reads. ADR-0015 §10 holds for every driver — +credentials never appear in metadata artefacts — so such a config now refuses it, by +name, with the remedy. It ships as `minor` under the launch-window convention for +accept-set narrowings. A driver WITH a contract (`postgres`, `mysql`, `mongodb`, +`turso`, `sqlite`, `sqlite-wasm`, `memory` and their aliases) is judged exactly as +before, at both doors. + +**What is refused at publish** (every door that parses `DatasourceSchema`: +`defineStack({ datasources })`, `PUT /api/v1/meta/datasource/:name`, and Setup → +Datasources create and update; the connection test answers `ok: false`), each at its +own `config.` — array elements included (`config.servers.0.password`): + +- **A value under a credential-shaped key:** a non-empty string, a number, non-empty + bytes (a `Buffer`, a typed array or an `ArrayBuffer`, judged as ONE value), or an + array holding a non-empty string, number or bytes (an array of objects there has + each object judged as a credential-shaped object, below). The key + (`isCredentialShapedConfigKey`) is NFKC-normalised first; a key that is then longer + than 256 characters is credential-shaped without being read. A key that still holds + a non-ASCII character is credential-shaped when that text sits inside or next to a + credential word: it holds a credential word of another script (`密码`, `密碼`, `口令`, + `密钥`, `密鑰`, `秘钥`, `令牌`, `凭证`, `憑證`, `パスワード`, `暗証番号`, `秘密鍵`, `トークン`, + `비밀번호`, `토큰`, `пароль`, `токен`, `contraseña`, matched inside the key's text); read + with its invisible format characters (soft hyphen, zero-width space and joiners, + word joiner, BOM) removed and its Cyrillic and Greek look-alike letters as the Latin + letters they render as, it is credential-shaped (a Cyrillic `а` inside `password`); + or, in a run with no ASCII separator that mixes ASCII letters or digits with + non-ASCII characters, the ASCII word touching a non-ASCII character is + credential-shaped on its own (`password密码`), or a credential word of four or more + letters is spelled with non-ASCII characters standing in for or inserted between at + most one letter in four, format characters inserted free (`tok€n`). Otherwise a run + of non-ASCII characters is a word of its own, so `客户名称`, `Größe` and `café` are not + credential-shaped. The key is judged on its whole name, split into words at + separators and camel-case boundaries, case-insensitively — words, never substrings. + It is never credential-shaped when its last word is a locator, identifier or + descriptor (`ref`, `refs`, `reference`, `arn`, `id`, `ids`, `name`, `names`, `path`, + `paths`, `file`, `files`, `filename`, `url`, `urls`, `uri`, `endpoint`, `env`, + `type`, `header`, `headers`, `field`, `prefix`, `mode`, `region`, `provider`, + `source`, `chain`, `policy`, `method`, `enabled`, `authentication`, `format`, + `version`, `expiry`, `expires`, `length`, `count`, or a word ending in `less`), or + when it has more than one word and its first is `use`, `enable`, `enabled`, + `disable`, `require`, `required`, `allow`, `has`, `is`, `no`, `skip`, `max`, `min`, + `num`, `count` or `total`. Otherwise it is credential-shaped when it is one of the + existing canonical spellings or, after dropping trailing `value`, `values`, `pem`, + `json`, `b64`, `base64`, `data`, `content`, `hex`, `string`, `str`, `raw` or `hash` + words and a plural `s` (never the `s` of a word already ending in `s`: `sass` is not + `sas`), when one of its words is `password`, `passwd`, `passphrase`, `secret` or + `credential`; when its last word is `token`, `pass`, `pw`, `pwd`, `jwt`, `pat`, + `cookie`, `sas`, `auth`, `authorization`, `bearer`, `apikey`, `privkey`, `pfx`, + `pkcs12` or `p12`, or folds + a compound ending (`db_accesstoken`); when its last word is `key` alone or beside + `api`, `private`, `secret`, `signing`, `master`, `encryption`, `decryption`, + `account`, `shared`, `client`, `session`, `auth`, `hmac`, `license`, `subscription`, + `ssh`, `aes`, `storage`, `ssl` or `tls`; when its last word is `signature` beside + `shared`, `access`, `sas`, `hmac` or `amz`; or when it names service-account key + material (`serviceAccountKey`, and `serviceAccount` before a dropped qualifier: + `serviceAccountJson`, `serviceAccountPem`). So `apiKeys`, `tokens`, `privateKeyPem`, + `apiKeyValue`, `tokenValue`, `keyJson`, `privateKeyData`, `tokenString`, `authData`, + `encryptionKeyHex`, `pwdHash`, `sslKey`, `tlsKey`, `ssl_key`, `privkey`, `pass`, + `pw`, `key`, `auth`, `Authorization`, `bearer`, `jwt`, `pat`, `cookie` and `sas` are + credential-shaped, while `credentialsRef`, `accessKeyId`, `tokenUrl`, + `passwordFile`, `secretsManagerRegion`, `credentialProvider`, + `useDefaultCredentials`, `passwordless`, `maxTokens`, `primaryKey`, `partitionKey`, + `sslMode`, `passive`, `bypass…` and a bare `accessKey` (the identity half of an + access-key pair) stay accepted. A one-word key with no boundary left (`APIKEY`, + `accesstoken`, `dbpassword`, `basicauth`) is judged on its folded spelling by the + same rules: it is credential-shaped when it is one of the stems above (`key` and + `signature` included), when it holds `password`, `passwd`, `passphrase`, `secret` + or `credential` followed by nothing, `key`, `accesskey`, `hash` or `string` + (`secretaccesskey` — not `secretary`), or when it ends in a folded key-material + compound (`accesstoken`, `apikey`, `privatekey`, `secretkey`, `serviceaccountkey`, + `serviceaccountjson`, `privkey`, `pfx`, `sslkey`, `tlskey`, `basicauth`, `bearerauth`, + `digestauth`, …); a one-word key starting with `max`, `min`, `num`, `total` or + `count` is not. **A bare `key` (or `keys`) in an object** is credential material + only inside a credential-shaped holder, a header-ish holder (a key one of whose words + is `header` or `headers`) when the object holding `key` is that holder's map and not + an element of a list under it — `headers: { key }` is the header named `key`, + `headers: [{ key: 'Authorization' }]` a pair's label — or a TLS holder (a key one of whose words is `ssl`, `tls`, `mtls`, `x509`, + `pfx`, `pkcs12` or starts with `cert`: `ssl: { key, cert, ca }`), or when its value + looks like key material: a string — or a list element — that looks like a secret + (`looksLikeSecretValue`: at least 16 characters with no whitespace that mix + upper-case letters, lower-case letters and digits, or at least 32 characters of + `A`–`Z`, `a`–`z`, `0`–`9`, `+`, `/`, `=`, `_`, `-`, `.`, `~` holding a digit), at + least 16 characters of hexadecimal holding a letter, or digit-free text of which 30% + to 70% of the letters are upper-case with at least four lower-to-upper steps that is + base64 (at least 16 characters of letters, `+` and `/` with up to two `=`, a length + divisible by four, holding a `+`, `/` or `=`) or at least 24 letters, `-` and `_`; + bytes count too. So `{ key: 'email' }` and `{ key: 'customerEmailAddress' }` are + accepted. + Everywhere else — a query, fragment or form parameter, a connection-string segment, + a pair label, a tuple's name — `key` keeps its full judgment. +- **The secret leaves of a credential-shaped object** (`credentials: {…}`, + `auth: {…}`): every leaf except one whose last word is a descriptor (the list above) + or an identity (`user`, `username`, `login`, `email`, `issuer`, `audience`, `scope`, + `scopes`, `algorithm`, `alg`, `domain`, `host`, `hostname`, `port`, `realm`, + `project`, `tenant`, `subject`, `kind`, `label`, `description`) — so + `credentials: { type, clientId }` is accepted whole. That exemption is for a LEAF + (a string, a number, bytes, or a list of them) only: an object below a descriptor + or identity key stays inside the credential-shaped context + (`auth: { source: { value } }` refuses `value`). +- **A header's value.** The `value` of a pair object — an object with a `value` and + any of `name`, `key`, `header` or `headerName` as a string label — when ANY of its + labels is credential-shaped (`{ key: 'Authorization', value }`); a label itself is + judged only for an embedded credential. Every element after the name of a + `[name, value, …]` tuple whose name is credential-shaped — a tuple inside a list + (two or more elements), or directly under a header-ish key (two elements, or an odd + number). In a flat list directly under a header-ish key with an even number of + elements (`rawHeaders: ['Authorization', '…', 'Accept', 'json']`), the element + after each credential-shaped name at an even position. +- **A string carrying a credential, anywhere** (pair labels included): a string + longer than 65,536 characters, which is not read at all (refused at publish, served + empty); a JSON-encoded object or array (the string's first non-space character is + `{` or `[` and it parses), walked by these same rules; PEM private-key armour + (`-----BEGIN … PRIVATE KEY-----`: `RSA`, `EC`, `DSA`, `ENCRYPTED`, `OPENSSH`, and + `PGP PRIVATE KEY BLOCK`), whose block is removed on read; a URL userinfo password (`scheme://`, a + stacked `jdbc:mysql://` or a scheme-relative `//`); a URL userinfo username with no + password, or an empty one, that looks like a secret (the rule above: + `https://ghp_…@host`); a URL query or fragment pair whose name is credential-shaped + or is `sig`, or whose `;key=value` run carries a credential (`?api_key=`, + `&X-Amz-Signature=`, `#access_token=`, `#password=`); a credential property in a + URL's `;key=value` tail (`sqlserver://h;user=u;password=p`); the Oracle thin-driver + userinfo (`jdbc:oracle:thin:user/password@…`); a scheme-less userinfo password + (`user:password@host/db`; after an opaque `mailto:`, `sip:`, `sips:`, `tel:`, + `urn:`, `xmpp:`, `news:`, `im:` or `pres:` prefix the rest is read instead, and a + time of day before the `@` — `12:30@` — is none); a `Name: value` header line, + on any line of a multi-line string or of any string under a header-ish key, whose + name is a header token that is credential-shaped and whose value is non-empty + (`Authorization: Bearer …`; a one-line `description: 'Password: …'` is prose); a libpq keyword/value pair whose keyword is + credential-shaped, found leniently — a keyword at the start of the string or after + whitespace, optional whitespace, `=`, and a value single-quoted with `\'` / `\\` + escapes or a run of non-space characters; any other token is skipped, and an + unclosed quote runs to the end of the string (`host=h password=p`, an unquoted `;` + in the value included); a credential segment of a semicolon-delimited connection + string (`Server=h;Password=p`, `Pwd=`, `AccountKey=`; quoted values honoured) — + a libpq or segment value that starts with a SQL bind placeholder (`$1`, `?`, + `:name`) ending at whitespace, `)`, `,`, `;` or the end is none + (`WHERE token = $1`); and + a credential pair of a form-encoded string — no whitespace, an `&` and an `=`, an + optional leading `?` (`a=b&pass=c`). Every key, segment key, parameter name and + header name inside a string is judged by the same key rule, the 256-character cap + and the non-ASCII reading included. Bytes outside a credential position are judged + by their UTF-8 text, as one value; more than 65,536 bytes are judged credential + material unread. +- **A subtree nested deeper than 16 levels** (a JSON-encoded string's contents + counted from where the string sits), and **a `Map` or a `Set` anywhere**, which + cannot be judged and are not accepted unjudged. + +**Still accepted:** an empty string (the explicit way to clear a stored value), a +boolean, a value made only of environment placeholders in the `${NAME}` grammar (an +upper-case environment name, `${API_KEY}`; any other `${…}` content is judged as +written), plain array data with no credential-shaped key, and every other key — the +config shape itself stays unjudged. + +**What an author sees now, and the remedy it names.** Each refusal is a `custom` issue +at the value's own `config.`, naming the position and the remedy: remove the +inline credential from `config` and bind it as the datasource's secret — the +connection form's secret field, or `external.credentialsRef`. The connect path hands +the decrypted value to the driver factory as the connection secret, so a plugin +driver receives it only if its factory reads that injected secret. No key is retired +or renamed; nothing an author wrote is rewritten. + +**Every read door withholds it, rows stored before this release included.** The one +read-path redactor (`redactDatasourceConfig`) now withholds, for a contractless +driver, every position the write door refuses — both doors read ONE walk +(`findContractlessCredentials`): a credential value, a subtree too deep to judge and a +`Map` or `Set` are dropped (inside an array element too: spliced from the end of its +array, nulled when siblings follow it), and a credential embedded in a string is +removed from it — a JSON-encoded string keeps its other members, a header line keeps +its name, and a string the rewrite cannot clear is served empty. What is served holds +no finding itself — it is what an untouched Save hands the write door: the projection +is judged again until it is clean (a pair's `key` label left alone directly under a +header-ish key goes too), and one that has not settled after 8 passes is served +empty. That covers `/api/v1/meta/datasource` (item, list, +`/published`, `/layers`, history), `/api/v1/datasources` (item and list), the generic +data door over `sys_metadata` / `sys_metadata_history`, and the audit ledger's and +activity feed's copies of those rows (new copies at write time; for copies written +before this release, run `os migrate audit-metadata-bodies --apply`, which projects +them through the same redactor). An untouched +Save of a legacy row's edit form carries the withheld values forward, as before. A +value withheld inside an array is carried onto the element it came from, by identity, +never by index alone: the same index in an array left exactly as served; otherwise the +one element equal to the served one — every non-credential sibling of the withheld +value included — unique in the served array and in the saved one, wherever it now sits +(a reorder, or a sibling deleted before it). A withheld value whose element changed (a +renamed header, an edited sibling field), is gone, or cannot be told apart from +another is dropped; one that is itself an array element (a tuple's or raw-headers +list's value) is carried only into an array left exactly as served. + +**`@objectstack/metadata-protocol`:** the `/api/v1/meta` PUT carry-forward +(`carryForwardRedactedValues`) applies that same identity rule to an array element with +no `id` and no identified element below it, and to an array inside an array — which it +used to skip, so an unchanged GET then PUT of a legacy contractless row silently +deleted every credential withheld inside an array. A flow node with no `id` is followed +the same way. + +**`@objectstack/service-datasource`:** `restoreRedactedConfig` follows the identity +rule above, and carries a value forward only where the read path would still withhold +it: the grafted config is redacted again and a graft survives only when its landing +position is still withheld, repeated until nothing more drops — so a pair's value +beside a label renamed to a non-credential name, a deleted label, or a renamed `key:` +label is dropped, never served. An untouched Save skips the second walk. The credential-migration planner now reports a +contractless row's top-level keys that hold any such finding as residue and refuses with its remedy, instead of answering +`nothing-to-migrate` while the value sits in cleartext. + +**New `@objectstack/spec/data` exports:** `isCredentialShapedConfigKey`, +`embeddedCredentialOf`, `redactEmbeddedCredentials`, `connectionStringCredentialKeys`, +`findContractlessCredentials` (with its `ContractlessCredentialFinding` type, whose +`kind` is `named`, `embedded`, `depth` or `opaque`, and +`CONTRACTLESS_CREDENTIAL_WALK_DEPTH`; an `embedded` finding under a header-ish key +carries `headerish: true`), `withholdContractlessCredentials`, `looksLikeSecretValue`, +`MAX_JUDGED_STRING_LENGTH`, the `EmbeddedCredentialOptions` type (`headerish`, taken +by `embeddedCredentialOf` and `redactEmbeddedCredentials`), and `isContractlessDriver`. + +⚠️ A config that inlines a value longer than 65,536 characters under a contractless +driver (a large CA bundle, for instance) is refused at publish and served empty; +reference such material by path instead. + +⚠️ **The out-of-repo consumer population is NOT MEASURED.** No datasource in this +repository uses a contractless driver; plugin drivers in other repositories that +authored inline credentials will see the refusal on their next publish. diff --git a/packages/metadata-protocol/src/metadata-redaction-array-carry-forward.test.ts b/packages/metadata-protocol/src/metadata-redaction-array-carry-forward.test.ts new file mode 100644 index 00000000000..7798f56e70f --- /dev/null +++ b/packages/metadata-protocol/src/metadata-redaction-array-carry-forward.test.ts @@ -0,0 +1,117 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The `/meta` PUT carry-forward ({@link carryForwardRedactedValues}) through + * arrays whose elements carry no `id`: a datasource of a driver the platform + * ships no config contract for has its credentials withheld inside array + * elements (`config.servers..password`), inside a header tuple + * (`config.headers..`, an array inside an array) and as an array element + * itself (`config.headers.` in a raw-headers list). + * + * An unchanged GET → PUT keeps every one of them; an edit carries a value only + * onto the element it came from — the same rule as the datasource admin + * service's `restoreRedactedConfig` — and drops it where that element changed, + * is gone, or cannot be told apart from another. + */ + +import { describe, expect, it } from 'vitest'; +import { carryForwardRedactedValues, redactMetadataItem } from './metadata-redaction.js'; + +const DRIVER = 'com.vendor.warehouse'; + +const STORED = { + name: 'warehouse', + driver: DRIVER, + config: { + host: 'h', + servers: [{ host: 'a', password: 'p-a' }, { host: 'b', password: 'p-b' }], + headers: [{ name: 'Authorization', value: 'Bearer h-1' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['Authorization', 'Bearer t-1']], + rawHeaders: ['Authorization', 'Bearer r-1', 'Accept', 'json'], + }, +}; + +/** What the read exits serve for STORED. */ +const served = () => structuredClone(redactMetadataItem('datasource', STORED)); + +/** `served()` with its `config` replaced key by key. */ +const edited = (config: Record) => { + const item = served(); + return { ...item, config: { ...item.config, ...config } }; +}; + +const carry = (incoming: unknown) => + carryForwardRedactedValues('datasource', incoming, STORED) as { config: Record }; + +describe('the /meta carry-forward through id-less array elements and nested arrays', () => { + it('the served body withholds every array-borne credential', () => { + expect(served().config).toEqual({ + host: 'h', + servers: [{ host: 'a' }, { host: 'b' }], + headers: [{ name: 'Authorization' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['Authorization']], + rawHeaders: ['Authorization', null, 'Accept', 'json'], + }); + }); + + it('an unchanged GET → PUT of a legacy row deletes nothing', () => { + expect(carry(served())).toEqual(STORED); + }); + + it('an element deleted before it: the remaining server keeps ITS OWN password', () => { + expect(carry(edited({ servers: [{ host: 'b' }] })).config.servers).toEqual([{ host: 'b', password: 'p-b' }]); + }); + + it('a reorder: every value follows its element, a tuple inside a list included', () => { + const out = carry( + edited({ + servers: [{ host: 'b' }, { host: 'a' }], + headers: [{ name: 'Accept', value: 'application/json' }, { name: 'Authorization' }], + tuples: [['Authorization'], ['Accept', 'json']], + }), + ); + expect(out.config).toMatchObject({ + servers: [{ host: 'b', password: 'p-b' }, { host: 'a', password: 'p-a' }], + headers: [{ name: 'Accept', value: 'application/json' }, { name: 'Authorization', value: 'Bearer h-1' }], + tuples: [['Authorization', 'Bearer t-1'], ['Accept', 'json']], + }); + }); + + it('a renamed header, an edited sibling field or an edited raw-headers list receives nothing', () => { + const out = carry( + edited({ + servers: [{ host: 'a2' }, { host: 'b' }], + headers: [{ name: 'X-Other' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['X-Other']], + rawHeaders: ['Authorization', null, 'Accept', 'xml'], + }), + ); + expect(out.config).toEqual({ + host: 'h', + servers: [{ host: 'a2' }, { host: 'b', password: 'p-b' }], + headers: [{ name: 'X-Other' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['X-Other']], + rawHeaders: ['Authorization', null, 'Accept', 'xml'], + }); + expect(JSON.stringify(out)).not.toMatch(/p-a|h-1|t-1|r-1/); + }); + + it('elements that cannot be told apart receive nothing once the array was edited', () => { + const stored = { + name: 'w', + driver: DRIVER, + config: { headers: [{ name: 'Authorization', value: 'v-1' }, { name: 'Authorization', value: 'v-2' }, { name: 'Accept', value: 'json' }] }, + }; + const item = structuredClone(redactMetadataItem('datasource', stored)); + expect(carryForwardRedactedValues('datasource', item, stored)).toEqual(stored); + const trimmed = { ...item, config: { headers: [{ name: 'Authorization' }, { name: 'Authorization' }] } }; + expect(carryForwardRedactedValues('datasource', trimmed, stored)).toEqual(trimmed); + }); + + it('the incoming body is never mutated', () => { + const incoming = served(); + const before = JSON.stringify(incoming); + carry(incoming); + expect(JSON.stringify(incoming)).toBe(before); + }); +}); diff --git a/packages/metadata-protocol/src/metadata-redaction.ts b/packages/metadata-protocol/src/metadata-redaction.ts index 9a22cc8a0ca..89708ea2feb 100644 --- a/packages/metadata-protocol/src/metadata-redaction.ts +++ b/packages/metadata-protocol/src/metadata-redaction.ts @@ -202,50 +202,88 @@ function elementWithIdentity( * the first identified element beneath it. * * An array hop that reaches neither — an element with no `id` and no - * identified element below it on the path, or an `id` shared with a sibling — - * resolves to nothing, and the path is skipped. + * identified element below it on the path (a datasource's `servers.` or + * `headers.`), or an element that is itself an array (a header tuple, + * `headers..`) — is resolved by its SERVED PROJECTION (`{ served, + * index }`): the element exactly as the read path served it, every + * non-credential sibling of the withheld value included. In a body whose + * array equals the served array it is the same index; otherwise it is the one + * element equal to that projection, which must be unique in the served array + * and in the body — so a reorder, or a sibling deleted before it, still + * carries the value onto its own element, while an element that changed (a + * renamed header, an edited sibling field), is gone, or cannot be told apart + * from another receives nothing. Same rule as the datasource admin service's + * `restoreRedactedConfig`. + * + * An `id` shared with a sibling resolves to nothing, and the path is skipped. */ type PathHop = | { readonly key: string } | { readonly elementId: string } - | { readonly anchor: readonly PathHop[] }; + | { readonly anchor: readonly PathHop[] } + | { readonly served: readonly unknown[]; readonly index: number }; /** * Resolve every CONTAINER hop of `segments` (all but the last, which names the - * redacted key itself) against `stored`. `undefined` when the stored body does - * not reach that far, or an array hop has no identity — which the caller reads - * as "nothing at rest to carry". + * redacted key itself) against `stored`, walking `served` (the read path's + * projection of `stored`, whose arrays keep their indices) alongside for the + * projection hops. `undefined` when the stored body does not reach that far, + * or an `id` is shared — which the caller reads as "nothing at rest to carry". */ -function resolveHops(stored: unknown, segments: readonly string[]): PathHop[] | undefined { - // First pass: key hops and `id` hops; `null` marks an element with no `id`. +function resolveHops(stored: unknown, served: unknown, segments: readonly string[]): PathHop[] | undefined { + // First pass: key hops, `id` hops and projection hops; `null` marks an + // id-less record element, anchored or projected in the second pass. const found: Array = []; + const projections: Array<{ served: readonly unknown[]; index: number } | undefined> = []; let node: unknown = stored; + let servedNode: unknown = served; for (let i = 0; i < segments.length - 1; i += 1) { const segment = segments[i] as string; if (Array.isArray(node)) { if (!/^(0|[1-9][0-9]*)$/.test(segment)) return undefined; - const element = node[Number(segment)]; - if (!isPlainRecord(element)) return undefined; - const elementId = identityOf(element); - if (elementId !== undefined && !elementWithIdentity(node, elementId)) return undefined; - found.push(elementId === undefined ? null : { elementId }); + const index = Number(segment); + const element = node[index]; + const projection = Array.isArray(servedNode) && index < servedNode.length + ? { served: servedNode as readonly unknown[], index } + : undefined; + if (isPlainRecord(element)) { + const elementId = identityOf(element); + if (elementId !== undefined && !elementWithIdentity(node, elementId)) return undefined; + found.push(elementId === undefined ? null : { elementId }); + projections.push(projection); + } else if (Array.isArray(element) && projection) { + found.push(projection); + projections.push(projection); + } else { + return undefined; + } node = element; + servedNode = Array.isArray(servedNode) ? servedNode[index] : undefined; continue; } if (!isPlainRecord(node)) return undefined; found.push({ key: segment }); + projections.push(undefined); node = node[segment]; + servedNode = isPlainRecord(servedNode) ? servedNode[segment] : undefined; } // Second pass, from the end: anchor each id-less element on the first // identified element below it. The anchor may itself cross an id-less // element (a branch inside a branch), whose own anchor is already built. + // An id-less element with no identified element below it is resolved by + // its served projection. const hops: PathHop[] = new Array(found.length); let nextIdentified = -1; for (let i = found.length - 1; i >= 0; i -= 1) { const hop = found[i]; if (hop === null) { - if (nextIdentified < 0) return undefined; - hops[i] = { anchor: hops.slice(i + 1, nextIdentified + 1) }; + if (nextIdentified >= 0) { + hops[i] = { anchor: hops.slice(i + 1, nextIdentified + 1) }; + continue; + } + const projection = projections[i]; + if (!projection) return undefined; + hops[i] = projection; continue; } hops[i] = hop as PathHop; @@ -266,6 +304,10 @@ function stepInto(node: unknown, hop: PathHop): { segment: string; next: unknown return isPlainRecord(node) ? { segment: hop.key, next: node[hop.key] } : undefined; } if (!Array.isArray(node)) return undefined; + if ('served' in hop) { + const index = projectedIndex(hop.served, node, hop.index); + return index === undefined ? undefined : { segment: String(index), next: node[index] }; + } let index: number | undefined; for (let i = 0; i < node.length; i += 1) { const hit = 'elementId' in hop @@ -278,6 +320,27 @@ function stepInto(node: unknown, hop: PathHop): { segment: string; next: unknown return index === undefined ? undefined : { segment: String(index), next: node[index] }; } +/** + * The index in `array` of the element the read path served at `index` of + * `served`, by its projection: the same index when the arrays are equal, else + * the one element equal to the served one when it is unique on both sides. + */ +function projectedIndex(served: readonly unknown[], array: readonly unknown[], index: number): number | undefined { + if (sameValue(served, array)) return index; + const element = served[index]; + if (served.filter((candidate) => sameValue(candidate, element)).length !== 1) return undefined; + let found: number | undefined; + for (let i = 0; i < array.length; i += 1) { + if (!sameValue(array[i], element)) continue; + if (found !== undefined) return undefined; + found = i; + } + return found; +} + +/** A container a redacted key sits in: a plain object, or an array whose ELEMENT was withheld. */ +type Container = Record | unknown[]; + /** * Walk `hops` in `root`: the plain object that OWNS the redacted key, and the * concrete segments (a key, or an array index in THIS body) that reach it. @@ -290,7 +353,7 @@ function stepInto(node: unknown, hop: PathHop): { segment: string; next: unknown * grafting `config.password` back onto it would MINT a config that holds * nothing but a credential. */ -function locate(root: unknown, hops: readonly PathHop[]): { at: string[]; container: Record } | undefined { +function locate(root: unknown, hops: readonly PathHop[]): { at: string[]; container: Container } | undefined { const at: string[] = []; let node: unknown = root; for (const hop of hops) { @@ -299,11 +362,11 @@ function locate(root: unknown, hops: readonly PathHop[]): { at: string[]; contai at.push(step.segment); node = step.next; } - return isPlainRecord(node) ? { at, container: node } : undefined; + return isPlainRecord(node) || Array.isArray(node) ? { at, container: node } : undefined; } /** {@link locate}, container only. */ -function containerAt(root: unknown, hops: readonly PathHop[]): Record | undefined { +function containerAt(root: unknown, hops: readonly PathHop[]): Container | undefined { return locate(root, hops)?.container; } @@ -340,10 +403,13 @@ function valueAt(root: unknown, at: readonly string[]): unknown { * An owner whose array is not held under a key — an array directly inside * another array — has no key to scope by, and is not relocated. * - * A path with no identified element — every datasource path, whose redactor - * never crosses an array — has no owner to find, and is unaffected. + * A path with no identified element — every datasource path, whose + * contractless-driver redactor crosses arrays of id-less elements + * (`config.servers..password`) but never an `id` — has no owner to find, + * and is unaffected: its array hops are resolved by projection instead + * ({@link resolveHops}). */ -function relocateById(root: unknown, hops: readonly PathHop[]): { at: string[]; container: Record } | undefined { +function relocateById(root: unknown, hops: readonly PathHop[]): { at: string[]; container: Container } | undefined { let owner = -1; for (let i = hops.length - 1; i >= 0; i -= 1) { if ('elementId' in (hops[i] as PathHop)) { @@ -394,7 +460,14 @@ function relocateById(root: unknown, hops: readonly PathHop[]): { at: string[]; * not mutate it in place. */ function withValueAt(root: unknown, at: readonly string[], key: string, value: unknown): unknown { - if (at.length === 0) return { ...(root as Record), [key]: value }; + if (at.length === 0) { + if (Array.isArray(root)) { + const next = root.slice(); + next[Number(key)] = value; + return next; + } + return { ...(root as Record), [key]: value }; + } const [head, ...rest] = at as [string, ...string[]]; if (Array.isArray(root)) { const next = root.slice(); @@ -405,6 +478,22 @@ function withValueAt(root: unknown, at: readonly string[], key: string, value: u return { ...record, [head]: withValueAt(record[head], rest, key, value) }; } +/** The value `segments` names in `root` (an array entered at an index segment), or `undefined` off the walk. */ +function valueAtSegments(root: unknown, segments: readonly string[]): unknown { + let node: unknown = root; + for (const segment of segments) { + if (Array.isArray(node)) { + if (!/^(0|[1-9][0-9]*)$/.test(segment)) return undefined; + node = node[Number(segment)]; + } else if (isPlainRecord(node)) { + node = node[segment]; + } else { + return undefined; + } + } + return node; +} + /** Structural equality for the values a redactor hides (scalars in practice; general by construction). */ function sameValue(a: unknown, b: unknown): boolean { if (a === b) return true; @@ -445,8 +534,13 @@ function sameValue(a: unknown, b: unknown): boolean { * that reorders the array still carries the value onto the element it came * from — see {@link resolveHops}. An element with no `id` is walked by the * identified element below it on the same path (a `parallel` branch, by the - * node inside it that holds the credential, #20590); one with neither, or an - * `id` shared with a sibling, is never carried into. + * node inside it that holds the credential, #20590); one with neither — or an + * element that is itself an array — is walked by its SERVED PROJECTION: the + * same index in an array left exactly as served, else the one element equal + * to the served one, unique on both sides, and otherwise nothing is carried + * into it. An `id` shared with a sibling is never carried into. A withheld + * value that is itself an array element is carried only into an array left + * exactly as served. * * ⛔ A carried value never lands where the read would SERVE it (#20590). The * array hop follows an identity, while a redactor chooses what to withhold by @@ -538,11 +632,12 @@ function planCarryForward(type: string, incoming: T, stored: unknown): { out: // {@link resolveHops}). const segments = path.split('.'); const key = segments[segments.length - 1] as string; - const hops = resolveHops(stored, segments); + const hops = resolveHops(stored, served.item, segments); if (!hops) continue; - const storedParent = containerAt(stored, hops); - const storedValue = storedParent?.[key]; + // The stored value sits at the redactor's own path into the stored + // body — its indices ARE the stored body's, so it is read by them. + const storedValue = valueAtSegments(stored, segments); if (storedValue === undefined) continue; // Where the container sits in the incoming body: along the stored @@ -552,7 +647,13 @@ function planCarryForward(type: string, incoming: T, stored: unknown): { out: if (!target) continue; const servedParent = containerAt(served.item, hops); - if (!sameValue(target.container[key], servedParent?.[key])) continue; + if (Array.isArray(target.container)) { + // The withheld value is itself an array ELEMENT (a header tuple's + // value): carried only into an array left exactly as served. + if (!/^(0|[1-9][0-9]*)$/.test(key) || !sameValue(target.container, servedParent)) continue; + } else if (!sameValue(target.container[key], (servedParent as Record | undefined)?.[key])) { + continue; + } grafts.push({ at: target.at, key, value: storedValue }); } diff --git a/packages/metadata-protocol/src/protocol.metadata-redaction.test.ts b/packages/metadata-protocol/src/protocol.metadata-redaction.test.ts index 7fdebb31881..ca9cfa03f63 100644 --- a/packages/metadata-protocol/src/protocol.metadata-redaction.test.ts +++ b/packages/metadata-protocol/src/protocol.metadata-redaction.test.ts @@ -579,11 +579,18 @@ describe('#20552 — carryForwardRedactedValues walks an array hop by IDENTITY', const out: any = carryForwardRedactedValues('flow', twins, stored); expect(out.nodes.every((n: any) => n.config?.secret === undefined)).toBe(true); - // A stored start node with no `id` gives the hop no identity to follow. + // A stored start node with no `id` is followed by its served projection + // instead: an untouched body carries the secret back (an unchanged + // GET → PUT deletes nothing), while a body in which that node CHANGED + // gives the hop nothing to follow — the value is dropped. const idless: any = storedInboundFlow(); delete idless.nodes[1].id; const idlessServed = redactMetadataItem('flow', idless) as any; - const idlessOut: any = carryForwardRedactedValues('flow', idlessServed, idless); + const untouched: any = carryForwardRedactedValues('flow', structuredClone(idlessServed), idless); + expect(startNodeOf(untouched).config.secret).toBe(FLOW_SECRET); + const relabelled = structuredClone(idlessServed); + startNodeOf(relabelled).label = 'Relabelled'; + const idlessOut: any = carryForwardRedactedValues('flow', relabelled, idless); expect(startNodeOf(idlessOut).config.secret).toBeUndefined(); }); }); diff --git a/packages/qa/dogfood/test/datasource-contractless-credentials.dogfood.test.ts b/packages/qa/dogfood/test/datasource-contractless-credentials.dogfood.test.ts new file mode 100644 index 00000000000..100610abcd0 --- /dev/null +++ b/packages/qa/dogfood/test/datasource-contractless-credentials.dogfood.test.ts @@ -0,0 +1,287 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Credential-shaped configuration of a datasource whose driver the platform + * ships NO config contract for — on the real showcase composition (ObjectQL + * over SQLite, security, REST, the metadata service, the datasource admin + * service, the audit writer), driven through the real doors, across a cold + * boot on one database file. + * + * Before: such a datasource's `config` was validated against nothing, so + * `apiKey`, `client_secret`, `secretAccessKey`, `privateKey`, `accessToken`, a + * `Password=` segment inside a connection string, a password inside an array + * element and an `Authorization` header pair were accepted at publish, + * stored cleartext in `sys_metadata` and its history, and served back on every + * administrator read door — only `password` / `token`-style names and URL + * credentials were withheld. + * + * Pinned here: + * + * 1. both write doors (`PUT /meta/datasource/:name`, `POST /datasources`) + * refuse that material before anything is stored, and accept the same + * datasource without it; + * 2. a row stored BEFORE the refusal existed (seeded at rest, then a cold + * boot) is served by every read door with its non-secret configuration and + * none of its credentials — `/meta` (item, list, published, layers, + * history), the datasource admin routes, the generic data door over + * `sys_metadata` / `sys_metadata_history`, and the audit ledger's copy; + * 3. the admin edit door carries a withheld value forward only where the read + * path would still withhold it: a pair's value beside a renamed label is + * dropped, not stored where the next read would serve it. + * + * The harness mounts the datasource admin SERVICE but not its REST routes, and + * no audit writer; this file mounts both itself, exactly as `os serve` does + * (`registerDatasourceAdminRoutes(httpServer, ctx, '/api/v1')` from a plugin + * resolving `http.server`, and `AuditPlugin`). + * + * Every value below is a probe sentinel, not a credential. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import showcaseStack from '@objectstack/example-showcase'; +import { AuditPlugin } from '@objectstack/plugin-audit'; +import { registerDatasourceAdminRoutes } from '@objectstack/service-datasource'; +import { bootStack, type VerifyStack } from '@objectstack/verify'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const NAME = 'zz_contractless_ds'; +const DRIVER = 'com.vendor.warehouse'; +/** A non-secret value each read door must still serve — the positive control. */ +const MARKER = 'wh-legacy-host-5c1e'; +const SECRETS = { + apiKey: 'pin-dogfood-apikey-7a21', + clientSecret: 'pin-dogfood-clientsecret-3b90', + secretAccessKey: 'pin-dogfood-sak-e4d2', + privateKey: 'pin-dogfood-privatekey-91fc', + accessToken: 'pin-dogfood-accesstoken-06ab', + connectionPassword: 'pin-dogfood-cspassword-c8e7', + serverPassword: 'pin-dogfood-serverpw-5d13', + headerBearer: 'pin-dogfood-headerbearer-a2f6', +} as const; +const ALL = Object.values(SECRETS); +const SYSTEM = { isSystem: true } as const; + +/** The datasource as an author wrote it before the refusal existed. */ +const LEGACY_BODY = { + name: NAME, + label: 'Contractless warehouse', + driver: DRIVER, + origin: 'runtime', + config: { + host: MARKER, + apiKey: SECRETS.apiKey, + oauth: { clientId: 'cid', client_secret: SECRETS.clientSecret }, + secretAccessKey: SECRETS.secretAccessKey, + privateKey: SECRETS.privateKey, + accessToken: SECRETS.accessToken, + connectionString: `Server=${MARKER};User Id=u;Password=${SECRETS.connectionPassword}`, + servers: [{ host: MARKER, password: SECRETS.serverPassword }], + headers: [{ name: 'Authorization', value: `Bearer ${SECRETS.headerBearer}` }], + }, +}; + +/** The same datasource with no credential material — what the write doors accept. */ +const CLEAN_BODY = { name: NAME, label: 'Contractless warehouse', driver: DRIVER, config: { host: MARKER } }; + +const rowsOf = (r: any): any[] => (Array.isArray(r) ? r : Array.isArray(r?.records) ? r.records : Array.isArray(r?.value) ? r.value : []); +const leaked = (text: string): string[] => ALL.filter((secret) => text.includes(secret)); + +describe('contractless-driver datasource credentials: refused at write, withheld on every read door', () => { + let stack: VerifyStack; + let token: string; + let prevCwd: string; + let dir: string; + let dbFile: string; + + const boot = async () => { + // The `/api/v1/datasources` routes, mounted the way `os serve` mounts them. + const adminRoutes = { + name: 'dogfood.datasource-admin-routes', + version: '1.0.0', + optionalDependencies: ['com.objectstack.server.hono'], + init: async (ctx: any) => { + const httpServer = ctx.getService?.('http.server') ?? ctx.getService?.('http-server'); + registerDatasourceAdminRoutes(httpServer, ctx, '/api/v1'); + }, + }; + stack = await bootStack(showcaseStack, { + databaseFile: dbFile, + extraPlugins: [new AuditPlugin(), adminRoutes as never], + }); + token = await stack.signIn(); + }; + + const call = async (method: string, path: string, body?: unknown) => { + const res = await stack.apiAs(token, method, path, body); + return { status: res.status, text: await res.text() }; + }; + + const stored = async (object: string): Promise => { + const ql: any = await stack.kernel.getServiceAsync('objectql'); + return rowsOf(await ql.find(object, { where: { name: NAME }, context: SYSTEM })); + }; + + beforeAll(async () => { + prevCwd = process.cwd(); + dir = mkdtempSync(join(tmpdir(), 'dogfood-contractless-ds-')); + process.chdir(dir); + dbFile = join(dir, 'showcase.db'); + await boot(); + }, 180_000); + + afterAll(async () => { + await stack?.stop(); + if (prevCwd) process.chdir(prevCwd); + if (dir) rmSync(dir, { recursive: true, force: true }); + }); + + it('1 — both write doors refuse the credential material before anything is stored', async () => { + // The metadata save door: the spec gate's refusal, every position named. + const meta = await call('PUT', `/meta/datasource/${NAME}`, LEGACY_BODY); + expect(meta.status, meta.text).toBe(422); + const metaBody = JSON.parse(meta.text); + expect(metaBody.code, meta.text).toBe('INVALID_METADATA'); + expect((metaBody.issues as Array<{ path: string }>).map((issue) => issue.path).sort()).toEqual([ + 'config.accessToken', + 'config.apiKey', + 'config.connectionString', + 'config.headers.0.value', + 'config.oauth.client_secret', + 'config.privateKey', + 'config.secretAccessKey', + 'config.servers.0.password', + ]); + expect(leaked(meta.text)).toEqual([]); + + // The Setup → Datasources create door. + const admin = await call('POST', '/datasources', LEGACY_BODY); + expect(admin.status, admin.text).toBe(400); + expect(admin.text).toContain('config.apiKey'); + expect(leaked(admin.text)).toEqual([]); + + expect(await stored('sys_metadata')).toEqual([]); + }); + + it('1b — control: the same datasource without credential material saves', async () => { + const meta = await call('PUT', `/meta/datasource/${NAME}`, CLEAN_BODY); + expect(meta.status, meta.text).toBe(200); + expect(await stored('sys_metadata')).toHaveLength(1); + }); + + it('2 — a row stored before the refusal existed is served by every read door without its credentials', async () => { + // Seed the pre-refusal state AT REST: the active row and every history + // row hold the legacy body, exactly as an earlier release stored it. + const ql: any = await stack.kernel.getServiceAsync('objectql'); + for (const object of ['sys_metadata', 'sys_metadata_history']) { + for (const row of await stored(object)) { + await ql.update(object, { metadata: JSON.stringify(LEGACY_BODY) }, { where: { id: row.id }, context: SYSTEM }); + } + } + // Precondition: the cleartext really is at rest (else nothing below is measured). + expect(leaked(JSON.stringify(await stored('sys_metadata')))).toEqual(ALL); + expect(leaked(JSON.stringify(await stored('sys_metadata_history')))).toEqual(ALL); + + // A cold boot on the same file, so every in-memory view is rebuilt from the stored rows. + await stack.stop(); + await boot(); + + const [active] = await stored('sys_metadata'); + expect(active?.id, 'the seeded row survived the restart').toBeTruthy(); + const filter = (where: Record) => encodeURIComponent(JSON.stringify(where)); + // `serves`: what the answer must contain — the positive control that the + // door read the seeded row at all. A door serving the body must carry the + // non-secret MARKER from its config; the admin list serves summaries (no + // config) and must name the row; the history list serves versions only. + const doors: Array<{ path: string; serves?: string }> = [ + { path: `/meta/datasource/${NAME}`, serves: MARKER }, + { path: '/meta/datasource', serves: MARKER }, + { path: `/meta/datasource/${NAME}/published`, serves: MARKER }, + { path: `/meta/datasource/${NAME}/layers`, serves: MARKER }, + { path: `/meta/datasource/${NAME}/history` }, + { path: `/datasources/${NAME}`, serves: MARKER }, + { path: '/datasources', serves: NAME }, + { path: `/data/sys_metadata?filter=${filter({ name: NAME })}`, serves: MARKER }, + { path: `/data/sys_metadata/${active.id}`, serves: MARKER }, + { path: `/data/sys_metadata_history?filter=${filter({ name: NAME })}`, serves: MARKER }, + ]; + for (const { path, serves } of doors) { + const res = await call('GET', path); + expect.soft(res.status, `${path} answers`).toBe(200); + expect.soft(leaked(res.text), `${path} serves no credential`).toEqual([]); + if (serves) expect.soft(res.text.includes(serves), `${path} serves the row (positive control)`).toBe(true); + } + + // The admin edit form's view names what it withheld and keeps the rest. + const form = JSON.parse((await call('GET', `/datasources/${NAME}`)).text); + const config = (form.datasource ?? form.data?.datasource ?? form).config; + expect(config).toEqual({ + host: MARKER, + oauth: { clientId: 'cid' }, + connectionString: `Server=${MARKER};User Id=u`, + servers: [{ host: MARKER }], + headers: [{ name: 'Authorization' }], + }); + }, 180_000); + + it('3 — the audit ledger records the seeded write without its credentials', async () => { + const ql: any = await stack.kernel.getServiceAsync('objectql'); + // The writer copies the audited row at write time — the seeding update + // above included — so this is the at-rest copy, read straight from the + // table, and then through the data door an administrator reads it by. + const audit = rowsOf(await ql.find('sys_audit_log', { where: { object_name: 'sys_metadata' }, context: SYSTEM })); + const ours = audit.filter((row) => JSON.stringify(row).includes(MARKER)); + expect(ours.length, 'the ledger recorded the datasource row').toBeGreaterThan(0); + expect(leaked(JSON.stringify(audit))).toEqual([]); + + const door = await call('GET', `/data/sys_audit_log?filter=${encodeURIComponent(JSON.stringify({ object_name: 'sys_metadata' }))}`); + expect(door.status, door.text).toBe(200); + expect(leaked(door.text)).toEqual([]); + }); + + it('4 — the edit door carries a withheld value forward only where the read path still withholds it', async () => { + // A pair's `value` is credential material only while its label names one. + // Seed a legacy row whose pairs sit under plain record keys, read the edit + // form, rename each label to a non-credential name, and save. + const EDIT = { proxy: 'pin-dogfood-proxybearer-4e07', probe: 'pin-dogfood-probekey-b31c' } as const; + const body = { + ...CLEAN_BODY, + origin: 'runtime', + config: { + host: MARKER, + proxyHeader: { name: 'Authorization', value: `Bearer ${EDIT.proxy}` }, + probe: { key: 'X-Api-Key', value: EDIT.probe }, + }, + }; + const ql: any = await stack.kernel.getServiceAsync('objectql'); + for (const row of await stored('sys_metadata')) { + await ql.update('sys_metadata', { metadata: JSON.stringify(body) }, { where: { id: row.id }, context: SYSTEM }); + } + const atRest = async () => JSON.stringify(await stored('sys_metadata')); + // Precondition: both values are at rest. + expect(await atRest()).toContain(EDIT.proxy); + expect(await atRest()).toContain(EDIT.probe); + await stack.stop(); + await boot(); + + const form = JSON.parse((await call('GET', `/datasources/${NAME}`)).text); + const config = structuredClone((form.datasource ?? form.data?.datasource ?? form).config); + expect(config.proxyHeader).toEqual({ name: 'Authorization' }); + config.proxyHeader.name = 'X-Trace'; + config.probe.key = 'Accept'; + const saved = await call('PATCH', `/datasources/${NAME}`, { config }); + expect(saved.status, saved.text).toBe(200); + + const rest = await atRest(); + expect(rest, 'the edit reached the stored row (positive control)').toContain('X-Trace'); + expect(rest).not.toContain(EDIT.proxy); + expect(rest).not.toContain(EDIT.probe); + for (const path of [`/datasources/${NAME}`, `/meta/datasource/${NAME}`]) { + const res = await call('GET', path); + expect.soft(res.status, path).toBe(200); + expect.soft(res.text, path).not.toContain(EDIT.proxy); + expect.soft(res.text, path).not.toContain(EDIT.probe); + } + }, 180_000); +}); diff --git a/packages/services/service-datasource/src/__tests__/datasource-contractless-credentials.test.ts b/packages/services/service-datasource/src/__tests__/datasource-contractless-credentials.test.ts new file mode 100644 index 00000000000..6d197e72e3e --- /dev/null +++ b/packages/services/service-datasource/src/__tests__/datasource-contractless-credentials.test.ts @@ -0,0 +1,340 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The datasource-admin doors for a datasource whose driver the platform ships + * NO config contract for: credential-shaped `config` values (`apiKey`, + * `client_secret`, `secretAccessKey`, `privateKey`, `accessToken`, a + * `Password=` connection-string segment) are refused at create/update, are + * withheld on `getDatasource`, survive an untouched round trip on a legacy row + * stored before the refusal existed, and are reported as residue by the + * credential-migration planner instead of `nothing-to-migrate`. + * + * The derivation itself is pinned in `@objectstack/spec` + * (`data/datasource-contractless-credentials.test.ts`); this file pins that the + * service's doors read it. + */ + +import { describe, it, expect } from 'vitest'; +import { + DatasourceAdminService, + type DatasourceAdminServiceConfig, + type StoredDatasource, +} from '../datasource-admin-service.js'; +import { restoreRedactedConfig } from '../datasource-config-redaction.js'; +import { planCredentialMigration } from '../datasource-credential-migration.js'; + +const DRIVER = 'com.vendor.warehouse'; + +/** A legacy runtime row stored before the write-door refusal existed. */ +const LEGACY: StoredDatasource = { + name: 'warehouse', + driver: DRIVER, + origin: 'runtime', + config: { + host: 'wh.internal', + apiKey: 'sk-live-1', + oauth: { clientId: 'cid', client_secret: 'cs-1' }, + secretAccessKey: 'sak-1', + privateKey: 'pk-1', + accessToken: 'at-1', + connectionString: 'Server=wh;User Id=u;Password=pw-1;Database=d', + servers: [{ host: 'a', password: 'srv-1' }, { host: 'b' }], + headers: [{ name: 'Authorization', value: 'Bearer hdr-1' }, { name: 'Accept', value: 'application/json' }], + }, +}; + +const CLEARTEXT = ['sk-live-1', 'cs-1', 'sak-1', 'pk-1', 'at-1', 'pw-1', 'srv-1', 'hdr-1']; + +function makeService(seed: StoredDatasource[] = []) { + const records: StoredDatasource[] = seed.map((r) => structuredClone(r)); + const ops: string[] = []; + const config: DatasourceAdminServiceConfig = { + probe: async () => ({ ok: true, latencyMs: 1 }), + listDatasourceRecords: async () => records.map((r) => structuredClone(r)), + getDatasourceRecord: async (name) => { + const r = records.find((x) => x.name === name); + return r ? structuredClone(r) : undefined; + }, + putDatasourceRecord: async (record) => { + ops.push(`put:${record.name}`); + const idx = records.findIndex((r) => r.name === record.name); + if (idx >= 0) records[idx] = structuredClone(record); + else records.push(structuredClone(record)); + }, + deleteDatasourceRecord: async () => {}, + writeSecret: async (_input, hint) => { + ops.push('writeSecret'); + return `sys_secret:${hint.name}`; + }, + removeSecret: async () => { + ops.push('removeSecret'); + }, + countBoundObjects: async () => 0, + registerPool: () => {}, + unregisterPool: () => {}, + }; + return { service: new DatasourceAdminService(config), records, ops }; +} + +describe('createDatasource / updateDatasource refuse inline credentials for a contractless driver', () => { + it.each([ + ['apiKey', { apiKey: 'sk-live-1' }], + ['client_secret', { client_secret: 'cs-1' }], + ['secretAccessKey', { secretAccessKey: 'sak-1' }], + ['privateKey', { privateKey: 'pk-1' }], + ['accessToken', { accessToken: 'at-1' }], + ['connectionString', { connectionString: 'Server=wh;Password=pw-1' }], + ['servers.0.password', { servers: [{ host: 'a', password: 'srv-1' }] }], + ['headers.0.value', { headers: [{ name: 'Authorization', value: 'Bearer hdr-1' }] }], + ])('create refuses `config.%s` and persists nothing, no secret minted', async (key, extra) => { + const { service, records, ops } = makeService(); + await expect( + service.createDatasource({ name: 'warehouse', driver: DRIVER, config: { host: 'wh', ...extra } }), + ).rejects.toThrow(`config.${key}`); + expect(records).toHaveLength(0); + expect(ops).toEqual([]); + }); + + it('create accepts a contractless config with no credential material, and a bound secret', async () => { + const { service, records } = makeService(); + await service.createDatasource( + { name: 'warehouse', driver: DRIVER, config: { host: 'wh', accessKeyId: 'AKIA1' } }, + { value: 'sk-live-1' }, + ); + expect(records[0]?.config).toEqual({ host: 'wh', accessKeyId: 'AKIA1' }); + expect(records[0]?.external?.credentialsRef).toBe('sys_secret:warehouse'); + }); + + it('update refuses a NEW inline credential typed into the form', async () => { + const { service, records } = makeService([ + { name: 'warehouse', driver: DRIVER, origin: 'runtime', config: { host: 'wh' } }, + ]); + await expect( + service.updateDatasource('warehouse', { config: { host: 'wh', clientSecret: 'cs-2' } }), + ).rejects.toThrow('config.clientSecret'); + expect(records[0]?.config).toEqual({ host: 'wh' }); + }); +}); + +describe('getDatasource withholds a legacy contractless row\'s credentials', () => { + it('serves the row with every credential-shaped value removed, and names nothing it kept', async () => { + const { service } = makeService([LEGACY]); + const ds = await service.getDatasource('warehouse'); + expect(ds?.config).toEqual({ + host: 'wh.internal', + oauth: { clientId: 'cid' }, + connectionString: 'Server=wh;User Id=u;Database=d', + servers: [{ host: 'a' }, { host: 'b' }], + headers: [{ name: 'Authorization' }, { name: 'Accept', value: 'application/json' }], + }); + const served = JSON.stringify(ds); + for (const secret of CLEARTEXT) expect(served).not.toContain(secret); + }); + + it('listDatasources serves no cleartext either', async () => { + const { service } = makeService([LEGACY]); + const served = JSON.stringify(await service.listDatasources()); + for (const secret of CLEARTEXT) expect(served).not.toContain(secret); + }); +}); + +describe('the edit round trip on a legacy row', () => { + it('restoreRedactedConfig carries every withheld value forward (an untouched Save deletes nothing)', () => { + // Exactly what getDatasource served for the row (pinned above). + const patch = { + host: 'wh.internal', + oauth: { clientId: 'cid' }, + connectionString: 'Server=wh;User Id=u;Database=d', + servers: [{ host: 'a' }, { host: 'b' }], + headers: [{ name: 'Authorization' }, { name: 'Accept', value: 'application/json' }], + }; + expect(restoreRedactedConfig(DRIVER, patch, LEGACY.config)).toEqual(LEGACY.config); + // The patch's own arrays are copied, never mutated. + expect(patch.servers).toEqual([{ host: 'a' }, { host: 'b' }]); + }); + + it('a `[name, value]` header tuple and a credential embedded with a `;` round-trip too', async () => { + const stored: StoredDatasource = { + name: 'tuples', + driver: DRIVER, + origin: 'runtime', + config: { + headers: [['Accept', 'application/json'], ['Authorization', 'Bearer tpl-1']], + libpq: 'host=wh password=lp-1;x dbname=d', + }, + }; + const { service, records } = makeService([stored]); + const read = await service.getDatasource('tuples'); + expect(read!.config).toEqual({ headers: [['Accept', 'application/json'], ['Authorization']], libpq: 'host=wh dbname=d' }); + expect(JSON.stringify(read)).not.toMatch(/tpl-1|lp-1/); + await service.updateDatasource('tuples', { config: read!.config }); + expect(records[0]!.config).toEqual(stored.config); + }); + + it('an author who changed an array element keeps their word; one who removed the array keeps it removed', () => { + const changed = { host: 'wh.internal', servers: [{ host: 'a', password: 'new-1' }, { host: 'b' }] }; + const restored = restoreRedactedConfig(DRIVER, changed, LEGACY.config) as Record; + expect(restored.servers).toEqual([{ host: 'a', password: 'new-1' }, { host: 'b' }]); + const removed = restoreRedactedConfig(DRIVER, { host: 'wh.internal' }, LEGACY.config) as Record; + expect(removed.servers).toBeUndefined(); + }); +}); + +describe('a withheld array value is carried only onto the element it came from', () => { + const STORED = { + servers: [{ host: 'a', password: 'p-a' }, { host: 'b', password: 'p-b' }], + headers: [{ name: 'Authorization', value: 'Bearer h-1' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['Authorization', 'Bearer t-1']], + }; + /** What getDatasource serves for STORED. */ + const SERVED = { + servers: [{ host: 'a' }, { host: 'b' }], + headers: [{ name: 'Authorization' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['Authorization']], + }; + const restore = (patch: Record) => restoreRedactedConfig(DRIVER, patch, STORED) as Record; + + it('control: the served config round-trips to the stored one', () => { + expect(restore(structuredClone(SERVED))).toEqual(STORED); + }); + + it('an element deleted before it: each remaining server keeps ITS OWN password', () => { + expect(restore({ ...structuredClone(SERVED), servers: [{ host: 'b' }] }).servers).toEqual([{ host: 'b', password: 'p-b' }]); + }); + + it('a reorder: every value follows its element', () => { + const out = restore({ + servers: [{ host: 'b' }, { host: 'a' }], + headers: [{ name: 'Accept', value: 'application/json' }, { name: 'Authorization' }], + tuples: [['Authorization'], ['Accept', 'json']], + }); + expect(out).toEqual({ + servers: [{ host: 'b', password: 'p-b' }, { host: 'a', password: 'p-a' }], + headers: [{ name: 'Accept', value: 'application/json' }, { name: 'Authorization', value: 'Bearer h-1' }], + tuples: [['Authorization', 'Bearer t-1'], ['Accept', 'json']], + }); + }); + + it('a renamed header, or an edited sibling field, receives nothing — the value is dropped', () => { + const out = restore({ + servers: [{ host: 'a2' }, { host: 'b' }], + headers: [{ name: 'X-Other' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['X-Other']], + }); + expect(out).toEqual({ + servers: [{ host: 'a2' }, { host: 'b', password: 'p-b' }], + headers: [{ name: 'X-Other' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Accept', 'json'], ['X-Other']], + }); + expect(JSON.stringify(out)).not.toMatch(/p-a|h-1|t-1/); + }); + + it('elements that cannot be told apart receive nothing once the array was edited; untouched, they keep their values', () => { + const stored = { + headers: [ + { name: 'Authorization', value: 'v-1' }, + { name: 'Authorization', value: 'v-2' }, + { name: 'Accept', value: 'json' }, + ], + }; + const served = { headers: [{ name: 'Authorization' }, { name: 'Authorization' }, { name: 'Accept', value: 'json' }] }; + expect(restoreRedactedConfig(DRIVER, structuredClone(served), stored)).toEqual(stored); + const edited = restoreRedactedConfig(DRIVER, { headers: [{ name: 'Authorization' }, { name: 'Authorization' }] }, stored); + expect(edited).toEqual({ headers: [{ name: 'Authorization' }, { name: 'Authorization' }] }); + }); + + it('a raw-headers list: its withheld value comes back only into the untouched list', () => { + const stored = { headers: ['Authorization', 'Bearer r-1', 'Accept', 'json'] }; + expect(restoreRedactedConfig(DRIVER, { headers: ['Authorization', null, 'Accept', 'json'] }, stored)).toEqual(stored); + expect(restoreRedactedConfig(DRIVER, { headers: ['Authorization', null, 'Accept', 'xml'] }, stored)).toEqual({ + headers: ['Authorization', null, 'Accept', 'xml'], + }); + }); + + it('through the service: deleting the first server keeps the second server\'s password on it', async () => { + const { service, records } = makeService([{ name: 'w', driver: DRIVER, origin: 'runtime', config: structuredClone(STORED) }]); + const read = await service.getDatasource('w'); + const config = structuredClone(read!.config) as Record; + config.servers = (config.servers as unknown[]).slice(1); + await service.updateDatasource('w', { config }); + expect(records[0]!.config?.servers).toEqual([{ host: 'b', password: 'p-b' }]); + }); +}); + +describe('a value is carried forward only where the read path would still withhold it', () => { + // A pair's `value` is credential material only while a label names a + // credential, so an edit to the LABEL changes whether the value beside it is + // withheld. The pair sits under a plain (non-credential) record key, so the + // array identity rule does not apply and only the re-judgment can decide. + const STORED = { + host: 'wh.internal', + proxyHeader: { name: 'Authorization', value: 'Bearer px-1' }, + probe: { key: 'X-Api-Key', value: 'pk-2' }, + }; + const SERVED = { host: 'wh.internal', proxyHeader: { name: 'Authorization' }, probe: { key: 'X-Api-Key' } }; + const restore = (patch: Record) => restoreRedactedConfig(DRIVER, patch, STORED) as Record; + + it('control: an untouched Save carries both values forward', () => { + expect(restore(structuredClone(SERVED))).toEqual(STORED); + }); + + it('a label renamed to a non-credential name: the value is dropped, not served', () => { + const out = restore({ ...structuredClone(SERVED), proxyHeader: { name: 'X-Trace' } }); + expect(out.proxyHeader).toEqual({ name: 'X-Trace' }); + expect(out.probe).toEqual(STORED.probe); + expect(JSON.stringify(out)).not.toContain('px-1'); + }); + + it('a label deleted: the value is dropped, not served', () => { + const out = restore({ ...structuredClone(SERVED), proxyHeader: {} }); + expect(out.proxyHeader).toEqual({}); + expect(JSON.stringify(out)).not.toContain('px-1'); + }); + + it('a `key:` label renamed: the value is dropped, not served', () => { + const out = restore({ ...structuredClone(SERVED), probe: { key: 'Accept' } }); + expect(out.probe).toEqual({ key: 'Accept' }); + expect(out.proxyHeader).toEqual(STORED.proxyHeader); + expect(JSON.stringify(out)).not.toContain('pk-2'); + }); + + it('a label renamed to ANOTHER credential name still carries the value (it stays withheld)', () => { + const out = restore({ ...structuredClone(SERVED), proxyHeader: { name: 'Proxy-Authorization' } }); + expect(out.proxyHeader).toEqual({ name: 'Proxy-Authorization', value: 'Bearer px-1' }); + }); + + it('whatever survives the restore is withheld again on the next read', async () => { + const { service } = makeService([{ name: 'w', driver: DRIVER, origin: 'runtime', config: structuredClone(STORED) }]); + const read = await service.getDatasource('w'); + const config = structuredClone(read!.config) as Record; + config.proxyHeader = { name: 'X-Trace' }; + config.probe = {}; + await service.updateDatasource('w', { config }); + const again = await service.getDatasource('w'); + expect(JSON.stringify(again)).not.toMatch(/px-1|pk-2/); + }); +}); + +describe('planCredentialMigration names a contractless row\'s credentials as residue', () => { + it('refuses with a remedy instead of reporting nothing-to-migrate', () => { + const plan = planCredentialMigration({ + name: 'warehouse', + driver: DRIVER, + origin: 'runtime', + config: { host: 'wh', apiKey: 'sk-live-1', connectionString: 'Server=wh;Password=pw-1' }, + }); + expect(plan.action).toBe('refuse'); + expect(plan.action === 'refuse' ? plan.reason : '').toContain('config.apiKey'); + expect(plan.action === 'refuse' ? plan.reason : '').toContain('config.connectionString'); + }); + + it('control: a contractless row with no credential material has nothing to migrate', () => { + const plan = planCredentialMigration({ + name: 'warehouse', + driver: DRIVER, + origin: 'runtime', + config: { host: 'wh', accessKeyId: 'AKIA1' }, + }); + expect(plan).toEqual({ action: 'none', status: 'nothing-to-migrate', remaining: [] }); + }); +}); diff --git a/packages/services/service-datasource/src/datasource-config-redaction.ts b/packages/services/service-datasource/src/datasource-config-redaction.ts index 368c2c80a32..45dcb32f55a 100644 --- a/packages/services/service-datasource/src/datasource-config-redaction.ts +++ b/packages/services/service-datasource/src/datasource-config-redaction.ts @@ -71,6 +71,43 @@ export { * consumes the redactor's exact `redactedPaths` segments, so a stored key * with a literal dot cannot be mis-split.) * + * ## Array positions: carried only onto an element that is provably the same + * + * The contractless-driver redaction withholds credentials inside array + * elements (`servers.0.password`, `headers.1.value`, `headers.0.1`), and a + * position is not an identity: after the author deletes, reorders or renames + * an element, the same index names a different element, and grafting by index + * would put a credential onto it — under another header name, onto another + * server. So every ARRAY hop of a redacted path is resolved against the patch + * by identity, never by index alone: + * + * - the patch's array equals the served array (nothing in it was touched) ⇒ + * the same index; + * - otherwise the served element — the element as the read path served it, + * every non-credential sibling of the withheld value included — must occur + * exactly ONCE in the served array and exactly once in the patch's array, + * and the value is carried onto that one element wherever it now sits + * (a reorder, or a sibling deleted before it); + * - anything else — the element changed (a header renamed, a sibling field + * edited), it is gone, or it cannot be told apart from another — and the + * withheld value is DROPPED: the author changed the element that held it, + * which is their word about it, and guessing would misbind it. + * + * A withheld value that IS an array element (a header tuple's value, a + * raw-headers list's value) is carried only when the patch's array equals the + * served array. + * + * ## Only where the read path would still withhold it + * + * Whether a position is withheld can depend on its siblings: the `value` of a + * `{ name, value }` pair is credential material only while a label names a + * credential. A value carried forward beside an EDITED sibling (the label + * renamed, deleted, or moved to another label key) could therefore land where + * the read path would serve it. So the grafted config is redacted again, and a + * graft survives only when the redaction still withholds its landing path — + * repeated until nothing more drops. A dropped graft leaves the patch as the + * author sent it there. + * * What this does NOT do is let a patch set a refused key: `assertValidConfig` * still runs on the merged record, so a caller that types `password` into the * config gets #8078's refusal exactly as it would without this function. @@ -84,35 +121,151 @@ export function restoreRedactedConfig( if (!stored || typeof stored !== 'object') return patch; const served = redactDatasourceConfig(driver, stored); - const out: Record = { ...patch }; - + const grafts: Array<{ landing: string[]; value: unknown }> = []; for (const path of served.redactedPaths) { const storedLeaf = valueAt(stored, path); if (storedLeaf === undefined) continue; - const parentPath = path.slice(0, -1); - const leafKey = path[path.length - 1] as string; - const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath); - if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue; + const landing = landingPath(served.config, patch, path); + if (landing) grafts.push({ landing, value: storedLeaf }); + } + if (grafts.length === 0) return patch; + + const graftAll = (kept: readonly { landing: string[]; value: unknown }[]): Record => { + const out: Record = { ...patch }; + for (const graft of kept) graftAt(out, graft.landing, graft.value); + return out; + }; + + // An untouched Save — the patch IS the served projection — grafts every + // withheld value back onto exactly what it was withheld from, so the merged + // config is the stored one and the read path withholds the same positions: + // no second walk is owed. + if (grafts.length === served.redactedPaths.length && sameValue(patch, served.config)) return graftAll(grafts); + + // Otherwise keep only what the read path would STILL withhold where it + // lands. The judgment of a position can depend on its siblings — a pair's + // `value` is a credential only while a label names one — so a value carried + // under an edited sibling may land where the read path would serve it. + // Each pass re-grafts the survivors onto the untouched patch and drops every + // graft the merged config's redaction no longer withholds AT its landing + // path; the set only shrinks, so this settles within one pass per graft. + // Same loop as the `/meta` carry-forward's (#20590). + let kept = grafts; + for (;;) { + const out = graftAll(kept); + const withheld = new Set(redactDatasourceConfig(driver, out).redactedPaths.map(pathKey)); + const next = kept.filter((graft) => withheld.has(pathKey(graft.landing))); + if (next.length === kept.length) return next.length === 0 ? patch : out; + kept = next; + } +} + +/** A path as one comparable string (segments may hold any character, so JSON, not a join). */ +const pathKey = (path: readonly string[]): string => JSON.stringify(path); + +/** An array position, as the redactor spells it in a path (its decimal index). */ +const isIndex = (segment: string): boolean => /^(0|[1-9][0-9]*)$/.test(segment); + +const isRecord = (value: unknown): value is Record => + !!value && typeof value === 'object' && !Array.isArray(value); + +/** + * Where in `patch` the stored value at `path` (a path into the stored and the + * served config) lands, or `undefined` when the patch does not speak to it as + * the read path served it — see {@link restoreRedactedConfig}. Judged on the + * patch as the caller sent it, so one graft never changes another's answer. + */ +function landingPath( + servedConfig: Record, + patch: Record, + path: readonly string[], +): string[] | undefined { + const landing: string[] = []; + let servedNode: unknown = servedConfig; + let patchNode: unknown = patch; + for (let i = 0; i < path.length - 1; i += 1) { + const segment = path[i] as string; + if (Array.isArray(servedNode)) { + if (!Array.isArray(patchNode) || !isIndex(segment)) return undefined; + const index = matchingElement(servedNode, patchNode, Number(segment)); + if (index === undefined) return undefined; + landing.push(String(index)); + servedNode = servedNode[Number(segment)]; + patchNode = patchNode[index]; + continue; + } + // A patch whose CONTAINER is gone (or is no longer a container of this + // kind) is the author's word: nothing is grafted there. + if (!isRecord(servedNode) || !isRecord(patchNode)) return undefined; + landing.push(segment); + servedNode = servedNode[segment]; + patchNode = patchNode[segment]; + } + const leaf = path[path.length - 1] as string; + if (Array.isArray(servedNode)) { + // The withheld value is itself an element: carried only into an untouched array. + if (!isIndex(leaf) || !Array.isArray(patchNode) || !sameValue(servedNode, patchNode)) return undefined; + } else { + if (!isRecord(servedNode) || !isRecord(patchNode)) return undefined; // What the read path served at this position: `undefined` for a dropped - // key, the rewritten string for a URL redaction. The patch speaks for the - // author exactly where it DIFFERS from that projection. - const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath); - const servedLeaf = - servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent) - ? (servedParent as Record)[leafKey] - : undefined; - if ((patchParent as Record)[leafKey] !== servedLeaf) continue; - graftAt(out, path, storedLeaf); + // key, the rewritten string for an embedded-credential redaction. The + // patch speaks for the author exactly where it DIFFERS from that projection. + if (!sameValue(patchNode[leaf], servedNode[leaf])) return undefined; } + landing.push(leaf); + return landing; +} - return out; +/** + * The index in `patchArray` of the element served at `servedIndex`, by + * identity: the same index when the arrays are equal, else the one element + * equal to the served one when it is unique on both sides; `undefined` when + * no single element answers. + */ +function matchingElement(servedArray: readonly unknown[], patchArray: readonly unknown[], servedIndex: number): number | undefined { + if (servedIndex >= servedArray.length) return undefined; + if (sameValue(servedArray, patchArray)) return servedIndex; + const servedElement = servedArray[servedIndex]; + if (servedArray.filter((element) => sameValue(element, servedElement)).length !== 1) return undefined; + let found: number | undefined; + for (let index = 0; index < patchArray.length; index += 1) { + if (!sameValue(patchArray[index], servedElement)) continue; + if (found !== undefined) return undefined; + found = index; + } + return found; } -/** The value at `path` inside a record-ish value, or `undefined` off the walk. */ +/** Structural equality over JSON-shaped values (and bytes). */ +function sameValue(a: unknown, b: unknown): boolean { + if (a === b) return true; + if (a === null || b === null || typeof a !== 'object' || typeof b !== 'object') { + return Number.isNaN(a as number) && Number.isNaN(b as number); + } + if (ArrayBuffer.isView(a) || ArrayBuffer.isView(b)) { + if (!ArrayBuffer.isView(a) || !ArrayBuffer.isView(b) || a.byteLength !== b.byteLength) return false; + const x = new Uint8Array(a.buffer, a.byteOffset, a.byteLength); + const y = new Uint8Array(b.buffer, b.byteOffset, b.byteLength); + return x.every((byte, i) => byte === y[i]); + } + if (Array.isArray(a) !== Array.isArray(b)) return false; + if (Array.isArray(a) && Array.isArray(b)) return a.length === b.length && a.every((v, i) => sameValue(v, b[i])); + const ao = a as Record; + const bo = b as Record; + const ak = Object.keys(ao); + if (ak.length !== Object.keys(bo).length) return false; + return ak.every((k) => Object.prototype.hasOwnProperty.call(bo, k) && sameValue(ao[k], bo[k])); +} + +/** + * The value at `path` inside a record-ish value, or `undefined` off the walk. + * An array is entered only at an index segment. + */ function valueAt(value: unknown, path: readonly string[]): unknown { let node: unknown = value; for (const segment of path) { - if (!node || typeof node !== 'object' || Array.isArray(node)) return undefined; + if (!node || typeof node !== 'object') return undefined; + if (Array.isArray(node) && !isIndex(segment)) return undefined; node = (node as Record)[segment]; } return node; @@ -122,14 +275,15 @@ function valueAt(value: unknown, path: readonly string[]): unknown { * Set `path` to `value` inside `out`, copying every container along the spine * so the caller's `{ ...patch }` shallow copy never aliases a mutation back * into the patch object the caller handed us. Every intermediate container is - * known to exist and be a record — the caller checked before grafting. + * known to exist — {@link landingPath} walked it in the patch. */ function graftAt(out: Record, path: readonly string[], value: unknown): void { - let node = out; + let node: Record = out; for (const segment of path.slice(0, -1)) { - const child = { ...(node[segment] as Record) }; + const current = node[segment]; + const child = Array.isArray(current) ? [...current] : { ...(current as Record) }; node[segment] = child; - node = child; + node = child as unknown as Record; } node[path[path.length - 1] as string] = value; } diff --git a/packages/services/service-datasource/src/datasource-credential-migration.ts b/packages/services/service-datasource/src/datasource-credential-migration.ts index bacf45e61c1..8ece420647f 100644 --- a/packages/services/service-datasource/src/datasource-credential-migration.ts +++ b/packages/services/service-datasource/src/datasource-credential-migration.ts @@ -71,6 +71,8 @@ */ import { + findContractlessCredentials, + isContractlessDriver, redactableConfigKeys, redactUrlCredentials, refusedCredentialKeys, @@ -145,7 +147,21 @@ function unbindableCredentialKeys( bindable: ReadonlySet, ): string[] { const present = new Set(stringValued(config).map(([key]) => key)); - return redactableConfigKeys(driver) + const named = new Set(redactableConfigKeys(driver)); + // A driver the platform ships no contract for is judged by the read path's + // own walk (`findContractlessCredentials`): every top-level key holding a + // finding is residue here too — never `nothing-to-migrate` for a row whose + // credential sits cleartext at rest. + if (isContractlessDriver(driver)) { + for (const finding of findContractlessCredentials(config)) { + const top = finding.path[0]; + if (top !== undefined) { + named.add(top); + present.add(top); + } + } + } + return [...named] .filter((key) => present.has(key) && !bindable.has(key)) .sort(); } diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 1f7f04170c2..f9a5c52c0c1 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -91,6 +91,7 @@ "CONTEXT_TOKEN_DESCRIPTIONS (const)", "CONTEXT_TOKEN_SUGGESTIONS (const)", "CONTEXT_TOKEN_WRAPPED_RE (const)", + "CONTRACTLESS_CREDENTIAL_WALK_DEPTH (const)", "CREDENTIAL_KEY_SPELLINGS (const)", "CREDENTIAL_URL_QUERY_PARAMS (const)", "CREDENTIAL_URL_QUERY_PARAM_NAMES (const)", @@ -118,6 +119,7 @@ "ContextTokenPlaceholder (type)", "ContextTokenPlaceholderSchema (const)", "ContextTokenSchema (const)", + "ContractlessCredentialFinding (interface)", "CrossFieldColumnVerdict (type)", "CrossFieldComparisonClass (type)", "CrossFieldComparisonFieldMeta (interface)", @@ -260,6 +262,7 @@ "ESignatureConfigParsed (type)", "ESignatureConfigSchema (const)", "EffectiveApiMethods (interface)", + "EmbeddedCredentialOptions (interface)", "EmptyOperatorArm (type)", "EmptyOperatorExpansion (interface)", "EnableLike (interface)", @@ -441,6 +444,7 @@ "MANAGED_WRITE_VERB_AFFORDANCE (const)", "MASKED_ON_READ_FIELD_TYPES (const)", "MAX_BULK_PER_ROW_HOOK_ROWS (const)", + "MAX_JUDGED_STRING_LENGTH (const)", "MEASURE_FIELD_TYPES (const)", "MONGO_OPTIONS_CREDENTIAL_PATHS (const)", "MULTI_CAPABLE_TYPES (const)", @@ -813,6 +817,7 @@ "checkManagedApiMethodAffordances (function)", "classifyDottedFilterHead (function)", "classifyFilterToken (function)", + "connectionStringCredentialKeys (function)", "containsUnresolvedPlaceholder (function)", "countAuthorableFields (function)", "credentialFreeMongoOptions (function)", @@ -840,10 +845,12 @@ "driverHasLocalDefault (function)", "driverSupportsTransactions (function)", "effectiveOperationsArray (function)", + "embeddedCredentialOf (function)", "emptyGroupValueFor (function)", "expandEmptyOperator (function)", "fieldForm (const)", "filterSubtreeProvenanceOf (function)", + "findContractlessCredentials (function)", "foldAsciiCase (function)", "foldQueryAliasSlots (function)", "formatUnknownAuthoringKey (function)", @@ -871,6 +878,8 @@ "isAppResolvedDefaultToken (function)", "isCompatible (function)", "isContextToken (function)", + "isContractlessDriver (function)", + "isCredentialShapedConfigKey (function)", "isCurrentUserDefaultToken (function)", "isDateMacroToken (function)", "isDateRangePresetName (function)", @@ -904,6 +913,7 @@ "likePatternToRegExp (function)", "likePatternToRegexSource (function)", "lintAuthoredRecordKeys (function)", + "looksLikeSecretValue (function)", "lowerFilterCondition (function)", "markFilterSubtreeProvenance (function)", "matchesLikePattern (function)", @@ -930,6 +940,7 @@ "readBooleanComparand (function)", "readNumericString (function)", "redactDatasourceConfig (function)", + "redactEmbeddedCredentials (function)", "redactUrlCredentialQueryParams (function)", "redactUrlCredentials (function)", "redactUrlPassword (function)", @@ -973,6 +984,7 @@ "utcInstantMs (function)", "validateDriverConfig (function)", "valueRoundTripDivergence (function)", - "valueSchemaFor (function)" + "valueSchemaFor (function)", + "withholdContractlessCredentials (function)" ] } diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index 68e506aa24a..47da62afd38 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -88,6 +88,7 @@ "CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)", "CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)", "CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)", + "CONTRACTLESS_CREDENTIAL_WALK_DEPTH": "src/data/driver/contractless-credentials.ts#CONTRACTLESS_CREDENTIAL_WALK_DEPTH (const)", "CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)", "CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)", "CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)", @@ -115,6 +116,7 @@ "ContextTokenPlaceholder": "src/data/context-tokens.zod.ts#ContextTokenPlaceholder (type)", "ContextTokenPlaceholderSchema": "src/data/context-tokens.zod.ts#ContextTokenPlaceholderSchema (const)", "ContextTokenSchema": "src/data/context-tokens.zod.ts#ContextTokenSchema (const)", + "ContractlessCredentialFinding": "src/data/driver/contractless-credentials.ts#ContractlessCredentialFinding (interface)", "CrossFieldColumnVerdict": "src/data/filter-cross-field-comparison-class.ts#CrossFieldColumnVerdict (type)", "CrossFieldComparisonClass": "src/data/filter-cross-field-comparison-class.ts#CrossFieldComparisonClass (type)", "CrossFieldComparisonFieldMeta": "src/data/filter-cross-field-comparison-class.ts#CrossFieldComparisonFieldMeta (interface)", @@ -255,6 +257,7 @@ "ESignatureConfigParsed": "src/data/document.zod.ts#ESignatureConfigParsed (type)", "ESignatureConfigSchema": "src/data/document.zod.ts#ESignatureConfigSchema (const)", "EffectiveApiMethods": "src/data/api-derivation.ts#EffectiveApiMethods (interface)", + "EmbeddedCredentialOptions": "src/data/driver/contractless-credentials.ts#EmbeddedCredentialOptions (interface)", "EmptyOperatorArm": "src/data/filter-empty-operator.ts#EmptyOperatorArm (type)", "EmptyOperatorExpansion": "src/data/filter-empty-operator.ts#EmptyOperatorExpansion (interface)", "EnableLike": "src/data/api-derivation.ts#EnableLike (interface)", @@ -431,6 +434,7 @@ "MANAGED_WRITE_VERB_AFFORDANCE": "src/data/managed-api-affordance.ts#MANAGED_WRITE_VERB_AFFORDANCE (const)", "MASKED_ON_READ_FIELD_TYPES": "src/data/masked-field-types.ts#MASKED_ON_READ_FIELD_TYPES (const)", "MAX_BULK_PER_ROW_HOOK_ROWS": "src/data/bulk-write-hook-conformance.ts#MAX_BULK_PER_ROW_HOOK_ROWS (const)", + "MAX_JUDGED_STRING_LENGTH": "src/data/driver/contractless-credentials.ts#MAX_JUDGED_STRING_LENGTH (const)", "MEASURE_FIELD_TYPES": "src/data/aggregation-policy.ts#MEASURE_FIELD_TYPES (const)", "MONGO_OPTIONS_CREDENTIAL_PATHS": "src/data/driver/common.zod.ts#MONGO_OPTIONS_CREDENTIAL_PATHS (const)", "MULTI_CAPABLE_TYPES": "src/data/field-value.zod.ts#MULTI_CAPABLE_TYPES (const)", @@ -800,6 +804,7 @@ "checkManagedApiMethodAffordances": "src/data/managed-api-affordance.ts#checkManagedApiMethodAffordances (function)", "classifyDottedFilterHead": "src/data/filter-dotted-head.ts#classifyDottedFilterHead (function)", "classifyFilterToken": "src/data/context-tokens.zod.ts#classifyFilterToken (function)", + "connectionStringCredentialKeys": "src/data/driver/contractless-credentials.ts#connectionStringCredentialKeys (function)", "containsUnresolvedPlaceholder": "src/data/driver/common.zod.ts#containsUnresolvedPlaceholder (function)", "countAuthorableFields": "src/data/record-surface.ts#countAuthorableFields (function)", "credentialFreeMongoOptions": "src/data/driver/common.zod.ts#credentialFreeMongoOptions (function)", @@ -827,10 +832,12 @@ "driverHasLocalDefault": "src/data/driver/config-registry.zod.ts#driverHasLocalDefault (function)", "driverSupportsTransactions": "src/data/driver.zod.ts#driverSupportsTransactions (function)", "effectiveOperationsArray": "src/data/api-derivation.ts#effectiveOperationsArray (function)", + "embeddedCredentialOf": "src/data/driver/contractless-credentials.ts#embeddedCredentialOf (function)", "emptyGroupValueFor": "src/data/aggregation-policy.ts#emptyGroupValueFor (function)", "expandEmptyOperator": "src/data/filter-empty-operator.ts#expandEmptyOperator (function)", "fieldForm": "src/data/field.form.ts#fieldForm (const)", "filterSubtreeProvenanceOf": "src/data/filter-subtree-provenance.ts#filterSubtreeProvenanceOf (function)", + "findContractlessCredentials": "src/data/driver/contractless-credentials.ts#findContractlessCredentials (function)", "foldAsciiCase": "src/data/filter.zod.ts#foldAsciiCase (function)", "foldQueryAliasSlots": "src/data/data-engine.zod.ts#foldQueryAliasSlots (function)", "formatUnknownAuthoringKey": "src/data/authoring-key-lint.ts#formatUnknownAuthoringKey (function)", @@ -858,6 +865,8 @@ "isAppResolvedDefaultToken": "src/data/default-value-tokens.ts#isAppResolvedDefaultToken (function)", "isCompatible": "src/data/type-compat.ts#isCompatible (function)", "isContextToken": "src/data/context-tokens.zod.ts#isContextToken (function)", + "isContractlessDriver": "src/data/datasource-credential-redaction.ts#isContractlessDriver (function)", + "isCredentialShapedConfigKey": "src/data/driver/contractless-credentials.ts#isCredentialShapedConfigKey (function)", "isCurrentUserDefaultToken": "src/data/default-value-tokens.ts#isCurrentUserDefaultToken (function)", "isDateMacroToken": "src/data/date-macros.zod.ts#isDateMacroToken (function)", "isDateRangePresetName": "src/data/date-range-presets.ts#isDateRangePresetName (function)", @@ -891,6 +900,7 @@ "likePatternToRegExp": "src/data/filter.zod.ts#likePatternToRegExp (function)", "likePatternToRegexSource": "src/data/filter.zod.ts#likePatternToRegexSource (function)", "lintAuthoredRecordKeys": "src/data/authoring-key-lint.ts#lintAuthoredRecordKeys (function)", + "looksLikeSecretValue": "src/data/driver/contractless-credentials.ts#looksLikeSecretValue (function)", "lowerFilterCondition": "src/data/filter-lowering.ts#lowerFilterCondition (function)", "markFilterSubtreeProvenance": "src/data/filter-subtree-provenance.ts#markFilterSubtreeProvenance (function)", "matchesLikePattern": "src/data/filter.zod.ts#matchesLikePattern (function)", @@ -917,6 +927,7 @@ "readBooleanComparand": "src/data/filter-boolean-comparand-declared-type.ts#readBooleanComparand (function)", "readNumericString": "src/data/filter-number-comparand-declared-type.ts#readNumericString (function)", "redactDatasourceConfig": "src/data/datasource-credential-redaction.ts#redactDatasourceConfig (function)", + "redactEmbeddedCredentials": "src/data/driver/contractless-credentials.ts#redactEmbeddedCredentials (function)", "redactUrlCredentialQueryParams": "src/data/datasource-credential-redaction.ts#redactUrlCredentialQueryParams (function)", "redactUrlCredentials": "src/data/datasource-credential-redaction.ts#redactUrlCredentials (function)", "redactUrlPassword": "src/data/datasource-credential-redaction.ts#redactUrlPassword (function)", @@ -960,6 +971,7 @@ "utcInstantMs": "src/data/calendar-day.ts#utcInstantMs (function)", "validateDriverConfig": "src/data/driver/config-registry.zod.ts#validateDriverConfig (function)", "valueRoundTripDivergence": "src/data/value-roundtrip-conformance.ts#valueRoundTripDivergence (function)", - "valueSchemaFor": "src/data/field-value.zod.ts#valueSchemaFor (function)" + "valueSchemaFor": "src/data/field-value.zod.ts#valueSchemaFor (function)", + "withholdContractlessCredentials": "src/data/driver/contractless-credentials.ts#withholdContractlessCredentials (function)" } } diff --git a/packages/spec/src/data/datasource-contractless-credentials.test.ts b/packages/spec/src/data/datasource-contractless-credentials.test.ts new file mode 100644 index 00000000000..551773529f6 --- /dev/null +++ b/packages/spec/src/data/datasource-contractless-credentials.test.ts @@ -0,0 +1,874 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Credential material in the `config` of a datasource whose driver the + * platform ships NO contract for. + * + * One walk (`findContractlessCredentials`, `driver/contractless-credentials.ts`) + * decides for both doors: the write door refuses every position it reports, + * the read door withholds every one. Each block pins a judgment in both + * directions; the last ones pin that a driver WITH a contract is judged + * exactly as before. + */ + +import { describe, expect, it } from 'vitest'; + +import { + connectionStringCredentialKeys, + embeddedCredentialOf, + findContractlessCredentials, + getDriverConfigSchema, + isCredentialShapedConfigKey, + looksLikeSecretValue, + redactEmbeddedCredentials, +} from './driver/index'; +import { isContractlessDriver, redactDatasourceConfig } from './datasource-credential-redaction'; +import { DatasourceSchema } from './datasource.zod'; + +const DRIVER = 'com.vendor.warehouse'; + +/** Credential-shaped key spellings, one or more per rule. */ +const CREDENTIAL_SHAPED = [ + // the canonical list and its former aliases + 'password', 'authToken', 'passwd', 'pwd', 'token', 'jwt', 'auth_token', 'authtoken', + // words that count anywhere + 'dbPassword', 'PASSWORD', 'proxy_passwd', 'sslPassphrase', 'clientSecret', 'client_secret', + 'Client-Secret', 'secretAccessKey', 'webhookSecret', 'secret', 'credentials', 'serviceCredential', + // head nouns + 'accessToken', 'refresh_token', 'sessionToken', 'bearerToken', 'dbPass', 'dbPw', 'dbPwd', 'githubPat', + 'sessionCookie', 'cookie', 'sas', 'sasToken', 'auth', 'basicAuth', 'Authorization', 'bearer', + // key material + 'key', 'apiKey', 'api_key', 'APIKEY', 'X-API-Key', 'privateKey', 'private_key', 'signingKey', 'masterKey', + 'encryptionKey', 'accountKey', 'sharedAccessKey', 'subscriptionKey', 'hmacKey', 'serviceAccountKey', + 'serviceAccountJson', 'sharedAccessSignature', + // a trailing plural or qualifier + 'apiKeys', 'tokens', 'passwords', 'privateKeyPem', 'apiKeyValue', 'tokenValue', 'keyJson', 'clientSecretValue', + // one-word stems, header names, folded compounds + 'pass', 'pw', 'pat', 'authorization', 'x-auth-token', 'Proxy-Authorization', 'set-cookie', 'service_account_json', + 'dbpassword', 'secretaccesskey', 'passwordhash', +] as const; + +/** Spellings that must NOT be judged credential-shaped. */ +const NOT_CREDENTIAL_SHAPED = [ + // ordinary connection configuration + 'host', 'port', 'database', 'username', 'user', 'region', 'warehouse', 'schema', 'ssl', 'oauth', 'pattern', + // `…key` that is not key material, and the identity halves of a pair + 'primaryKey', 'partitionKey', 'sortKey', 'cacheKey', 'idempotencyKey', 'accessKey', 'accessKeyId', 'clientId', + 'tenantId', + // words that merely contain a stem + 'passive', 'bypass', 'bypassCache', 'passThrough', 'cookieDomain', 'tokenTtl', 'compass', 'author', 'authorName', + 'tokenizer', 'keyspace', 'secretary', 'credentialing', + // a stem already ending in `s` takes no plural `s` + 'sass', 'compileSass', + // references, locators, identifiers, descriptors + 'credentialsRef', 'secretArn', 'secretName', 'passwordFile', 'privateKeyPath', 'tokenUrl', 'tokenEndpoint', + 'passwordEnv', 'tokenType', 'apiKeyHeader', 'tokenPrefix', 'clientSecretId', 'secretsManagerRegion', + 'credentialProvider', 'credentialSource', 'credentialChain', 'passwordPolicy', 'authMethod', 'passwordEnabled', + 'passwordAuthentication', 'passwordless', + // flags and measures + 'useDefaultCredentials', 'usePassword', 'requirePassword', 'maxTokens', + '', +] as const; + +describe('isCredentialShapedConfigKey — the one name judgment both doors read', () => { + it.each(CREDENTIAL_SHAPED)('%s is credential-shaped', (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(true); + }); + + it.each(NOT_CREDENTIAL_SHAPED)('%j is not credential-shaped', (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(false); + }); +}); + +describe('embedded credentials in a string value', () => { + it.each([ + ['URL userinfo', 'postgresql://u:p@h/db'], + ['URL query parameter by name', 'https://h/x?api_key=k'], + ['URL `;key=value` tail', 'sqlserver://h:1433;user=u;password=p'], + ['semicolon connection string', 'Server=h;User Id=u;Password=p;Database=d'], + ['Azure account key segment', 'AccountName=a;AccountKey=k==;EndpointSuffix=x'], + ['libpq keyword/value', 'host=h port=5432 password=p'], + ['libpq quoted value', "host=h password='a b'"], + ['scheme-less userinfo', 'u:p@h/db'], + ['scheme-relative userinfo', '//u:p@h/db'], + ['stacked-scheme userinfo', 'jdbc:postgresql://u:p@h/db'], + ['Oracle thin userinfo', 'jdbc:oracle:thin:scott/tiger@//h:1521/svc'], + ['libpq unquoted `;` in a password', 'host=h password=a;b'], + ['URL userinfo with `;` in the password', 'sqlserver://u:a;b=c@h/db'], + ['URL tail property whose value holds `@`', 'sqlserver://h;password=a@b;databaseName=d'], + ['URL tail property whose value holds `:` and `@`', 'sqlserver://h;password=a:b@c'], + ['query pair holding `;`', 'https://h/x?token=a;b'], + ['query pair whose `;` run carries a credential', 'https://h/x?mode=ro;password=p'], + ])('finds %s', (_label, value) => { + expect(embeddedCredentialOf(value)).toBeDefined(); + }); + + it.each([ + ['a plain URL', 'https://u@h/x?mode=ro'], + ['an empty URL password', 'https://u:@h/x'], + ['a URL tail with no credential', 'sqlserver://h;databaseName=d'], + ['a connection string with no credential', 'Server=h;User Id=u;Database=d;'], + ['an empty password segment', 'Server=h;Password=;Database=d'], + ['libpq with no credential', 'host=h port=5432 dbname=d'], + ['an email address', 'ops@example.com'], + ['a label', 'just a label'], + ['a stacked-scheme URL with no userinfo', 'jdbc:postgresql://h/db?ssl=true'], + ['an Oracle thin URL with no userinfo', 'jdbc:oracle:thin:@//h:1521/svc'], + ['a scheme-relative URL with no userinfo', '//h/db'], + ])('finds nothing in %s', (_label, value) => { + expect(embeddedCredentialOf(value)).toBeUndefined(); + expect(redactEmbeddedCredentials(value)).toBe(value); + }); + + it('names the credential segment keys of each connection-string form', () => { + expect(connectionStringCredentialKeys('Server=h;Password=p')).toEqual(['Password']); + expect(connectionStringCredentialKeys('host=h password=p')).toEqual(['password']); + expect(connectionStringCredentialKeys('sqlserver://h;user=u;password=p')).toEqual(['password']); + }); + + it('strips exactly the credential, keeping the rest', () => { + expect(redactEmbeddedCredentials('Server=h;User Id=u;Password=p;Database=d')).toBe('Server=h;User Id=u;Database=d'); + expect(redactEmbeddedCredentials('sqlserver://u:x@h:1433;user=u;password=p;databaseName=d')).toBe( + 'sqlserver://u@h:1433;user=u;databaseName=d', + ); + expect(redactEmbeddedCredentials('host=h password=p dbname=d')).toBe('host=h dbname=d'); + expect(redactEmbeddedCredentials("host=h password='a b' dbname=d")).toBe('host=h dbname=d'); + expect(redactEmbeddedCredentials('u:p@h/db')).toBe('u@h/db'); + expect(redactEmbeddedCredentials('https://h/x?access_token=t&mode=ro#f')).toBe('https://h/x?mode=ro#f'); + }); + + it('a quoted value is one segment even when it contains `;`', () => { + expect(redactEmbeddedCredentials('Server=h;Password="a;b""c";Database=d')).toBe('Server=h;Database=d'); + expect(redactEmbeddedCredentials('Driver={x};PWD={a;}}b};Database=d')).toBe('Driver={x};Database=d'); + }); + + it.each([ + ['Server=h;Password=SEK;RIT=a;b;Database=d', 'Server=h;Database=d'], + ['Password=SEK;RIT;Server=h', 'Server=h'], + ['host=h password=SEK;RIT dbname=d', 'host=h dbname=d'], + ["host=h password='SEK RIT' dbname=d", 'host=h dbname=d'], + ['sqlserver://u:SEK;RIT=x@h:1433;databaseName=d', 'sqlserver://u@h:1433;databaseName=d'], + ['sqlserver://h;password=SEK;RIT=x;databaseName=d', 'sqlserver://h;databaseName=d'], + ['sqlserver://h;password=SEK@RIT;databaseName=d', 'sqlserver://h;databaseName=d'], + ['sqlserver://h;password=SEK:RIT@x;databaseName=d', 'sqlserver://h;databaseName=d'], + ['https://h/x?token=SEK;RIT&mode=ro', 'https://h/x?mode=ro'], + ['https://h/x?mode=ro;password=SEKRIT#f', 'https://h/x#f'], + ['//u:SEKRIT@h/db', '//u@h/db'], + ['jdbc:mysql://u:SEKRIT@h/db?useSSL=true', 'jdbc:mysql://u@h/db?useSSL=true'], + ['jdbc:oracle:thin:scott/SEKRIT@//h:1521/svc', 'jdbc:oracle:thin:scott@//h:1521/svc'], + ])('an unquoted credential holding `;`, `=`, `:` or `@` leaves no tail: %s', (value, expected) => { + expect(embeddedCredentialOf(value)).toBeDefined(); + const out = redactEmbeddedCredentials(value); + expect(out).toBe(expected); + expect(out).not.toMatch(/SEK|RIT/); + // What the read door serves is itself credential-free: the write door accepts it back. + expect(embeddedCredentialOf(out)).toBeUndefined(); + }); +}); + +/** The DatasourceSchema issues a parse raises, as dotted paths (every one a `custom` refusal). */ +function refusals(config: Record, driver = DRIVER): string[] { + const result = DatasourceSchema.safeParse({ name: 'warehouse', driver, config }); + if (result.success) return []; + return result.error.issues.map((issue) => { + expect(issue.code).toBe('custom'); + return issue.path.join('.'); + }); +} + +describe('write door: DatasourceSchema refuses inline credentials for a driver with no shipped contract', () => { + it('the driver under test has no contract; every builtin does', () => { + expect(getDriverConfigSchema(DRIVER)).toBeUndefined(); + expect(isContractlessDriver(DRIVER)).toBe(true); + for (const known of ['postgres', 'mysql', 'mongodb', 'turso', 'sqlite', 'sqlite-wasm', 'memory', 'pg', 'mongo']) { + expect(isContractlessDriver(known), known).toBe(false); + } + }); + + // A bare `key` is narrowed by its value (pinned in its own block below). + it.each(CREDENTIAL_SHAPED.filter((key) => key !== 'key'))('refuses a non-empty %s, at its own path', (key) => { + expect(refusals({ host: 'h', [key]: 'cleartext-value' })).toEqual([`config.${key}`]); + }); + + it('names the refused position in the message', () => { + const result = DatasourceSchema.safeParse({ name: 'warehouse', driver: DRIVER, config: { apiKey: 'k' } }); + expect(result.success ? '' : result.error.issues[0]?.message).toContain('`config.apiKey`'); + }); + + it('refuses credentials inside array elements, judged by key', () => { + expect( + refusals({ + servers: [{ host: 'a', password: 'p1' }, { host: 'b', port: 1 }], + headers: [ + { name: 'Authorization', value: 'Bearer t' }, + { name: 'X-API-Key', value: 'k' }, + { name: 'Accept', value: 'application/json' }, + ], + hosts: ['https://u:p@h/x', 'https://h/y'], + }).sort(), + ).toEqual(['config.headers.0.value', 'config.headers.1.value', 'config.hosts.0', 'config.servers.0.password']); + }); + + it('a credential-shaped key inside array data is refused — array data is no longer off the walk', () => { + // Inverts the earlier pin that accepted `seed: [{ password: 'row-data' }]`. + expect(refusals({ seed: [{ password: 'row-data' }] })).toEqual(['config.seed.0.password']); + }); + + it('refuses the value of a `[name, value]` header tuple naming a credential, and only that', () => { + expect( + refusals({ headers: [['Authorization', 'Bearer t'], ['Cookie', 'sid=1'], ['Accept', 'application/json']] }).sort(), + ).toEqual(['config.headers.0.1', 'config.headers.1.1']); + expect(refusals({ pairs: [['token', '']], range: ['password', 'x'] })).toEqual([]); + }); + + it('control: plain row data in an array is accepted', () => { + expect(refusals({ seed: [{ name: 'a', amount: 1 }, { name: 'b', amount: 2 }], tags: ['x', 'y'] })).toEqual([]); + }); + + it('a credential-shaped object: its secret leaves are refused, its descriptors and identities are not', () => { + expect(refusals({ credentials: { type: 'service_account', clientId: 'cid' } })).toEqual([]); + expect(refusals({ credentials: { type: 'service_account', value: 'sa' } })).toEqual(['config.credentials.value']); + expect(refusals({ auth: { user: 'u', accessToken: 'at' } })).toEqual(['config.auth.accessToken']); + expect(refusals({ oauth: { clientId: 'cid', client_secret: 's' } })).toEqual(['config.oauth.client_secret']); + }); + + it('refuses a number and an array of values under a credential-shaped key; a boolean is a flag', () => { + expect(refusals({ pin: 1, password: 1234, apiKeys: ['k1', 'k2'] }).sort()).toEqual([ + 'config.apiKeys', + 'config.password', + ]); + expect(refusals({ password: true, secret: false, apiKeys: [], token: null })).toEqual([]); + }); + + it('refuses a subtree too deep to judge instead of skipping it', () => { + let deep: Record = { leaf: 'x' }; + for (let i = 0; i < 20; i += 1) deep = { n: deep }; + const found = refusals({ host: 'h', deep }); + expect(found).toHaveLength(1); + expect(found[0]).toMatch(/^config\.deep(\.n)+$/); + }); + + it('refuses credentials embedded in strings — URL, URL tail, connection strings, scheme-less userinfo', () => { + expect( + refusals({ + url: 'https://u:p@h/x', + dsn: 'https://h/x?password=q', + jdbc: 'sqlserver://h;user=u;password=p', + connectionString: 'Server=h;Password=p', + libpq: 'host=h password=p', + target: 'u:p@h/db', + libpqSemicolon: 'host=h password=a;b', + oracle: 'jdbc:oracle:thin:scott/tiger@//h:1521/svc', + }), + ).toEqual([ + 'config.url', 'config.dsn', 'config.jdbc', 'config.connectionString', 'config.libpq', 'config.target', + 'config.libpqSemicolon', 'config.oracle', + ]); + }); + + it('accepts an environment-name placeholder in place of a value — and only that grammar', () => { + expect( + refusals({ + apiKey: '${API_KEY}', + token: '${A}${B_2}', + connectionString: 'Server=h;Password=${DB_PASSWORD}', + url: 'https://u:${PW}@h/x', + }), + ).toEqual([]); + expect( + refusals({ apiKey: 'sk-live-${SUFFIX}', secret: '${lower_case}', token: '${API_KEY:-fallback}' }).sort(), + ).toEqual(['config.apiKey', 'config.secret', 'config.token']); + }); + + it('accepts ordinary configuration and an empty credential', () => { + expect( + refusals({ + host: 'h', + primaryKey: 'id', + accessKeyId: 'AKIA1', + credentialsRef: 'r', + secretsManagerRegion: 'eu-west-1', + useDefaultCredentials: true, + maxTokens: 4096, + password: '', + url: 'https://u@h/x?mode=ro', + connectionString: 'Server=h;User Id=u;Database=d', + }), + ).toEqual([]); + }); + + it('a driver WITH a contract is judged exactly as before', () => { + const pg = DatasourceSchema.safeParse({ name: 'pg', driver: 'postgres', config: { host: 'h', password: 'p' } }); + expect(pg.success).toBe(false); + expect(pg.success ? [] : pg.error.issues.map((issue) => issue.message)).not.toEqual( + expect.arrayContaining([expect.stringContaining('ships no config contract')]), + ); + expect(refusals({ url: 'mongodb://h/db', options: { apiKey: 'k' } }, 'mongodb')).toEqual([]); + }); +}); + +/** A stored contractless config exercising every finding kind. */ +const STORED = { + host: 'h', + apiKey: 'k', + pin: 1234, + apiKeys: ['k1', 'k2'], + oauth: { clientId: 'cid', client_secret: 'cs' }, + credentials: { type: 'service_account', value: 'sa' }, + servers: [{ host: 'a', password: 'p1' }, { host: 'b' }], + headers: [{ name: 'Authorization', value: 'Bearer t' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Authorization', 'Bearer t2'], ['Accept', 'application/json']], + connectionString: 'Server=h;User Id=u;Password=p;Database=d', + libpq: 'host=h password=p dbname=d', + url: 'https://u:p@h/x?mode=ro', + seed: [{ name: 'a', amount: 1 }], + useDefaultCredentials: true, +}; + +describe('read door: redactDatasourceConfig for a driver with no shipped contract', () => { + it('withholds every finding, array elements included, and keeps everything else', () => { + const { config, redactedKeys } = redactDatasourceConfig(DRIVER, STORED); + expect(config).toEqual({ + host: 'h', + pin: 1234, + oauth: { clientId: 'cid' }, + credentials: { type: 'service_account' }, + servers: [{ host: 'a' }, { host: 'b' }], + headers: [{ name: 'Authorization' }, { name: 'Accept', value: 'application/json' }], + tuples: [['Authorization'], ['Accept', 'application/json']], + connectionString: 'Server=h;User Id=u;Database=d', + libpq: 'host=h dbname=d', + url: 'https://u@h/x?mode=ro', + seed: [{ name: 'a', amount: 1 }], + useDefaultCredentials: true, + }); + expect(redactedKeys).toEqual([ + 'apiKey', + 'apiKeys', + 'connectionString', + 'credentials.value', + 'headers.0.value', + 'libpq', + 'oauth.client_secret', + 'servers.0.password', + 'tuples.0.1', + 'url', + ]); + expect(JSON.stringify(config)).not.toMatch(/"k"|k1|"cs"|"sa"|p1|Bearer|Password=p|password=p|u:p@/); + }); + + it('withholds a subtree too deep to judge — the same position the write door refuses', () => { + let deep: Record = { leaf: 'x' }; + for (let i = 0; i < 20; i += 1) deep = { n: deep }; + const { config, redactedKeys } = redactDatasourceConfig(DRIVER, { host: 'h', deep }); + expect(redactedKeys).toHaveLength(1); + expect(['config', ...(redactedKeys[0] as string).split('.')].join('.')).toEqual(refusals({ host: 'h', deep })[0]); + expect(JSON.stringify(config)).not.toContain('leaf'); + }); + + it('an array element withheld before its siblings is nulled, never shifting them; one at the end is spliced', () => { + // Twenty nested `[inner, 'sib']` pairs: the walk's depth cap lands on an + // `inner` that has a sibling after it. + let nested: unknown = ['x', 'sib']; + for (let i = 0; i < 20; i += 1) nested = [nested, 'sib']; + const { config, redactedPaths } = redactDatasourceConfig(DRIVER, { list: nested }); + expect(redactedPaths).toHaveLength(1); + let node = (config as { list: unknown }).list; + let levels = 0; + while (Array.isArray(node)) { + expect(node).toHaveLength(2); + expect(node[1]).toBe('sib'); + node = node[0]; + levels += 1; + } + expect(node).toBeNull(); + expect(levels).toBe((redactedPaths[0] as readonly string[]).length - 1); + // A withheld element at the END of its array is spliced. + const tail = redactDatasourceConfig(DRIVER, { headers: [['Accept', 'json'], ['Authorization', 'Bearer t']] }); + expect(tail.config).toEqual({ headers: [['Accept', 'json'], ['Authorization']] }); + }); + + it('the input is never mutated', () => { + const before = JSON.stringify(STORED); + redactDatasourceConfig(DRIVER, STORED); + expect(JSON.stringify(STORED)).toBe(before); + }); + + it('one walk, two doors: the write door refuses what the read door withholds, and accepts what it serves', () => { + const refused = refusals(STORED).sort(); + const { config, redactedKeys } = redactDatasourceConfig(DRIVER, STORED); + expect(refused).toEqual(redactedKeys.map((key) => `config.${key}`).sort()); + expect(refused).toEqual(findContractlessCredentials(STORED).map((f) => ['config', ...f.path].join('.')).sort()); + expect(refusals(config)).toEqual([]); + }); + + it('leaves ordinary configuration — and the reference to a bound secret — alone', () => { + const stored = { host: 'h', primaryKey: 'id', accessKeyId: 'AKIA1', tokenUrl: 'https://h/t', credentialsRef: 'r' }; + expect(redactDatasourceConfig(DRIVER, stored)).toEqual({ config: stored, redactedKeys: [], redactedPaths: [] }); + }); + + it('a driver WITH a contract is judged exactly as before (no name-shape guess layered on)', () => { + const mongo = redactDatasourceConfig('mongodb', { url: 'mongodb://h/db', options: { apiKey: 'k' } }); + expect(mongo.config).toEqual({ url: 'mongodb://h/db', options: { apiKey: 'k' } }); + const pg = redactDatasourceConfig('postgres', { host: 'h', applicationName: 'a=b;Password=c' }); + expect(pg.config).toEqual({ host: 'h', applicationName: 'a=b;Password=c' }); + }); +}); + +// --------------------------------------------------------------------------- +// Round 3: bounded judgment, lenient parses, header shapes, key names, +// embedded shapes, context, bytes, and the bare `key`. +// --------------------------------------------------------------------------- + +/** Wall-clock milliseconds one call takes. */ +function elapsed(run: () => unknown): number { + const started = performance.now(); + run(); + return performance.now() - started; +} + +describe('the judgment is linear and bounded: an over-long key is judged conservatively', () => { + const CAPITALS = 'A'.repeat(100_000); + + it('a 100 KB run of capitals is judged fast — as a key, a segment key, a pair label, a header name', () => { + expect(elapsed(() => isCredentialShapedConfigKey(CAPITALS))).toBeLessThan(2000); + expect(elapsed(() => findContractlessCredentials({ [CAPITALS]: 'x', [`${CAPITALS}b`]: 'y' }))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf(`${CAPITALS}=v`))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf(`${CAPITALS}: v`))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf(`https://h/x?${CAPITALS}=v`))).toBeLessThan(2000); + }); + + it('a long libpq string is parsed in one pass', () => { + expect(elapsed(() => embeddedCredentialOf('k=v '.repeat(50_000)))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf('x '.repeat(50_000)))).toBeLessThan(2000); + }); + + it('a key longer than 256 characters is credential-shaped unread; one within the cap is judged by its words', () => { + expect(isCredentialShapedConfigKey('A'.repeat(257))).toBe(true); + expect(isCredentialShapedConfigKey('host'.repeat(70))).toBe(true); + expect(isCredentialShapedConfigKey('A'.repeat(256))).toBe(false); + expect(isCredentialShapedConfigKey(CAPITALS)).toBe(true); + expect(refusals({ [CAPITALS]: 'x' })).toEqual([`config.${CAPITALS}`]); + // A connection-string segment key, too. + expect(embeddedCredentialOf(`${'X'.repeat(300)}=v`)).toBeDefined(); + expect(embeddedCredentialOf(`${'X'.repeat(200)}=v`)).toBeUndefined(); + }); +}); + +describe('libpq keyword/value pairs are found leniently', () => { + it.each([ + ['a stray token', 'host=h stray password=p'], + ['a stray quoted token', "host=h 'quoted stray' password=p"], + ['a leading word', 'note password=p'], + ['spaces around `=`', 'host = h password = p'], + ['an unclosed quote', "host=h password='a b"], + ])('finds the credential beside %s', (_label, value) => { + expect(embeddedCredentialOf(value)).toBeDefined(); + expect(connectionStringCredentialKeys(value)).toContain('password'); + expect(redactEmbeddedCredentials(value)).not.toMatch(/password/); + }); + + it('keeps everything else, the stray token included', () => { + expect(redactEmbeddedCredentials('host=h stray password=p dbname=d')).toBe('host=h stray dbname=d'); + expect(redactEmbeddedCredentials('password=p host=h')).toBe('host=h'); + }); + + it.each([ + ['no credential keyword', 'host=h stray dbname=d'], + ['a credential word as a VALUE', 'mode=password'], + ['a descriptor keyword', 'mode=ro passwordless=1'], + ['an empty value', 'host=h password='], + ])('finds nothing with %s', (_label, value) => { + expect(embeddedCredentialOf(value)).toBeUndefined(); + }); +}); + +describe('header shapes', () => { + it('a `Name: value` header line naming a credential is found and loses its value', () => { + // A single-line string is read as a header line only under a header-ish key. + expect(embeddedCredentialOf('Authorization: Bearer t', { headerish: true })).toBeDefined(); + expect(embeddedCredentialOf('X-Api-Key:k', { headerish: true })).toBeDefined(); + expect(embeddedCredentialOf('Accept: json\nX-Api-Key:k')).toBeDefined(); + expect(embeddedCredentialOf('Authorization: Bearer t')).toBeUndefined(); + expect(redactEmbeddedCredentials('Accept: json\r\nAuthorization: Bearer t')).toBe('Accept: json\r\nAuthorization:'); + expect(refusals({ headers: ['Authorization: Bearer t', 'Accept: json'] })).toEqual(['config.headers.0']); + }); + + it('a header line naming no credential, or with no value, is not', () => { + expect(embeddedCredentialOf('Content-Type: application/json')).toBeUndefined(); + expect(embeddedCredentialOf('Authorization:')).toBeUndefined(); + expect(embeddedCredentialOf('note: see the runbook')).toBeUndefined(); + }); + + it('a flat raw-headers list under a header-ish key: each credential name\'s value', () => { + expect(refusals({ headers: ['Authorization', 'Bearer t', 'Accept', 'json', 'Cookie', 'sid=1'] })).toEqual([ + 'config.headers.1', + 'config.headers.5', + ]); + expect(refusals({ rawHeaders: ['Accept', 'token', 'X-Kind', 'v'] })).toEqual([]); + }); + + it('a flat list under a key that is not header-ish is plain data', () => { + expect(refusals({ tags: ['token', 'profile'], scopes: ['password', 'email'] })).toEqual([]); + }); + + it('a tuple directly under a header-ish key, and a tuple longer than two', () => { + expect(refusals({ header: ['Authorization', 'Bearer t'] })).toEqual(['config.header.1']); + expect(refusals({ list: [['Authorization', 'Bearer', 't']] }).sort()).toEqual(['config.list.0.1', 'config.list.0.2']); + expect(refusals({ list: [['Accept', 'json', 'xml']] })).toEqual([]); + }); + + it('every label key of a pair is judged — `{ key: \'Authorization\', value }` holds its secret in `value`', () => { + expect( + refusals({ + headers: [ + { key: 'Authorization', value: 'Bearer t' }, + { name: 'h1', key: 'X-Api-Key', value: 'k' }, + { header: 'Accept', name: 'Cookie', value: 'sid=1' }, + { key: 'Accept', value: 'json' }, + ], + }).sort(), + ).toEqual(['config.headers.0.value', 'config.headers.1.value', 'config.headers.2.value']); + }); + + it('a pair label is still judged for an embedded credential', () => { + expect(refusals({ list: [{ name: 'https://u:p@h/x', value: 'v' }] })).toEqual(['config.list.0.name']); + }); +}); + +describe('key names: TLS key material, `privkey`, trailing qualifiers, normalisation', () => { + it.each([ + 'sslKey', 'tlsKey', 'ssl_key', 'SSL-Key', 'sslkey', 'tlskey', 'privkey', 'sslPrivkey', 'privateKeyData', + 'tokenString', 'tokenStr', 'authData', 'encryptionKeyHex', 'pwdHash', 'apiKeyRaw', 'secretContent', 'basicauth', + 'bearerauth', + // full-width `password`, read as `password` after NFKC + 'ｐａｓｓｗｏｒｄ', + // a key holding non-ASCII letters (Cyrillic), or a zero-width character inside a word + 'пароль', + 'pass​word', + ])('%j is credential-shaped', (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(true); + expect(refusals({ [key]: 'cleartext-value' })).toEqual([`config.${key}`]); + }); + + it.each(['sslMode', 'tlsVersion', 'sslCert', 'rawData', 'userData', 'contentType', 'metadata', 'dataSource', 'hashAlgorithm', 'oauth', 'keyspace'])( + '%j is not credential-shaped', + (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(false); + }, + ); +}); + +describe('embedded credentials: token usernames, signatures, JSON, form encoding, fragments', () => { + const TOKEN = 'ghp_16C7e42F292c6912E7710c838347Ae178B4a'; + + it.each([ + ['a token-shaped URL username with no password', `https://${TOKEN}@github.com/o/r.git`, 'https://github.com/o/r.git'], + ['a token-shaped username with an empty password', `https://${TOKEN}:@github.com/o/r.git`, 'https://github.com/o/r.git'], + ['a `sig` query parameter', 'https://a.blob.core.windows.net/c?sv=2020&sig=abc%3D', 'https://a.blob.core.windows.net/c?sv=2020'], + ['an `X-Amz-Signature` query parameter', 'https://b.s3.amazonaws.com/k?X-Amz-Date=1&X-Amz-Signature=abc', 'https://b.s3.amazonaws.com/k?X-Amz-Date=1'], + ['a JSON-encoded object', '{"apiKey":"k","host":"h"}', '{"host":"h"}'], + ['a JSON-encoded header list', '[{"name":"Authorization","value":"Bearer t"}]', '[{"name":"Authorization"}]'], + ['a JSON-encoded connection string', '{"dsn":"postgres://u:p@h/db"}', '{"dsn":"postgres://u@h/db"}'], + ['a form-encoded string', 'a=b&pass=c', 'a=b'], + ['a form-encoded string with a leading `?`', '?client_secret=s&grant_type=x', '?grant_type=x'], + ['a `#password=` fragment', 'https://h/x#password=p', 'https://h/x'], + ['an implicit-grant fragment', 'https://h/cb#access_token=t&state=s', 'https://h/cb#state=s'], + ])('finds and strips %s', (_label, value, expected) => { + expect(embeddedCredentialOf(value)).toBeDefined(); + const out = redactEmbeddedCredentials(value); + expect(out).toBe(expected); + expect(embeddedCredentialOf(out)).toBeUndefined(); + }); + + it.each([ + ['a plain URL username', 'https://deploy@github.com/o/r.git'], + ['an email-shaped URL username', 'https://ops%40example.com@h/x'], + ['an ordinary query', 'https://h/x?signal=1&design=2'], + ['a JSON object with no credential', '{"host":"h","port":5432}'], + ['a JSON-looking string that does not parse', '{not json, password=p'], + ['a form-encoded string with no credential', 'a=b&c=d'], + ['a fragment that is an anchor', 'https://h/docs#section-2'], + ])('finds nothing in %s', (_label, value) => { + // (The JSON-looking string that does not parse is judged by the other readings.) + if (value.startsWith('{not')) { + expect(embeddedCredentialOf(value)).toBeDefined(); + return; + } + expect(embeddedCredentialOf(value)).toBeUndefined(); + expect(redactEmbeddedCredentials(value)).toBe(value); + }); + + it('the write and read doors agree on a JSON-encoded config value', () => { + const stored = { options: '{"auth":{"user":"u","password":"p"},"timeoutMs":5}' }; + expect(refusals(stored)).toEqual(['config.options']); + const { config } = redactDatasourceConfig(DRIVER, stored); + expect(JSON.parse((config as { options: string }).options)).toEqual({ auth: { user: 'u' }, timeoutMs: 5 }); + expect(refusals(config)).toEqual([]); + }); + + it('a JSON-encoded value nested past the walk depth is not accepted unjudged', () => { + let nested = '"x"'; + for (let i = 0; i < 20; i += 1) nested = JSON.stringify({ n: JSON.parse(nested) }); + expect(embeddedCredentialOf(nested)).toBeDefined(); + expect(embeddedCredentialOf(redactEmbeddedCredentials(nested))).toBeUndefined(); + }); +}); + +describe('a descriptor key does not reset the credential context for an object below it', () => { + it('an object under a descriptor key inside a credential-shaped object is still judged as credential material', () => { + expect(refusals({ auth: { source: { value: 'abc' } } })).toEqual(['config.auth.source.value']); + expect(refusals({ credentials: { provider: { kind: 'gcp', blob: 'xyz' } } })).toEqual([ + 'config.credentials.provider.blob', + ]); + }); + + it('the descriptor exemption still holds for a leaf — and for a list of leaves', () => { + expect(refusals({ auth: { scopes: ['read', 'write'], type: 'oauth', provider: 'google' } })).toEqual([]); + }); +}); + +describe('bytes and opaque containers', () => { + it('bytes under a credential-shaped key are ONE finding, withheld whole', () => { + expect(refusals({ apiKey: Buffer.from('secret-bytes') })).toEqual(['config.apiKey']); + const { config, redactedKeys } = redactDatasourceConfig(DRIVER, { host: 'h', privateKey: new Uint8Array([1, 2, 3]) }); + expect(config).toEqual({ host: 'h' }); + expect(redactedKeys).toEqual(['privateKey']); + expect(refusals({ tokens: [new Uint8Array([1])] })).toEqual(['config.tokens']); + }); + + it('bytes elsewhere are judged by their text, as one value', () => { + expect(refusals({ blob: new Uint8Array([1, 2, 3]) })).toEqual([]); + expect(refusals({ blob: Buffer.from('https://u:p@h/x') })).toEqual(['config.blob']); + expect(redactDatasourceConfig(DRIVER, { blob: Buffer.from('https://u:p@h/x') }).config).toEqual({}); + }); + + it('a Map or a Set is not accepted unjudged, and is withheld whole', () => { + const result = DatasourceSchema.safeParse({ name: 'w', driver: DRIVER, config: { opts: new Map([['a', 1]]) } }); + expect(result.success).toBe(false); + expect(result.success ? '' : result.error.issues[0]?.message).toContain('Map or a Set'); + expect(refusals({ list: [new Set(['x'])] })).toEqual(['config.list.0']); + expect(redactDatasourceConfig(DRIVER, { host: 'h', opts: new Map([['password', 'p']]) }).config).toEqual({ host: 'h' }); + }); +}); + +describe('a bare `key` is credential material only where it can be key material', () => { + it('`{ key: \'email\' }` and other names are accepted', () => { + expect( + refusals({ key: 'email', sort: { key: 'created_at', order: 'asc' }, keys: ['email', 'name'], index: { key: 42 } }), + ).toEqual([]); + }); + + it('a value that looks like a secret, or a credential-shaped or header-ish holder, makes it one', () => { + expect(refusals({ key: 'sk_live_51HxQ2bL9aZ0rT7yU' })).toEqual(['config.key']); + expect(refusals({ keys: ['email', 'a3f9c2d17b4e8a6f0c5d9e2b1a7f4c3e'] })).toEqual(['config.keys']); + expect(refusals({ headers: { key: 'abc' } })).toEqual(['config.headers.key']); + expect(refusals({ apiKey: { key: 'abc' } })).toEqual(['config.apiKey.key']); + expect(refusals({ auth: { key: 'abc' } })).toEqual(['config.auth.key']); + }); + + it('elsewhere `key` keeps its full judgment: a query parameter, a connection-string segment', () => { + expect(isCredentialShapedConfigKey('key')).toBe(true); + expect(embeddedCredentialOf('https://maps.example.com/api?key=abc')).toBeDefined(); + }); + + it('the secret-looking predicate', () => { + for (const secret of ['sk_live_51HxQ2bL9aZ0rT7yU', 'AIzaSyD-9tSrke72PouQMnMX-a7eZSW0jkFMBWY', 'a3f9c2d17b4e8a6f0c5d9e2b1a7f4c3e']) { + expect(looksLikeSecretValue(secret), secret).toBe(true); + } + for (const name of ['email', 'customer_email_2', 'orders.created_at', 'AKIA1234567890', 'two words here please']) { + expect(looksLikeSecretValue(name), name).toBe(false); + } + }); +}); + +describe('PEM private keys, and a bare `key` in TLS options', () => { + const PEM = '-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASC\nBKcwggSjAgEAAoIBAQC7\n-----END PRIVATE KEY-----\n'; + const ARMOURS = [ + PEM, + PEM.replace(/PRIVATE KEY/g, 'RSA PRIVATE KEY'), + PEM.replace(/PRIVATE KEY/g, 'EC PRIVATE KEY'), + PEM.replace(/PRIVATE KEY/g, 'ENCRYPTED PRIVATE KEY'), + PEM.replace(/PRIVATE KEY/g, 'OPENSSH PRIVATE KEY'), + '-----BEGIN PGP PRIVATE KEY BLOCK-----\nlQOYBF\n-----END PGP PRIVATE KEY BLOCK-----', + ]; + + it.each(['ssl', 'tls', 'certificate', 'clientCert', 'mtls'])('a PEM key under a bare `key` inside `%s` is refused and withheld', (holder) => { + const config = { host: 'h', [holder]: { key: PEM, cert: '-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----', rejectUnauthorized: true } }; + expect(refusals(config)).toEqual([`config.${holder}.key`]); + const served = redactDatasourceConfig(DRIVER, config).config; + expect(JSON.stringify(served)).not.toContain('MIIEvQ'); + expect((served[holder] as Record).cert).toContain('CERTIFICATE'); + }); + + it('a bare `key` under a TLS holder is withheld whatever its value; under another holder a name stays', () => { + expect(refusals({ ssl: { key: 'k' } })).toEqual(['config.ssl.key']); + expect(refusals({ sort: { key: 'created_at' }, sslMode: 'require' })).toEqual([]); + }); + + it.each(ARMOURS.map((armour, i) => [i, armour] as const))('armour %i is secret material in any string, wherever it sits', (_i, armour) => { + expect(embeddedCredentialOf(armour)).toBeDefined(); + expect(refusals({ options: { material: armour } })).toEqual(['config.options.material']); + expect(refusals({ list: [`prefix ${armour}`] })).toEqual(['config.list.0']); + const served = redactDatasourceConfig(DRIVER, { options: { material: `before ${armour} after` } }).config; + expect(JSON.stringify(served)).not.toMatch(/PRIVATE KEY|MIIEvQ|lQOYBF/); + expect(embeddedCredentialOf(redactEmbeddedCredentials(armour))).toBeUndefined(); + }); + + it('PEM bytes, and `pfx` bytes or strings, are judged', () => { + expect(refusals({ blob: Buffer.from(PEM) })).toEqual(['config.blob']); + expect(refusals({ pfx: Buffer.from([0x30, 0x82, 0x01]) })).toEqual(['config.pfx']); + expect(refusals({ tls: { pfx: 'MIIKCQIBAzCCCc8GCSqGSIb3' } })).toEqual(['config.tls.pfx']); + expect(refusals({ sslPfx: 'MIIK' })).toEqual(['config.sslPfx']); + expect(refusals({ tls: { pfx: [{ buf: 'MIIK', passphrase: 'pp' }] } })).toEqual(['config.tls.pfx.0.buf', 'config.tls.pfx.0.passphrase']); + }); + + it('control: a certificate, a CA bundle and a public key are not private keys', () => { + expect(refusals({ ssl: { ca: '-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----', cert: '-----BEGIN PUBLIC KEY-----\nMIIB\n-----END PUBLIC KEY-----' } })).toEqual([]); + expect(isCredentialShapedConfigKey('pfxPath')).toBe(false); + }); +}); + +describe('a non-ASCII key is credential-shaped only when the non-ASCII text sits inside or next to a credential word', () => { + it.each(['客户名称', 'Größe', 'café', 'größeInBytes', 'naïve_mode', 'nombre_compañía', '名前'])('%j is not credential-shaped', (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(false); + }); + + it('`fieldMap: { 客户名称: … }` is accepted at the write door and served whole', () => { + const config = { fieldMap: { '客户名称': 'customer_name', 'Größe': 'size', 'café': 'cafe' } }; + expect(refusals(config)).toEqual([]); + expect(redactDatasourceConfig(DRIVER, config).config).toEqual(config); + }); + + it.each([ + // a Cyrillic `а` (U+0430) inside `password`, and a Cyrillic `е` (U+0435) inside `key` / `token` + 'pаssword', 'db_pаss', 'kеy', 'tokеn', 'api_kеy', + // a Greek omicron inside `password` + 'passwοrd', + // zero-width characters inside a word + 'pass​word', 'to‍ken', 'sec­ret', + // a non-ASCII character standing in for, or inserted into, a letter + 'pas§word', 'tok€n', 'secéret', + // next to a credential word + 'password密码', 'token値', '客户password', 'apiKeyé', + // a credential word of another script + '数据库密码', 'пароль', 'パスワード', + // full-width, read after NFKC + 'ｐａｓｓｗｏｒｄ', + ])('%j is credential-shaped', (key) => { + expect(isCredentialShapedConfigKey(key)).toBe(true); + expect(refusals({ [key]: 'cleartext-value' })).toEqual([`config.${key}`]); + }); + + it('the same rule applies to query, form and segment parameter names', () => { + expect(embeddedCredentialOf('https://h/x?客户名称=a')).toBeUndefined(); + expect(embeddedCredentialOf('https://h/x?pаssword=a')).toBeDefined(); + expect(embeddedCredentialOf('Größe=1;Server=h')).toBeUndefined(); + expect(embeddedCredentialOf('Server=h;pаssword=a')).toBeDefined(); + expect(embeddedCredentialOf('café=1&x=2')).toBeUndefined(); + expect(embeddedCredentialOf('x=1&tokеn=2')).toBeDefined(); + }); +}); + +describe('prose, SQL and opaque URIs are not credential strings', () => { + it.each([ + ['a one-line description', { description: 'Password: provided via the secret store' }], + ['a SQL bind placeholder', { query: 'SELECT id FROM t WHERE token = $1' }], + ['a `?` placeholder', { query: 'SELECT id FROM t WHERE password = ? AND x = 1' }], + ['a named placeholder', { query: 'UPDATE t SET a = 1 WHERE token = :token' }], + ['a mailto URI', { url: 'mailto:alice@example.com' }], + ['a sip URI', { contact: 'sip:alice@example.com' }], + ['a tel URI', { contact: 'tel:+1-201-555-0123' }], + ['a urn', { id: 'urn:isbn:0451450523' }], + ['a time of day', { note: '12:30@office' }], + ['a time with seconds', { note: '09:05:59@x' }], + ])('%s is accepted', (_label, config) => { + expect(refusals(config)).toEqual([]); + expect(redactDatasourceConfig(DRIVER, config).config).toEqual(config); + }); + + it.each([ + ['a header line in a multi-line string', { note: 'Accept: json\nAuthorization: Bearer abc' }, 'config.note'], + ['a one-line header under a header-ish key', { headers: ['Authorization: Bearer abc'] }, 'config.headers.0'], + ['a one-line header string under `extraHeaders`', { extraHeaders: 'X-Api-Key: abc' }, 'config.extraHeaders'], + ['a SQL literal, not a placeholder', { query: "host=h password=hunter2" }, 'config.query'], + ['a sip URI that carries a password', { contact: 'sip:alice:secret@example.com' }, 'config.contact'], + ['userinfo that is not a time', { dsn: 'admin:hunter2@db.internal/app' }, 'config.dsn'], + ])('control: %s is still refused and withheld', (_label, config, path) => { + expect(refusals(config)).toEqual([path]); + expect(JSON.stringify(redactDatasourceConfig(DRIVER, config).config)).not.toMatch(/abc|hunter2|secret@/); + }); +}); + +describe('an over-long string is judged conservatively', () => { + const LONG = 'a'.repeat(64 * 1024 + 1); + + it('longer than 64 KiB: refused at write, withheld whole on read', () => { + expect(embeddedCredentialOf(LONG)).toBeDefined(); + expect(refusals({ blob: LONG })).toEqual(['config.blob']); + expect(refusals({ bytes: Buffer.alloc(64 * 1024 + 1, 0x61) })).toEqual(['config.bytes']); + expect(redactDatasourceConfig(DRIVER, { host: 'h', blob: LONG }).config).toEqual({ host: 'h', blob: '' }); + }); + + it('control: at the cap a string is still read', () => { + expect(refusals({ blob: 'a'.repeat(64 * 1024) })).toEqual([]); + }); +}); + +describe('a bare `key` value: hex and digit-free base64 key material', () => { + it.each(['deadbeefcafebabe', '0123456789abcdef', 'DEADBEEFCAFEBABE0123', 'kPqRzXwYvTnMbLcD+aHf/QeGsJuWiOoK', 'kPqRzXwYvTnMbLcDaHfQeG==', 'kPqRzXwYvTnMbLcDaHfQeGsJuWiOoK'])( + '`{ key: %j }` is key material', + (value) => { + expect(refusals({ key: value })).toEqual(['config.key']); + }, + ); + + it.each(['email', 'customerEmailAddress', 'XMLHttpRequestURLBuilder', 'getCustomerEmailAddressForAccount', 'created_at', '2024010112000000', 'orders/customer/name', 'Orders/Customer/Name', 'Sales/Region/Quarter/Total'])( + '`{ key: %j }` stays a name', + (value) => { + expect(refusals({ key: value })).toEqual([]); + }, + ); +}); + +describe('the round-4 readings stay bounded', () => { + it('a 256-character mixed-script key, PEM-shaped and placeholder-shaped strings are judged fast', () => { + expect(elapsed(() => isCredentialShapedConfigKey('pé'.repeat(128)))).toBeLessThan(2000); + expect(elapsed(() => isCredentialShapedConfigKey(`${'​'.repeat(250)}pass`))).toBeLessThan(2000); + expect(elapsed(() => isCredentialShapedConfigKey('§a'.repeat(128)))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf(`-----BEGIN ${'A '.repeat(30_000)}`))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf('-----BEGIN PRIVATE KEY-----'.repeat(2_000)))).toBeLessThan(2000); + expect(elapsed(() => redactEmbeddedCredentials('-----BEGIN PRIVATE KEY-----x'.repeat(2_000)))).toBeLessThan(2000); + expect(elapsed(() => embeddedCredentialOf('token = $1 '.repeat(5_000)))).toBeLessThan(2000); + }); +}); + +describe('what the read door serves is what the write door accepts back', () => { + const PEM = '-----BEGIN PRIVATE KEY-----\nMIIEvQ\n-----END PRIVATE KEY-----'; + it.each([ + ['a `key:`-labelled pair in a headers list keeps its label', { headers: [{ key: 'Authorization', value: 'Bearer x' }, { key: 'Accept', value: 'json' }] }], + ['a `key:`-labelled pair directly under a header-ish key', { probeHeader: { key: 'X-Api-Key', value: 'k-1' } }], + ['a `name:`-labelled pair under a plain key', { proxy: { name: 'Authorization', value: 'Bearer x' } }], + ['TLS options', { ssl: { key: PEM, cert: 'c', passphrase: 'pp' } }], + ['a header map', { headers: { key: 'abc', Authorization: 'Bearer x', Accept: 'json' } }], + ['strings', { dsn: 'admin:pw@db/app', note: 'Accept: json\nAuthorization: Bearer x', blob: PEM }], + ])('%s', (_label, config) => { + const served = redactDatasourceConfig(DRIVER, config).config; + expect(findContractlessCredentials(served)).toEqual([]); + expect(refusals(served)).toEqual([]); + }); + + it('a `key:` label in a list element is a label, not the header named `key`', () => { + const served = redactDatasourceConfig(DRIVER, { headers: [{ key: 'Authorization', value: 'Bearer x' }] }); + expect(served.config).toEqual({ headers: [{ key: 'Authorization' }] }); + expect(refusals({ headers: [{ key: 'Authorization' }] })).toEqual([]); + // control: in a header map, `key` is the header named `key` + expect(refusals({ headers: { key: 'Authorization' } })).toEqual(['config.headers.key']); + }); + + it('a label left alone under a header-ish key is withheld too, and named among the withheld paths', () => { + const served = redactDatasourceConfig(DRIVER, { probeHeader: { key: 'X-Api-Key', value: 'k-1' } }); + expect(served.config).toEqual({ probeHeader: {} }); + expect(served.redactedKeys).toEqual(['probeHeader.key', 'probeHeader.value']); + }); +}); diff --git a/packages/spec/src/data/datasource-credential-redaction.ts b/packages/spec/src/data/datasource-credential-redaction.ts index 43066bf2622..11c09716970 100644 --- a/packages/spec/src/data/datasource-credential-redaction.ts +++ b/packages/spec/src/data/datasource-credential-redaction.ts @@ -63,12 +63,18 @@ * which is the one question this module answers. * * For a driver the platform ships no contract for, source 1 is empty — the - * registry is saying "nothing to check against", not "nothing to protect". The - * canonical spellings are therefore redacted by NAME for unknown drivers too. - * That asymmetry with the write gate (which deliberately lets an unknown - * driver's config through untouched) is intentional: declining to REFUSE an - * unrecognised key is a boundary choice about authoring, while serving a key - * literally named `password` back in cleartext is a leak under any boundary. + * registry is saying "nothing to check against", not "nothing to protect". Such + * a driver's config is judged by NAME and value SHAPE instead, through the one + * walk in `driver/contractless-credentials.ts` (`findContractlessCredentials`, + * built on `isCredentialShapedConfigKey`): values under credential-shaped keys, + * credential-shaped objects, `{ name, value }` header pairs, array elements, + * and credentials embedded in URLs and connection strings + * ({@link isContractlessDriver}). The write door judges the SAME + * predicate: since ADR-0015 §10 ("credentials never appear in metadata + * artefacts"), a contractless driver's config refuses the same credential + * material at publish (`data/datasource.zod.ts`), so this read half is what + * protects rows stored before that refusal existed — and rows written through + * a door that does not parse. * * ## URL-embedded credentials * @@ -109,6 +115,7 @@ import { FORMER_CREDENTIAL_ALIASES, MONGO_OPTIONS_CREDENTIAL_PATHS, } from './driver/common.zod'; +import { withholdContractlessCredentials } from './driver/contractless-credentials'; import { getDriverConfigSchema, resolveDriverId } from './driver/config-registry.zod'; // The canonical spellings and former aliases MOVED to `driver/common.zod.ts` @@ -366,6 +373,25 @@ export function refusedCredentialKeys(driver: unknown): string[] { .map(([key]) => key); } +/** + * Does the platform ship NO config contract for `driver` (a plugin-contributed + * driver, or a spelling no builtin claims)? + * + * For such a driver the read path does not run the four sources below: it + * withholds every position `findContractlessCredentials` + * (`driver/contractless-credentials.ts`) reports — the same walk the write + * door refuses by. Without a contract nothing else can say which key is + * the secret, and serving one back in cleartext is a leak under any boundary. + * A driver WITH a contract is unaffected: its measured lists decide. + */ +export function isContractlessDriver(driver: unknown): boolean { + try { + return getDriverConfigSchema(driver) === undefined; + } catch { + return true; + } +} + /** Every config key this module hides for `driver`, canonical + alias + writable-but-secret. */ export function redactableConfigKeys(driver: unknown): string[] { const derived = refusedCredentialKeys(driver); @@ -474,7 +500,9 @@ export interface RedactedDatasourceConfig { * Remove every stored credential from a driver `config` for serving on a read * path — at EVERY object depth, not only the top level. * - * Four removal sources, applied in order: + * Four removal sources, applied in order (and, for a driver the platform ships + * no contract for, two NAME judgments folded into the first two — see + * {@link isContractlessDriver}): * * 1. The credential-name judgment ({@link redactableConfigKeys}) at every * object depth. It was top-level-only until the nested-position finding: @@ -517,14 +545,19 @@ export function redactDatasourceConfig( ): RedactedDatasourceConfig { if (!config || typeof config !== 'object') return { config: {}, redactedKeys: [], redactedPaths: [] }; + const contractless = isContractlessDriver(driver); + if (contractless) return redactContractlessConfig(config); + const hidden = new Set(redactableConfigKeys(driver)); + const isHidden = (key: string): boolean => hidden.has(key); + const redactString = (value: string): string => redactUrlCredentials(value); const removed: (readonly string[])[] = []; const scrub = (node: Record, prefix: readonly string[]): Record => { const out: Record = {}; for (const [key, value] of Object.entries(node)) { const path = [...prefix, key]; - if (hidden.has(key)) { + if (isHidden(key)) { // Dropped, not masked. A mask would round-trip back through the wizard // as a literal new password, and post-#8078 the canonical spellings // would then be REFUSED at the write door — turning an untouched @@ -534,7 +567,7 @@ export function redactDatasourceConfig( continue; } if (typeof value === 'string') { - const redacted = redactUrlCredentials(value); + const redacted = redactString(value); if (redacted !== value) { out[key] = redacted; removed.push(path); @@ -577,6 +610,19 @@ export function redactDatasourceConfig( }; } +/** + * The read half of the contractless-driver judgment: every position + * `findContractlessCredentials` (`driver/contractless-credentials.ts`) reports + * — the SAME walk the write door refuses by — is withheld, through + * `withholdContractlessCredentials` in that module. Every position is + * reported, array indices included, so the write-path inverses can restore + * exactly what was withheld. Pure: the input is never mutated. + */ +function redactContractlessConfig(config: Record): RedactedDatasourceConfig { + const { config: out, paths } = withholdContractlessCredentials(config); + return { config: out, redactedKeys: paths.map((path) => path.join('.')), redactedPaths: paths }; +} + /** * `container` without the leaf at `path`, copying only the spine — pure, like * everything else on this read path. `dropped` is `false` when the walk falls diff --git a/packages/spec/src/data/datasource.zod.ts b/packages/spec/src/data/datasource.zod.ts index bea475baf59..5655a7c986a 100644 --- a/packages/spec/src/data/datasource.zod.ts +++ b/packages/spec/src/data/datasource.zod.ts @@ -5,6 +5,7 @@ import { lazySchema } from '../shared/lazy-schema'; import { strictObject } from '../shared/strict-object'; import { MetadataProtectionFields } from '../kernel/metadata-protection.zod'; import { urlUserinfoUsername } from './driver/common.zod'; +import { embeddedCredentialOf, findContractlessCredentials } from './driver/contractless-credentials'; import { resolveDriverId, validateDriverConfig } from './driver/config-registry.zod'; /* @@ -30,9 +31,12 @@ import { resolveDriverId, validateDriverConfig } from './driver/config-registry. * is prescribed toward `external.credentialsRef` rather than merely relocated. * * A driver the platform ships no contract for (a plugin's - * `com.vendor.snowflake`) keeps an unvalidated `config`. That is the honest + * `com.vendor.snowflake`) keeps an unvalidated config SHAPE. That is the honest * boundary, not a leftover hole — see the registry's own note on why inventing * a verdict against a shape we do not have would be worse than the silence. + * What such a config does not keep is inline credential material: ADR-0015 §10 + * holds for every driver, so credential-shaped keys and embedded credentials + * are refused by name (`reportContractlessInlineCredentials` below). */ const CAPABILITIES_REMOVED_PREFIX = @@ -468,12 +472,134 @@ const CREDENTIALS_REF_MONGO_NO_USERNAME_REFUSED = + 'discrete fields with a `config.url` that names a user is a third valid shape; it is judged ' + 'by the URL-branch refusal, not by this one.)'; +/** + * The remedy every contractless-driver credential refusal ends on. Names the + * one working route a credential has into a driver the platform ships no + * contract for (ADR-0062 D3): the bound secret is resolved at connect and + * handed to the driver factory as the connection secret. Deliberately offers no + * environment-placeholder escape — nothing interpolates one in a stored + * datasource `config`, so a placeholder would be persisted and sent verbatim. + */ +const CONTRACTLESS_CREDENTIAL_REMEDY = + 'Remove it from `config` and bind the secret instead: the Setup → Datasources connection ' + + "form's secret field (or `external.credentialsRef`) encrypts it into `sys_secret`, stores " + + 'only an opaque handle on the datasource, and the connect path hands the decrypted value to ' + + "this driver's factory as the connection secret. The driver receives it only if its factory " + + 'reads that injected secret — a driver that needs it under this key must be taught to.'; + +/** Refusal for credential material under a credential-shaped key in a contractless driver's `config`. */ +const CONTRACTLESS_INLINE_CREDENTIAL_REFUSED = (path: string, driver: string): string => + `\`${path}\` holds credential material and is not accepted in the \`config\` of a datasource ` + + `whose driver ('${driver}') the platform ships no config contract for: the datasource is ` + + 'persisted whole into `sys_metadata` and its history, so an inline credential lands in ' + + 'cleartext at rest — credentials never appear in metadata artefacts. ' + + CONTRACTLESS_CREDENTIAL_REMEDY; + +/** Refusal for credential material EMBEDDED in a contractless driver's string value. */ +const CONTRACTLESS_EMBEDDED_CREDENTIAL_REFUSED = (path: string, driver: string, what: string): string => + `\`${path}\` carries ${what} — credential material that is not accepted in the \`config\` of a ` + + `datasource whose driver ('${driver}') the platform ships no config contract for: the ` + + 'datasource is persisted whole into `sys_metadata` and its history, so the embedded ' + + 'credential lands in cleartext at rest. Strip it from the string (a URL that keeps only ' + + '`user@host`, a connection string without the credential segment, is accepted). ' + + CONTRACTLESS_CREDENTIAL_REMEDY; + +/** Refusal for a subtree nested past the depth the credential walk judges. */ +const CONTRACTLESS_TOO_DEEP_REFUSED = (path: string, driver: string): string => + `\`${path}\` is nested deeper than the credential check of a datasource whose driver ('${driver}') ` + + 'the platform ships no config contract for reads, so whether it holds credential material ' + + 'cannot be judged, and it is not accepted unjudged. Flatten the configuration. ' + + CONTRACTLESS_CREDENTIAL_REMEDY; + +/** Refusal for a `Map` or `Set`, whose entries the credential walk does not read. */ +const CONTRACTLESS_OPAQUE_REFUSED = (path: string, driver: string): string => + `\`${path}\` is a Map or a Set, whose entries the credential check of a datasource whose driver ` + + `('${driver}') the platform ships no config contract for does not read, so whether it holds ` + + 'credential material cannot be judged, and it is not accepted unjudged. Use a plain object or array. ' + + CONTRACTLESS_CREDENTIAL_REMEDY; + +/** + * An environment placeholder in the one grammar a value may be replaced by + * (`${API_KEY}`: an upper-case environment name in `${…}`, the shell/compose + * convention the placeholder census measured authors writing). Such a span + * carries no credential material, so a value made only of them is not refused; + * any other `${…}` content is judged as written. + */ +const ENV_NAME_PLACEHOLDER_RE = /\$\{[A-Z_][A-Z0-9_]*\}/g; + +/** + * Refuse inline credential material in the `config` of a datasource whose + * driver the platform ships NO contract for (ADR-0015 §10: credentials never + * appear in metadata artefacts). + * + * A driver WITH a contract refuses its credential slots through the contract + * (`z.never()`, #8078) and is not judged here. A contractless driver has no + * slot to read, so the judgment is by NAME — the one predicate the read-path + * redactor also uses (`isCredentialShapedConfigKey`), so publish-time refusal + * and read-time redaction treat a position identically: + * + * The judgment is ONE walk (`findContractlessCredentials`, + * `driver/contractless-credentials.ts`) that the read-path redactor reads too, + * so every position refused here is a position withheld there: a value under + * a credential-shaped key (strings, numbers, bytes, arrays of them), the + * non-descriptor leaves of a credential-shaped object, the `value` of a + * `{ name, value }` pair naming a credential, a header tuple's or raw-headers + * list's value, a string with an embedded credential (URL userinfo, a + * token-shaped URL username, a URL query or fragment parameter, a URL + * `;key=value` tail, a semicolon, libpq or form-encoded string, a + * `Name: value` header line, scheme-less `user:password@host`, a JSON-encoded + * value), array elements included, and a subtree too deep to judge or a + * `Map` / `Set` it cannot read. + * + * Accepted: an EMPTY string (the `user:@host` posture, and the explicit way to + * clear a stored value), a boolean, and a value made only of environment + * placeholders in the `${NAME}` grammar ({@link ENV_NAME_PLACEHOLDER_RE}) — + * judged with those spans removed, so a literal secret beside one is still + * found. + * + * Routing the value into the secret store instead was the other option, and + * is not taken: for a contractless driver nothing says which key the factory + * reads its credential from, so moving `config.apiKey` into the single bound + * secret would hand the factory a secret it may never read and silently strip + * the key it does — the reason the credential-migration planner already + * refuses to re-home such a row. + */ +function reportContractlessInlineCredentials( + ctx: z.RefinementCtx, + driver: unknown, + config: unknown, + basePath: (string | number)[], +): void { + const driverName = String(driver); + for (const finding of findContractlessCredentials(config)) { + const dotted = ['config', ...finding.path].join('.'); + let message: string; + if (finding.kind === 'depth') { + message = CONTRACTLESS_TOO_DEEP_REFUSED(dotted, driverName); + } else if (finding.kind === 'opaque') { + message = CONTRACTLESS_OPAQUE_REFUSED(dotted, driverName); + } else if (finding.kind === 'named') { + if (typeof finding.value === 'string' && finding.value.replace(ENV_NAME_PLACEHOLDER_RE, '').trim() === '') continue; + message = CONTRACTLESS_INLINE_CREDENTIAL_REFUSED(dotted, driverName); + } else { + const what = embeddedCredentialOf(String(finding.value).replace(ENV_NAME_PLACEHOLDER_RE, ''), { + headerish: finding.headerish === true, + }); + if (!what) continue; + message = CONTRACTLESS_EMBEDDED_CREDENTIAL_REFUSED(dotted, driverName, what); + } + ctx.addIssue({ code: 'custom', path: [...basePath, ...finding.path], message }); + } +} + /** * Replay a driver-config parse onto the datasource's own issue list (#4410). * - * A no-op for a driver the platform ships no contract for — `known: false` is - * the registry saying "nothing to check against", which is deliberately NOT the - * same answer as "checked and clean". + * For a driver the platform ships no contract for — `known: false` is the + * registry saying "nothing to check against", which is deliberately NOT the + * same answer as "checked and clean" — the shape is left unjudged, and only + * inline credential material is refused + * ({@link reportContractlessInlineCredentials}). */ function reportDriverConfigIssues( ctx: z.RefinementCtx, @@ -482,7 +608,10 @@ function reportDriverConfigIssues( basePath: (string | number)[], ): void { const result = validateDriverConfig(driver, config); - if (!result.known) return; + if (!result.known) { + reportContractlessInlineCredentials(ctx, driver, config, basePath); + return; + } for (const issue of result.issues) { ctx.addIssue({ code: 'custom', diff --git a/packages/spec/src/data/driver/contractless-credentials.ts b/packages/spec/src/data/driver/contractless-credentials.ts new file mode 100644 index 00000000000..c78074e7186 --- /dev/null +++ b/packages/spec/src/data/driver/contractless-credentials.ts @@ -0,0 +1,1651 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Credential material in the `config` of a datasource whose driver the + * platform ships NO config contract for (a plugin-contributed driver). + * + * For such a driver no schema types a position, so the inline-credential + * refusal at write (`data/datasource.zod.ts`) and the read-path redaction + * (`data/datasource-credential-redaction.ts`) both judge by NAME and by value + * SHAPE — and both read the ONE walk in this module + * ({@link findContractlessCredentials}), so the two doors cannot drift: every + * position the write door refuses is a position the read door withholds. + * + * ⛔ Never consulted for a driver WITH a contract: there the contract's own + * `z.never()` slots and the measured passthrough tables decide, and a + * name-shape guess layered over a measured list would refuse configuration a + * measured client reads. + */ + +import { CREDENTIAL_KEY_SPELLINGS } from './common.zod'; + +// --------------------------------------------------------------------------- +// The key-name judgment +// --------------------------------------------------------------------------- + +/** + * A key split into lower-case words at separators and camel-case boundaries: + * `secretAccessKey` → `secret access key`, `X-API-Key` → `x api key`, + * `client_secret` → `client secret`. Judging WORDS rather than substrings is + * what keeps `bypass`, `passive`, `primaryKey` and `partitionKey` out while + * `pass`, `key` and `apiKey` are in. A run of non-ASCII characters is a word + * of its own (`客户名称` is one word, `Größe` is `gr`, `öß`, `e`); every ASCII + * character other than a letter or a digit is a separator. + */ +function wordsOf(key: string): string[] { + // Every pattern here is linear: each matches a fixed number of characters + // per attempt. The split before the last capital of a run (`APIKey` → + // `API Key`) looks ahead instead of capturing the run, so a long run of + // capitals is not re-scanned at every position. + return key + .replace(/([a-z0-9])([A-Z])/g, '$1 $2') + .replace(/([A-Z])(?=[A-Z][a-z])/g, '$1 ') + .replace(/([^\x00-\x7f]+)/g, ' $1 ') + .toLowerCase() + .split(/[\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\x7f]+/) + .filter(Boolean); +} + +/** + * Longer than this (after NFKC normalisation), a key — an object key, a + * connection-string segment key, a query or form parameter name, a header + * name — is not split into words at all: it is judged credential-shaped + * unread, so an over-long name can neither cost the judgment time nor hide a + * credential from it. + */ +const MAX_JUDGED_KEY_LENGTH = 256; + +/** + * A key as the word judgment reads it: NFKC-normalised (so a full-width + * `ｐａｓｓｗｏｒｄ` reads as `password`), or `undefined` when it is longer than + * {@link MAX_JUDGED_KEY_LENGTH} — such a key is judged credential-shaped. + */ +function judgedKey(key: string): string | undefined { + if (key.length > MAX_JUDGED_KEY_LENGTH * 4) return undefined; + const normalized = key.normalize('NFKC'); + return normalized.length > MAX_JUDGED_KEY_LENGTH ? undefined : normalized; +} + +/** The words of a key as the judgment reads them; `undefined` past the length cap. */ +function judgedWords(key: string): string[] | undefined { + const judged = judgedKey(key); + return judged === undefined ? undefined : wordsOf(judged); +} + +const NON_ASCII_RE = /[^\x00-\x7f]/; +const isNonAscii = (c: string): boolean => c.charCodeAt(0) > 0x7f; + +/** + * Invisible format characters (soft hyphen, zero-width space / joiners, + * word joiner, BOM, Mongolian vowel separator): NFKC keeps them, and inside a + * word they hide it from a word judgment. + */ +const FORMAT_CHAR_RE = /[\u00ad\u180e\u200b-\u200f\u2060-\u2064\ufeff]/g; + +/** + * Cyrillic and Greek letters that render as a Latin letter (`а` U+0430 → `a`): + * a key spelled with them reads to a person as the Latin word. + */ +const CONFUSABLE_LETTERS: Readonly> = { + 'а': 'a', 'в': 'b', 'е': 'e', 'ё': 'e', 'к': 'k', 'м': 'm', 'н': 'h', 'о': 'o', 'р': 'p', 'с': 'c', 'т': 't', + 'у': 'y', 'х': 'x', 'і': 'i', 'ї': 'i', 'ј': 'j', 'ѕ': 's', 'ԁ': 'd', 'һ': 'h', 'ӏ': 'l', 'ԛ': 'q', 'ԝ': 'w', + 'α': 'a', 'β': 'b', 'ε': 'e', 'η': 'n', 'ι': 'i', 'κ': 'k', 'ν': 'v', 'ο': 'o', 'ρ': 'p', 'τ': 't', 'υ': 'u', + 'χ': 'x', 'ω': 'w', +}; + +/** The key with invisible format characters removed and confusable letters read as Latin ones. */ +function confusableReading(key: string): string { + return key.replace(FORMAT_CHAR_RE, '').replace(/[^\x00-\x7f]/g, (c) => { + const lower = c.toLowerCase(); + const latin = CONFUSABLE_LETTERS[lower]; + if (latin === undefined) return c; + return lower === c ? latin : latin.toUpperCase(); + }); +} + +/** + * Credential words in other scripts, matched as a SUBSTRING of a key's + * lower-cased non-ASCII text (`数据库密码`): CJK text has no word separators. + */ +const NON_LATIN_CREDENTIAL_WORDS: readonly string[] = [ + '密码', '密碼', '口令', '密钥', '密鑰', '秘钥', '令牌', '凭证', '憑證', + 'パスワード', '暗証番号', '秘密鍵', 'トークン', + '비밀번호', '토큰', + 'пароль', 'токен', + 'contraseña', +]; +/** {@link CREDENTIAL_KEY_SPELLINGS} folded to one lower-case word each (`auth_token` → `authtoken`). */ +const FOLDED_CREDENTIAL_KEY_SPELLINGS: ReadonlySet = new Set( + CREDENTIAL_KEY_SPELLINGS.map((key) => wordsOf(key).join('')), +); + +/** Words that make a key credential-shaped in ANY position (`dbPassword`, `secretAccessKey`, `serviceCredential`). */ +const CREDENTIAL_WORDS_ANYWHERE: ReadonlySet = new Set([ + 'password', + 'passwd', + 'passphrase', + 'secret', + 'credential', +]); + +/** + * Words that make a key credential-shaped as its LAST word (`accessToken`, + * `dbPass`, `githubPat`, `basicAuth`, `sasToken`): each is also an ordinary + * leading word (`tokenUrl`, `passThrough`, `cookieDomain`, `authMethod`), so + * only the head noun of the key counts. + */ +const CREDENTIAL_WORDS_LAST: ReadonlySet = new Set([ + 'token', + 'pass', + 'pw', + 'pwd', + 'jwt', + 'pat', + 'cookie', + 'sas', + 'auth', + 'authorization', + 'bearer', + 'apikey', + 'privkey', + 'pfx', + 'pkcs12', + 'p12', +]); + +/** + * `key` as the last word is key material only beside one of these + * (`apiKey`, `privateKey`, `accountKey`, `sharedAccessKey`, + * `serviceAccountKey`) or alone (`key`). `primaryKey`, `partitionKey`, + * `sortKey` and `cacheKey` are ordinary configuration, and `accessKey` alone is + * the IDENTITY half of an access-key pair (`accessKeyId` / `secretAccessKey`). + */ +const KEY_MATERIAL_QUALIFIERS: ReadonlySet = new Set([ + 'api', + 'private', + 'secret', + 'signing', + 'master', + 'encryption', + 'decryption', + 'account', + 'shared', + 'client', + 'session', + 'auth', + 'hmac', + 'license', + 'subscription', + 'ssh', + 'aes', + 'storage', + 'ssl', + 'tls', +]); + +/** `signature` as the last word is credential material beside one of these (`sharedAccessSignature`). */ +const SIGNATURE_QUALIFIERS: ReadonlySet = new Set(['shared', 'access', 'sas', 'hmac', 'amz']); + +/** + * Trailing words that qualify a credential stem without changing what it is + * (`apiKeyValue`, `tokenValue`, `privateKeyPem`, `keyJson`, `privateKeyData`, + * `tokenString`, `authData`, `encryptionKeyHex`, `pwdHash`). + */ +const CREDENTIAL_TRAILING_QUALIFIERS: ReadonlySet = new Set([ + 'value', + 'values', + 'pem', + 'json', + 'b64', + 'base64', + 'data', + 'content', + 'hex', + 'string', + 'str', + 'raw', + 'hash', +]); + +/** + * LAST words that name WHERE or WHAT a credential is rather than the credential + * itself: a reference into a secret store (`credentialsRef`, `secretArn`, + * `secretName`), a locator (`passwordFile`, `privateKeyPath`, `tokenUrl`, + * `tokenEndpoint`, `passwordEnv`, `secretsManagerRegion`), an identifier + * (`clientSecretId`, `accessKeyId`) or a descriptor (`tokenType`, + * `apiKeyHeader`, `tokenPrefix`, `credentialProvider`, `credentialSource`, + * `credentialChain`, `passwordPolicy`, `authMethod`, `passwordEnabled`, + * `passwordAuthentication`). A last word ending in `less` (`passwordless`) is + * a descriptor too. + */ +const CREDENTIAL_DESCRIPTOR_ENDINGS: ReadonlySet = new Set([ + 'ref', + 'refs', + 'reference', + 'arn', + 'id', + 'ids', + 'name', + 'names', + 'path', + 'paths', + 'file', + 'files', + 'filename', + 'url', + 'urls', + 'uri', + 'endpoint', + 'env', + 'type', + 'header', + 'headers', + 'field', + 'prefix', + 'mode', + 'region', + 'provider', + 'source', + 'chain', + 'policy', + 'method', + 'enabled', + 'authentication', + 'format', + 'version', + 'expiry', + 'expires', + 'length', + 'count', +]); + +/** + * FIRST words that make a multi-word key a flag or a measure, never a value: + * `useDefaultCredentials`, `requirePassword`, `maxTokens`. + */ +const NON_CREDENTIAL_LEADING_WORDS: ReadonlySet = new Set([ + 'use', + 'enable', + 'enabled', + 'disable', + 'require', + 'required', + 'allow', + 'has', + 'is', + 'no', + 'skip', + 'max', + 'min', + 'num', + 'count', + 'total', +]); + +/** + * Folded compound endings that make a ONE-word key credential-shaped — the + * spelling an all-lower-case or all-upper-case key (`APIKEY`, `accesstoken`, + * `secretaccesskey`) collapses to, where no word boundary survives. + */ +const FOLDED_CREDENTIAL_ENDINGS: readonly string[] = [ + 'token', + 'pwd', + 'apikey', + 'privatekey', + 'secretkey', + 'signingkey', + 'masterkey', + 'encryptionkey', + 'accountkey', + 'sharedkey', + 'sharedaccesskey', + 'clientkey', + 'sessionkey', + 'authkey', + 'hmackey', + 'licensekey', + 'subscriptionkey', + 'serviceaccountkey', + 'serviceaccountjson', + 'accesssignature', + 'privkey', + 'pfx', + 'sslkey', + 'tlskey', + 'basicauth', + 'bearerauth', + 'digestauth', +]; + +/** Every stem a trailing `s` may pluralize (`apiKeys`, `tokens`, `credentials`). */ +const PLURALIZABLE_STEMS: ReadonlySet = new Set([ + ...CREDENTIAL_WORDS_ANYWHERE, + ...CREDENTIAL_WORDS_LAST, + 'key', + 'signature', +]); + +/** A trailing `s` is a plural only after a stem that does not itself end in `s` (`sass` is not `sas`s). */ +const pluralOf = (word: string): boolean => word.endsWith('s') && !word.endsWith('ss'); + +function singular(word: string): string { + return pluralOf(word) && PLURALIZABLE_STEMS.has(word.slice(0, -1)) ? word.slice(0, -1) : word; +} + +/** + * What may FOLLOW a {@link CREDENTIAL_WORDS_ANYWHERE} word inside a one-word + * key (`dbpassword`, `secretkey`, `secretaccesskey`, `passwordhash`) — so the + * word counts only as a whole word of the folded key, never as a substring of + * a longer one (`secretary`, `credentialing`). + */ +const FOLDED_ANYWHERE_FOLLOWERS: readonly string[] = ['', 'key', 'accesskey', 'hash', 'string']; + +const holdsAnywhereWord = (folded: string): boolean => + [...CREDENTIAL_WORDS_ANYWHERE].some((word) => { + for (let at = folded.indexOf(word); at !== -1; at = folded.indexOf(word, at + 1)) { + if (FOLDED_ANYWHERE_FOLLOWERS.includes(folded.slice(at + word.length))) return true; + } + return false; + }); + +/** The ONE-word judgment, on a key with no surviving word boundary. */ +function isCredentialShapedWord(word: string): boolean { + if (FOLDED_CREDENTIAL_KEY_SPELLINGS.has(word)) return true; + if (word.endsWith('less')) return false; + if (/^(max|min|num|total|count)/.test(word)) return false; + for (const ending of CREDENTIAL_DESCRIPTOR_ENDINGS) { + if (word.length > ending.length && word.endsWith(ending)) return false; + } + let stem = word; + for (let changed = true; changed;) { + changed = false; + for (const qualifier of CREDENTIAL_TRAILING_QUALIFIERS) { + if (stem.length > qualifier.length && stem.endsWith(qualifier)) { + stem = stem.slice(0, -qualifier.length); + changed = true; + } + } + } + const candidates = pluralOf(stem) ? [stem, stem.slice(0, -1)] : [stem]; + return candidates.some((candidate) => + PLURALIZABLE_STEMS.has(candidate) + || holdsAnywhereWord(candidate) + || FOLDED_CREDENTIAL_ENDINGS.some((ending) => candidate.endsWith(ending)), + ) || word.endsWith('serviceaccountjson') || word.endsWith('serviceaccountpem'); +} + +/** + * Does a key's non-ASCII text hide a credential word — sit inside one or next + * to one? (`judged` is NFKC-normalised and holds a non-ASCII character.) + * + * - it holds a credential word of another script + * ({@link NON_LATIN_CREDENTIAL_WORDS}: `数据库密码`, `пароль`); + * - read with its invisible format characters removed and its confusable + * Cyrillic and Greek letters as the Latin letters they render as + * ({@link confusableReading}: `pаssword` with a Cyrillic `а`, `kеy`), the key + * is credential-shaped; + * - in a run of characters with no ASCII separator in it that mixes ASCII + * letters or digits with non-ASCII characters, the ASCII word that TOUCHES + * a non-ASCII character is credential-shaped on its own (`password密码`, + * `token値`); or + * - in such a run, a credential word of four or more letters is spelled with + * non-ASCII characters standing in for, or inserted between, some of its + * letters — at most one per four letters (`pas§word`, `tok€n`) — invisible + * format characters inserted free (`pass` U+200B `word`). + * + * Anything else non-ASCII is an ordinary word of its own ({@link wordsOf}): + * `客户名称`, `Größe` and `café` are not credential-shaped. + */ +function nonAsciiHidesCredential(judged: string): boolean { + const lower = judged.toLowerCase(); + if (NON_LATIN_CREDENTIAL_WORDS.some((word) => lower.includes(word))) return true; + const reading = confusableReading(judged); + if (reading !== judged && judgeWords(wordsOf(reading))) return true; + for (const run of judged.split(/[\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\x7f]+/)) { + if (!NON_ASCII_RE.test(run) || !/[A-Za-z0-9]/.test(run)) continue; + if (touchingWordIsCredential(run)) return true; + const chars = [...run.toLowerCase()]; + if (FUZZY_CREDENTIAL_WORDS.some((word) => spelledAround(chars, word))) return true; + } + return false; +} + +/** In a run mixing ASCII and non-ASCII characters: is an ASCII word that touches a non-ASCII character credential-shaped? */ +function touchingWordIsCredential(run: string): boolean { + const pieces = run.split(/([^\x00-\x7f]+)/); + for (let i = 0; i < pieces.length; i += 2) { + const ascii = pieces[i] as string; + if (ascii === '') continue; + const words = wordsOf(ascii); + if (words.length === 0) continue; + if (i > 0 && isCredentialShapedWord(words[0] as string)) return true; + if (i < pieces.length - 1 && isCredentialShapedWord(words[words.length - 1] as string)) return true; + } + return false; +} + +/** The credential words a non-ASCII spelling is matched against (four letters or more). */ +const FUZZY_CREDENTIAL_WORDS: readonly string[] = [...new Set([ + ...CREDENTIAL_WORDS_ANYWHERE, + ...CREDENTIAL_WORDS_LAST, + 'signature', +])].filter((word) => word.length >= 4); + +const isFormatChar = (c: string): boolean => /^[\u00ad\u180e\u200b-\u200f\u2060-\u2064\ufeff]$/.test(c); + +/** + * Does some stretch of `chars` spell `word` with at least one, and at most one + * per four letters, non-ASCII character standing in for a letter or inserted + * between two — invisible format characters inserted between letters free? + * Bounded: a key is at most 256 characters, a word at most 13, the edit budget + * at most 3. + */ +function spelledAround(chars: readonly string[], word: string): boolean { + const budget = Math.floor(word.length / 4); + const align = (i: number, j: number, left: number, used: boolean): boolean => { + if (j === word.length) return used; + if (i >= chars.length) return false; + const c = chars[i] as string; + if (c === word[j] && align(i + 1, j + 1, left, used)) return true; + if (j > 0 && isFormatChar(c)) return align(i + 1, j, left, true); + if (!isNonAscii(c) || left === 0) return false; + return align(i + 1, j + 1, left - 1, true) || (j > 0 && align(i + 1, j, left - 1, true)); + }; + for (let start = 0; start < chars.length; start += 1) if (align(start, 0, budget, false)) return true; + return false; +} + +/** + * Is `key` spelled like a credential — judged on its WHOLE name, case- and + * separator-insensitively? + * + * The key is NFKC-normalised first. A key that is then longer than 256 + * characters is credential-shaped unread ({@link judgedKey}), and so is one + * whose non-ASCII text hides a credential word ({@link nonAsciiHidesCredential}). + * Otherwise the key is split into words ({@link wordsOf}; a run of non-ASCII + * characters is a word of its own). It is NOT credential-shaped + * when its last word is a descriptor ({@link CREDENTIAL_DESCRIPTOR_ENDINGS}, + * or a word ending in `less`) or its first word marks a flag or a measure + * ({@link NON_CREDENTIAL_LEADING_WORDS}). Otherwise, after dropping trailing + * qualifiers ({@link CREDENTIAL_TRAILING_QUALIFIERS}) and a plural `s`, it IS + * credential-shaped when: + * + * - folded, it is one of {@link CREDENTIAL_KEY_SPELLINGS} (`password`, + * `authToken` and the former aliases), or + * - any word is one of {@link CREDENTIAL_WORDS_ANYWHERE}, or + * - its last word is one of {@link CREDENTIAL_WORDS_LAST}, or + * - its last word is `key` alone or beside a {@link KEY_MATERIAL_QUALIFIERS} + * word, or `signature` beside a {@link SIGNATURE_QUALIFIERS} word, or + * - it names service-account key material (`serviceAccountKey`, + * `serviceAccountJson`). + * + * A key with one word only (`APIKEY`, `accesstoken`, `dbpassword`) has no + * boundary left to split at, so it is judged on its folded spelling: one of + * {@link CREDENTIAL_KEY_SPELLINGS} is credential-shaped; otherwise ending in + * `less`, starting with `max` / `min` / `num` / `total` / `count`, or ending in + * a descriptor longer than nothing (`tokentype`, `secretid`) is not. After + * dropping trailing qualifiers and a plural `s` (never the `s` of a stem + * already ending in `s`: `sass` is not `sas`), it is credential-shaped when it + * IS one of the stems above (`pass`, `pw`, `key`, `signature`, `auth`, …), + * when it holds a {@link CREDENTIAL_WORDS_ANYWHERE} word followed by nothing, + * `key`, `accesskey`, `hash` or `string` (`dbpassword`, `secretaccesskey` — + * not `secretary`), or when it ends in one of + * {@link FOLDED_CREDENTIAL_ENDINGS} (`accesstoken`, `apikey`). + */ +export function isCredentialShapedConfigKey(key: string): boolean { + if (typeof key !== 'string' || key === '') return false; + const judged = judgedKey(key); + if (judged === undefined) return true; + if (NON_ASCII_RE.test(judged) && nonAsciiHidesCredential(judged)) return true; + return judgeWords(wordsOf(judged)); +} + +/** The word judgment of {@link isCredentialShapedConfigKey}, on a key already split. */ +function judgeWords(words: readonly string[]): boolean { + if (words.length === 0) return false; + if (words.length === 1) return isCredentialShapedWord(words[0] as string); + + const last = words[words.length - 1] as string; + if (CREDENTIAL_DESCRIPTOR_ENDINGS.has(last) || last.endsWith('less')) return false; + if (NON_CREDENTIAL_LEADING_WORDS.has(words[0] as string)) return false; + if (FOLDED_CREDENTIAL_KEY_SPELLINGS.has(words.join(''))) return true; + + const stem = [...words]; + let stripped: string | undefined; + while (stem.length > 1 && CREDENTIAL_TRAILING_QUALIFIERS.has(stem[stem.length - 1] as string)) { + stripped = stem.pop(); + } + if (stripped !== undefined && stem.join('').endsWith('serviceaccount')) return true; + + const singulars = stem.map(singular); + if (singulars.some((word) => CREDENTIAL_WORDS_ANYWHERE.has(word))) return true; + const head = singulars[singulars.length - 1] as string; + const before = singulars.slice(0, -1); + if (CREDENTIAL_WORDS_LAST.has(head)) return true; + if (head === 'key') return before.length === 0 || before.some((word) => KEY_MATERIAL_QUALIFIERS.has(word)); + if (head === 'signature') return before.some((word) => SIGNATURE_QUALIFIERS.has(word)); + return FOLDED_CREDENTIAL_ENDINGS.some((ending) => head.length > ending.length && head.endsWith(ending)); +} + +/** + * Inside an object reached under a credential-shaped key (`credentials: {…}`, + * `auth: {…}`), the leaves that still carry no secret: descriptors and + * identities (`type`, `clientId`, `user`, `email`, `scope`, `region`, …). + * Every other leaf there is judged credential material. + */ +const IDENTITY_LEAF_WORDS: ReadonlySet = new Set([ + 'user', + 'username', + 'login', + 'email', + 'issuer', + 'audience', + 'scope', + 'scopes', + 'algorithm', + 'alg', + 'domain', + 'host', + 'hostname', + 'port', + 'realm', + 'project', + 'tenant', + 'subject', + 'kind', + 'label', + 'description', +]); + +function isDescriptorLeafKey(key: string): boolean { + const words = judgedWords(key); + if (words === undefined) return false; + const last = words[words.length - 1]; + if (last === undefined) return false; + return CREDENTIAL_DESCRIPTOR_ENDINGS.has(last) || IDENTITY_LEAF_WORDS.has(last) || last.endsWith('less'); +} + +/** Is `key` the bare word `key` (or `keys`), in any case and with any separators? */ +function isBareKeyName(key: string): boolean { + const words = judgedWords(key); + if (words === undefined) return false; + return words.length === 1 && (words[0] === 'key' || words[0] === 'keys'); +} + +/** Is `key` header-ish — does one of its words name a header (`headers`, `httpHeaders`, `rawHeaders`)? */ +function isHeaderishKey(key: string | undefined): boolean { + if (key === undefined) return false; + return judgedWords(key)?.some((word) => word === 'header' || word === 'headers') ?? false; +} + +/** + * Words of a holder that make a bare `key` below it key material: the TLS + * options of a client (`ssl: { key, cert, ca }`, `tls: { key }`, pg / mysql2 / + * mongodb / `tls.connect`), and any word starting with `cert` + * (`certificate: { key }`, `clientCert: { key }`). + */ +const KEY_MATERIAL_HOLDER_WORDS: ReadonlySet = new Set(['ssl', 'tls', 'mtls', 'x509', 'pfx', 'pkcs12']); + +/** Is `key` a holder whose bare `key` is TLS key material? */ +function isKeyMaterialHolder(key: string): boolean { + return judgedWords(key)?.some((word) => KEY_MATERIAL_HOLDER_WORDS.has(word) || word.startsWith('cert')) ?? false; +} + +/** + * Does a string LOOK like a secret rather than a name — at least 16 characters + * with no whitespace, and either mixing upper-case letters, lower-case letters + * and digits (`sk_live_51Hx…`, `ghp_…`, `AIza…`), or at least 32 characters of + * a token alphabet holding a digit (a hex or base64 key)? `email`, + * `customer_email_2` and `orders.created_at` do not. + */ +export function looksLikeSecretValue(value: string): boolean { + if (typeof value !== 'string' || value.length < 16) return false; + if (/\s/.test(value)) return false; + const lower = /[a-z]/.test(value); + const upper = /[A-Z]/.test(value); + const digit = /[0-9]/.test(value); + if (lower && upper && digit) return true; + return value.length >= 32 && digit && /^[A-Za-z0-9+/=_\-.~]+$/.test(value); +} + +// --------------------------------------------------------------------------- +// Credential material embedded in a string value +// --------------------------------------------------------------------------- + +/** + * A URL-ish prefix: `scheme://`, a stacked `jdbc:mysql://`, or a scheme-relative `//`. + * Linear: a scheme group ends at its `:`, which its character class excludes, + * so no input can be split into groups more than one way. + */ +const URL_PREFIX_RE = /^(?:[a-z][a-z0-9+.\-]*:)*\/\//i; + +/** + * Connection-string keywords that are NOT part of a credential: after a dropped + * credential segment, a following segment is taken as the tail of that + * credential (an unquoted `;` or `=` inside the password) until a segment whose + * key is one of these — or is itself credential-shaped — starts again. + * Folded (lower-case, letters and digits only). + */ +const CONNECTION_STRING_KEYWORDS: ReadonlySet = new Set([ + 'server', 'datasource', 'address', 'addr', 'networkaddress', 'host', 'hostname', 'hostaddr', 'port', + 'database', 'initialcatalog', 'db', 'dbname', 'user', 'userid', 'uid', 'username', 'login', 'role', + 'integratedsecurity', 'trustedconnection', 'encrypt', 'trustservercertificate', 'hostnameincertificate', + 'connectiontimeout', 'connecttimeout', 'timeout', 'commandtimeout', 'pooling', 'maxpoolsize', 'minpoolsize', + 'applicationname', 'app', 'applicationintent', 'multipleactiveresultsets', 'multisubnetfailover', 'driver', + 'dsn', 'provider', 'schema', 'sslmode', 'ssl', 'sslcert', 'sslrootcert', 'charset', 'characterset', + 'defaultendpointsprotocol', 'accountname', 'endpointsuffix', 'endpoint', 'blobendpoint', 'queueendpoint', + 'sharedaccesskeyname', 'entitypath', 'warehouse', 'account', 'region', 'authentication', 'mode', + 'persistsecurityinfo', 'failoverpartner', 'workstationid', 'wsid', 'packetsize', 'language', 'attachdbfilename', + 'tenant', 'tenantid', 'clientid', 'options', 'targetsessionattrs', 'currentlanguage', 'readonly', + 'databasename', 'servername', 'portnumber', 'instancename', 'logintimeout', 'sockettimeout', 'querytimeout', + 'authenticationscheme', 'domain', 'sendstringparametersasunicode', 'selectmethod', 'responsebuffering', +]); + +const foldKeyword = (key: string): string => key.toLowerCase().replace(/[^a-z0-9]/g, ''); + +/** One `key=value` segment of a semicolon-delimited connection string, with its byte range. */ +interface ConnectionStringSegment { + /** The segment's key, verbatim; `undefined` for a segment with no `=`. */ + key: string | undefined; + /** The segment's value, trimmed and unquoted; `''` for a keyless segment. */ + value: string; + start: number; + end: number; +} + +/** + * Split a semicolon-delimited `key=value` connection string (the ADO.NET / + * ODBC / JDBC-property shape: `Server=h;User Id=u;Password=p`) into segments. + * A value may be quoted — `"…"` or `'…'` with the quote doubled to escape it, + * or `{…}` with `}}` escaping the brace — and a `;` inside a quoted value does + * not end the segment. Anything else runs to the next `;`. + */ +function semicolonSegments(value: string): ConnectionStringSegment[] { + const out: ConnectionStringSegment[] = []; + let i = 0; + while (i <= value.length) { + const start = i; + while (i < value.length && value[i] !== '=' && value[i] !== ';') i += 1; + if (i >= value.length || value[i] === ';') { + out.push({ key: undefined, value: '', start, end: i }); + i += 1; + continue; + } + const key = value.slice(start, i); + i += 1; + while (i < value.length && (value[i] === ' ' || value[i] === '\t')) i += 1; + const open = value[i]; + const close = open === '"' || open === "'" ? open : open === '{' ? '}' : undefined; + let raw: string; + if (close) { + let j = i + 1; + let inner = ''; + while (j < value.length) { + if (value[j] === close) { + if (value[j + 1] === close) { + inner += close; + j += 2; + continue; + } + j += 1; + break; + } + inner += value[j]; + j += 1; + } + while (j < value.length && value[j] !== ';') j += 1; + raw = inner; + i = j; + } else { + const valueStart = i; + while (i < value.length && value[i] !== ';') i += 1; + raw = value.slice(valueStart, i).trim(); + } + out.push({ key, value: raw, start, end: i }); + i += 1; + } + return out; +} + +/** + * A value that STARTS with a SQL bind placeholder (`$1`, `?`, `:name`) + * ending at whitespace, `)`, `,`, `;` or the end — `WHERE token = $1` is a + * query, not a credential. + */ +const SQL_PLACEHOLDER_RE = /^(?:\$[0-9]+|\?|:[A-Za-z_][A-Za-z0-9_]*)(?=[\s),;]|$)/; + +const isCredentialSegment = (segment: ConnectionStringSegment): boolean => + segment.key !== undefined + && segment.value !== '' + && !SQL_PLACEHOLDER_RE.test(segment.value) + && isCredentialShapedConfigKey(segment.key.trim()); + +/** Strip the credential segments of a semicolon-delimited string — and the tail each one drags (see {@link CONNECTION_STRING_KEYWORDS}). */ +function redactSemicolonSegments(value: string): string { + const segments = semicolonSegments(value); + if (!segments.some(isCredentialSegment)) return value; + const kept: string[] = []; + let dropping = false; + for (const segment of segments) { + if (isCredentialSegment(segment)) { + dropping = true; + continue; + } + if (dropping) { + const keyword = segment.key !== undefined && CONNECTION_STRING_KEYWORDS.has(foldKeyword(segment.key)); + if (!keyword) continue; + dropping = false; + } + kept.push(value.slice(segment.start, segment.end)); + } + while (kept.length > 0 && kept[kept.length - 1] === '') kept.pop(); + return kept.join(';'); +} + +/** One `keyword = value` pair of a libpq keyword/value string, with its byte range. */ +interface LibpqPair { + key: string; + value: string; + start: number; + end: number; +} + +const isLibpqSpace = (c: string | undefined): boolean => c === ' ' || c === '\t' || c === '\n' || c === '\r'; +const isLibpqKeyStart = (c: string | undefined): boolean => + c !== undefined && ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c === '_'); +const isLibpqKeyChar = (c: string | undefined): boolean => isLibpqKeyStart(c) || (c !== undefined && c >= '0' && c <= '9'); + +/** + * Every libpq-style `keyword = value` pair in a string (`host=h port=5432 + * password='a b'`), found LENIENTLY: a pair is a keyword at the start of the + * string or after whitespace, optional whitespace, `=`, and a value — + * single-quoted with `\'` / `\\` escapes (an unclosed quote runs to the end of + * the string), or a run of non-space characters. Any other token is skipped, + * so one stray word does not hide the pairs around it. One pass, linear. + */ +function libpqPairs(value: string): LibpqPair[] { + const out: LibpqPair[] = []; + let i = 0; + while (i < value.length) { + while (isLibpqSpace(value[i])) i += 1; + if (i >= value.length) break; + const start = i; + let j = i; + if (isLibpqKeyStart(value[j])) { + j += 1; + while (isLibpqKeyChar(value[j])) j += 1; + } + let k = j; + while (isLibpqSpace(value[k])) k += 1; + if (j === i || value[k] !== '=') { + // Not a pair: skip the rest of this token. + i = j > i ? j : i + 1; + while (i < value.length && !isLibpqSpace(value[i])) i += 1; + continue; + } + const key = value.slice(start, j); + i = k + 1; + while (isLibpqSpace(value[i])) i += 1; + let raw = ''; + if (value[i] === "'") { + i += 1; + while (i < value.length) { + if (value[i] === '\\' && i + 1 < value.length) { + raw += value[i + 1]; + i += 2; + continue; + } + if (value[i] === "'") { + i += 1; + break; + } + raw += value[i]; + i += 1; + } + } else { + while (i < value.length && !isLibpqSpace(value[i])) { + if (value[i] === '\\' && i + 1 < value.length) { + raw += value[i + 1]; + i += 2; + continue; + } + raw += value[i]; + i += 1; + } + } + out.push({ key, value: raw, start, end: i }); + } + return out; +} + +const isCredentialLibpqPair = (pair: LibpqPair): boolean => + pair.value !== '' && !SQL_PLACEHOLDER_RE.test(pair.value) && isCredentialShapedConfigKey(pair.key); + +/** The string without its credential libpq pairs, each cut with the whitespace that separated it. */ +function redactLibpqPairs(value: string): string { + const pairs = libpqPairs(value); + if (!pairs.some(isCredentialLibpqPair)) return value; + let out = ''; + let from = 0; + for (const pair of pairs) { + if (!isCredentialLibpqPair(pair)) continue; + let cutStart = pair.start; + let cutEnd = pair.end; + // Take the whitespace after the pair; for the last pair, the whitespace before it. + while (isLibpqSpace(value[cutEnd])) cutEnd += 1; + if (cutEnd >= value.length) while (cutStart > from && isLibpqSpace(value[cutStart - 1])) cutStart -= 1; + out += value.slice(from, cutStart); + from = cutEnd; + } + return out + value.slice(from); +} + +/** + * `user:password@host…` with no scheme — the userinfo half of a URL written + * without one. The password group allows `@` and is greedy, so the match ends + * at the LAST `@` before the host (a malformed literal `@` must not decide how + * much leaks), and it excludes `/`, so a `scheme://` URL never matches. + * Linear: the anchored groups backtrack over one run each, and the lookahead + * after each candidate `@` stops at the next `@`. + */ +const SCHEMELESS_USERINFO_RE = /^([^\s/?#@:;=]+):([^\s/?#]+)@(?=[^\s/?#@]*[A-Za-z0-9])/; + +/** + * Opaque URI schemes whose `scheme:` prefix is not a userinfo username + * (`mailto:alice@example.com`): the judgment reads what follows the prefix + * instead, so `sip:alice:secret@host` is still found. + */ +const OPAQUE_SCHEME_RE = /^(?:mailto|sips?|tel|urn|xmpp|news|im|pres):/i; + +/** A time of day (`12:30@`, `9:05:59.250@`) — not a `user:password@` pair. */ +const TIME_USERINFO_RE = /^[0-9]{1,2}:[0-9]{2}(?::[0-9]{2}(?:\.[0-9]+)?)?@/; + +/** + * Where a scheme-less `user:password@host` userinfo starts in `value` — after + * an opaque scheme prefix, else `0` — or `-1` when the string holds none + * (a time of day before the `@` is none). + */ +function schemelessUserinfoAt(value: string): number { + const opaque = OPAQUE_SCHEME_RE.exec(value); + const from = opaque ? opaque[0].length : 0; + const rest = value.slice(from); + if (TIME_USERINFO_RE.test(rest) || !SCHEMELESS_USERINFO_RE.test(rest)) return -1; + return from; +} + +/** + * `jdbc:oracle:thin:user/password@…` — the Oracle thin-driver form, whose + * userinfo splits user from password with `/` and carries no `//`. Anchored; + * each group backtracks over one run, so linear. + */ +const ORACLE_THIN_USERINFO_RE = /^(jdbc:oracle:[a-z]+:)([^\s/@:]*)\/([^\s@]+)@/i; + +/** + * The byte layout of a URL-ish string (see {@link URL_PREFIX_RE}), or + * `undefined` when it has none. Boundaries are RFC 3986's, as in + * `urlUserinfo` (`driver/common.zod.ts`): the authority runs from after `//` + * to the first `/`, `?` or `#`, and userinfo ends at its LAST `@`. + */ +interface UrlLayout { + /** Index of the authority's first byte (just past `//`). */ + authorityStart: number; + /** Index of the userinfo's `@`, or `-1`. */ + at: number; + /** The userinfo password (after the userinfo's first `:`), or `undefined` when there is none or it is empty. */ + password: string | undefined; + /** A userinfo with NO password whose username looks like a token (`https://ghp_…@host`). */ + tokenUsername: boolean; + /** Index of the `;` that starts a `;key=value` property tail, or `-1`. */ + tail: number; + /** Index of the `?` or `#` that ends the part a property tail can occupy (the string's length when none). */ + queryStart: number; + /** Index of the `#` that starts the fragment, or `-1`. */ + hash: number; +} + +function urlLayout(value: string): UrlLayout | undefined { + const prefix = URL_PREFIX_RE.exec(value); + if (!prefix) return undefined; + const authorityStart = prefix[0].length; + const authorityRel = value.slice(authorityStart).search(/[/?#]/); + const authorityEnd = authorityRel === -1 ? value.length : authorityStart + authorityRel; + const atIdx = value.lastIndexOf('@', authorityEnd - 1); + const at = atIdx >= authorityStart ? atIdx : -1; + let password: string | undefined; + let tokenUsername = false; + if (at !== -1) { + const userinfo = value.slice(authorityStart, at); + const colon = userinfo.indexOf(':'); + // A `;` before the `:` makes it no userinfo at all but a property tail + // whose value holds a `:` and an `@` (`sqlserver://h;password=a:b@c`). + if (colon !== -1 && colon < userinfo.length - 1 && !userinfo.slice(0, colon).includes(';')) { + password = userinfo.slice(colon + 1); + } else if (!userinfo.slice(0, colon === -1 ? userinfo.length : colon).includes(';')) { + // No password (or an empty one): the username alone may be the token. + let username = colon === -1 ? userinfo : userinfo.slice(0, colon); + try { + username = decodeURIComponent(username); + } catch { + /* judge the raw username */ + } + tokenUsername = looksLikeSecretValue(username); + } + } + // A property tail starts at the first `;` after the userinfo when a + // password ended it (`sqlserver://u:a;b@h;db=d` — that `;` is the + // password's), and otherwise at the first `;` after `//` — so a credential + // property whose value carries an `@` (`sqlserver://h;password=a@b`) is + // still read as a property, not as userinfo. + const tailFrom = password !== undefined || tokenUsername ? at + 1 : authorityStart; + const queryRel = value.slice(tailFrom).search(/[?#]/); + const queryStart = queryRel === -1 ? value.length : tailFrom + queryRel; + const semi = value.indexOf(';', tailFrom); + const hash = value.indexOf('#', queryStart); + return { + authorityStart, + at, + password, + tokenUsername, + tail: semi !== -1 && semi < queryStart ? semi : -1, + queryStart, + hash, + }; +} + +/** Query and fragment parameter names that are credential material although not credential-shaped as config keys (`sig`, the SAS signature). */ +const CREDENTIAL_PARAMETER_NAMES: ReadonlySet = new Set(['sig']); + +/** Is one `&`-separated query, fragment or form pair credential material — a credential-shaped name, or a `;key=value` run carrying one? */ +function isCredentialQueryPair(pair: string): boolean { + const eq = pair.indexOf('='); + if (eq <= 0 || eq === pair.length - 1) return false; + let key = pair.slice(0, eq); + try { + key = decodeURIComponent(key.replace(/\+/g, ' ')); + } catch { + /* keep the raw key */ + } + if (CREDENTIAL_PARAMETER_NAMES.has(key.trim().toLowerCase())) return true; + // `token=a;b` is ONE pair (the `;` is the value's); `mode=ro;password=p` + // carries a credential in its `;` run — both are credential material. + return isCredentialShapedConfigKey(key) || semicolonSegments(pair).some(isCredentialSegment); +} + +/** The query (`?…`, up to `#`) of a URL, split into its `&` pairs; `[]` when there is none. */ +function queryPairs(value: string, url: UrlLayout): string[] { + if (value[url.queryStart] !== '?') return []; + return value.slice(url.queryStart + 1, url.hash === -1 ? value.length : url.hash).split('&'); +} + +/** The fragment (`#…`) of a URL, split into its `&` pairs (`#access_token=…&state=…`); `[]` when there is none. */ +function fragmentPairs(value: string, url: UrlLayout): string[] { + return url.hash === -1 ? [] : value.slice(url.hash + 1).split('&'); +} + +/** + * A form-encoded string (`a=b&pass=c`): no whitespace, an `&`, and an `=`; + * a leading `?` is allowed. `undefined` when the string is not one. + */ +function formPairs(value: string): string[] | undefined { + if (!value.includes('&') || !value.includes('=') || /\s/.test(value)) return undefined; + return (value.startsWith('?') ? value.slice(1) : value).split('&'); +} + +/** A header-name token (RFC 9110 `token`). */ +const HEADER_NAME_RE = /^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/; + +/** The name of a `Name: value` header line whose name is credential-shaped and whose value is non-empty, or `undefined`. */ +function credentialHeaderLineName(line: string): string | undefined { + const colon = line.indexOf(':'); + if (colon <= 0) return undefined; + const name = line.slice(0, colon).trim(); + if (!HEADER_NAME_RE.test(name)) return undefined; + if (line.slice(colon + 1).trim() === '') return undefined; + return isCredentialShapedConfigKey(name) ? name : undefined; +} + +/** The lines of a string, each without its line terminator. */ +const linesOf = (value: string): string[] => value.split('\n').map((line) => (line.endsWith('\r') ? line.slice(0, -1) : line)); + +/** Does a key/value connection string (libpq, `;`-delimited, or form-encoded) carry a credential, and in which form? */ +function keyValueCredentialOf(value: string): string | undefined { + if (!value.includes('=')) return undefined; + // Every reading is judged: a libpq value may hold an unquoted `;` + // (`host=h password=a;b`), and a `;` string reads as one libpq pair whose + // value runs to the next space. Any finding is a finding. + if (libpqPairs(value).some(isCredentialLibpqPair)) { + return 'a credential keyword inside a keyword/value connection string'; + } + if (semicolonSegments(value).some(isCredentialSegment)) return 'a credential segment inside a connection string'; + if (formPairs(value)?.some(isCredentialQueryPair)) return 'a credential parameter inside a form-encoded string'; + return undefined; +} + +/** The form-encoded reading's inverse: the credential pairs removed. */ +function redactFormPairs(value: string): string { + const form = formPairs(value); + if (!form?.some(isCredentialQueryPair)) return value; + return `${value.startsWith('?') ? '?' : ''}${form.filter((pair) => !isCredentialQueryPair(pair)).join('&')}`; +} + +/** + * {@link keyValueCredentialOf}'s inverse: every reading's credentials removed. + * The reading that delimits the string is applied first, so a credential's + * own delimiter-shaped bytes go with it: the form reading for a string with + * no `;`, then the libpq reading when the string holds more than one + * whitespace-separated pair (`password=a;b host=h`), and the semicolon + * reading first otherwise (`Password=a b;Server=h`). + */ +function redactKeyValueCredentials(value: string): string { + let out = value.includes(';') ? value : redactFormPairs(value); + out = libpqPairs(out).length > 1 + ? redactSemicolonSegments(redactLibpqPairs(out)) + : redactLibpqPairs(redactSemicolonSegments(out)); + return redactFormPairs(out); +} + +/** A string that may be JSON-encoded: its first non-space character opens an object or an array. */ +const looksLikeJson = (value: string): boolean => /^\s*[[{]/.test(value); + +/** `value` parsed, when it is a JSON-encoded object or array; `undefined` otherwise. */ +function parsedJson(value: string): object | undefined { + if (!looksLikeJson(value)) return undefined; + try { + const parsed: unknown = JSON.parse(value); + return parsed && typeof parsed === 'object' ? parsed : undefined; + } catch { + return undefined; + } +} + +/** + * Longer than this, a string (or the UTF-8 text of bytes) is not read at all: + * it is judged credential material unread — refused at write, withheld whole + * on read — so an over-long value can neither cost the judgment time nor hide + * a credential from it. + */ +export const MAX_JUDGED_STRING_LENGTH = 64 * 1024; + +/** + * PEM private-key armour — `-----BEGIN PRIVATE KEY-----`, `RSA`, `EC`, `DSA`, + * `ENCRYPTED`, `OPENSSH` and `PGP … BLOCK` forms included. Linear: each + * candidate starts at a literal `-----BEGIN `, and its letter run ends at the + * next `-`. + */ +const PEM_PRIVATE_KEY_RE = /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z0-9 ]*-----/i; + +/** A PEM private-key block, from its BEGIN line to its END line — or to the end of the string when it has none. */ +const PEM_PRIVATE_KEY_BLOCK_RE = /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z0-9 ]*-----[\s\S]*?(?:-----END [A-Z0-9 ]*PRIVATE KEY[A-Z0-9 ]*-----|$)/gi; + +/** + * Is a string read for `Name: value` header lines? A multi-line string is; a + * single-line one only under a header-ish key (`headers: ['Authorization: …']`) + * — a one-line `description: 'Password: provided via the secret store'` is prose. + */ +const readsHeaderLines = (value: string, headerish: boolean): boolean => headerish || value.includes('\n'); + +/** {@link embeddedCredentialOf} at a walk depth — a JSON-encoded string is walked one level deeper. */ +function embeddedCredentialAt(value: string, depth: number, headerish = false): string | undefined { + if (typeof value !== 'string' || value === '') return undefined; + if (value.length > MAX_JUDGED_STRING_LENGTH) return `a value longer than ${MAX_JUDGED_STRING_LENGTH} characters, too long to judge`; + const json = parsedJson(value); + if (json !== undefined) { + if (depth > CONTRACTLESS_CREDENTIAL_WALK_DEPTH) return 'a JSON-encoded value nested too deeply to judge'; + return collectFindings(json, depth + 1).length > 0 ? 'credential material inside a JSON-encoded string' : undefined; + } + if (PEM_PRIVATE_KEY_RE.test(value)) return 'a PEM private key'; + const url = urlLayout(value); + if (url) { + if (url.password !== undefined) return 'a userinfo password inside a URL'; + if (url.tokenUsername) return 'a token-shaped userinfo username inside a URL'; + if (queryPairs(value, url).some(isCredentialQueryPair)) return 'a credential query parameter inside a URL'; + if (fragmentPairs(value, url).some(isCredentialQueryPair)) return 'a credential parameter in a URL fragment'; + if (url.tail !== -1 && semicolonSegments(value.slice(url.tail + 1, url.queryStart)).some(isCredentialSegment)) { + return "a credential property in a URL's `;key=value` tail"; + } + return undefined; + } + if (ORACLE_THIN_USERINFO_RE.test(value)) return 'a userinfo password (`user/password@host`)'; + if (schemelessUserinfoAt(value) !== -1) return 'a userinfo password (`user:password@host`)'; + if (readsHeaderLines(value, headerish) && linesOf(value).some((line) => credentialHeaderLineName(line) !== undefined)) { + return 'a credential header line (`Name: value`)'; + } + return keyValueCredentialOf(value); +} + +/** + * The credential a string value carries EMBEDDED, named for the author, or + * `undefined` when it carries none. Judged shapes: + * + * - a JSON-encoded object or array (the string starts with `{` or `[`): it is + * parsed and walked exactly as a config is ({@link findContractlessCredentials}); + * - a URL userinfo password (`scheme://user:pass@host`, `//user:pass@host`, + * `jdbc:mysql://user:pass@host`), a userinfo username with no password that + * looks like a token ({@link looksLikeSecretValue}: `https://ghp_…@host`), a + * URL query or fragment pair whose name is credential-shaped (`?password=`, + * `?api_key=`, `#access_token=`) or is `sig`, or whose `;key=value` run + * carries one; + * - a `;key=value` property tail after a URL's authority + * (`sqlserver://h;user=u;password=p`); + * - the Oracle thin-driver userinfo (`jdbc:oracle:thin:user/pass@host`); + * - a scheme-less userinfo password (`user:pass@host/db`); + * - a `Name: value` header line (one per line) whose name is + * credential-shaped (`Authorization: Bearer …`) — see below for when a + * string is read for header lines; + * - a libpq keyword/value pair (`host=h password=p`), found leniently; + * - a semicolon-delimited connection-string segment (`Server=h;Password=p`); + * - a form-encoded pair (`a=b&pass=c`). + * + * - PEM private-key armour (`-----BEGIN … PRIVATE KEY-----`), anywhere in it. + * + * A string longer than {@link MAX_JUDGED_STRING_LENGTH} is not read: it is + * judged credential material. A header line is read in a multi-line string, + * or in a single-line one only with {@link EmbeddedCredentialOptions.headerish}. + * A libpq or segment value starting with a SQL bind placeholder (`$1`, `?`, + * `:name`) is none; an opaque URI scheme prefix (`mailto:`, `sip:`, …) is not + * a userinfo username, and a time of day (`12:30@`) is no userinfo. + * Every key inside a string is judged by {@link isCredentialShapedConfigKey}, + * and only a NON-EMPTY credential counts. + */ +export function embeddedCredentialOf(value: string, options?: EmbeddedCredentialOptions): string | undefined { + return embeddedCredentialAt(value, 0, options?.headerish === true); +} + +/** How {@link embeddedCredentialOf} and {@link redactEmbeddedCredentials} read a string. */ +export interface EmbeddedCredentialOptions { + /** + * The string sits under a header-ish key (one of whose words is `header` or + * `headers`), so a single-line string is read as a `Name: value` header line + * too — a multi-line string always is. A finding's + * {@link ContractlessCredentialFinding.headerish} carries this. + */ + headerish?: boolean; +} + +/** + * The credential-shaped segment keys of a connection string — semicolon, + * libpq, or a URL's `;key=value` tail — with a non-empty value, in order and + * without repeats (`Server=h;Password=p` → `['Password']`). + */ +export function connectionStringCredentialKeys(value: string): string[] { + if (typeof value !== 'string' || !value.includes('=')) return []; + const url = urlLayout(value); + const keys: string[] = []; + const add = (key: string) => { + if (!keys.includes(key)) keys.push(key); + }; + const fromSegments = (text: string) => { + for (const segment of semicolonSegments(text)) if (isCredentialSegment(segment)) add((segment.key as string).trim()); + }; + if (url) { + if (url.tail !== -1) fromSegments(value.slice(url.tail + 1, url.queryStart)); + return keys; + } + for (const pair of libpqPairs(value)) if (isCredentialLibpqPair(pair)) add(pair.key); + fromSegments(value); + return keys; +} + +/** One rewrite pass of {@link redactEmbeddedCredentials}. */ +function redactEmbeddedOnce(input: string, depth: number, headerish: boolean): string { + if (input.length > MAX_JUDGED_STRING_LENGTH) return ''; + const json = parsedJson(input); + if (json !== undefined) { + if (depth > CONTRACTLESS_CREDENTIAL_WALK_DEPTH) return ''; + return JSON.stringify(withholdFindings(json, collectFindings(json, depth + 1), depth + 1)); + } + // A PEM private key goes whole, armour included; the rest of the string is + // judged as it would be without it. + const value = PEM_PRIVATE_KEY_RE.test(input) ? input.replace(PEM_PRIVATE_KEY_BLOCK_RE, '') : input; + const url = urlLayout(value); + if (url) { + const headEnd = url.tail === -1 ? url.queryStart : url.tail; + let out: string; + if (url.password !== undefined) { + out = `${value.slice(0, url.authorityStart + value.slice(url.authorityStart, url.at).indexOf(':'))}@${value.slice(url.at + 1, headEnd)}`; + } else if (url.tokenUsername) { + out = `${value.slice(0, url.authorityStart)}${value.slice(url.at + 1, headEnd)}`; + } else { + out = value.slice(0, headEnd); + } + if (url.tail !== -1) { + const rest = redactSemicolonSegments(value.slice(url.tail + 1, url.queryStart)); + if (rest !== '') out += `;${rest}`; + } + if (value[url.queryStart] === '?') { + const kept = queryPairs(value, url).filter((pair) => !isCredentialQueryPair(pair)); + if (kept.length > 0) out += `?${kept.join('&')}`; + } else if (url.hash === -1) { + out += value.slice(url.queryStart); + } + if (url.hash !== -1) { + const fragment = fragmentPairs(value, url); + if (!fragment.some(isCredentialQueryPair)) out += value.slice(url.hash); + else { + const kept = fragment.filter((pair) => !isCredentialQueryPair(pair)); + if (kept.length > 0) out += `#${kept.join('&')}`; + } + } + return out; + } + if (ORACLE_THIN_USERINFO_RE.test(value)) return value.replace(ORACLE_THIN_USERINFO_RE, '$1$2@'); + const userinfoAt = schemelessUserinfoAt(value); + if (userinfoAt !== -1) return value.slice(0, userinfoAt) + value.slice(userinfoAt).replace(SCHEMELESS_USERINFO_RE, '$1@'); + let out = value; + if (readsHeaderLines(out, headerish) && linesOf(out).some((line) => credentialHeaderLineName(line) !== undefined)) { + out = out + .split('\n') + .map((line) => { + const cr = line.endsWith('\r') ? '\r' : ''; + const name = credentialHeaderLineName(cr ? line.slice(0, -1) : line); + return name === undefined ? line : `${name}:${cr}`; + }) + .join('\n'); + } + return redactKeyValueCredentials(out); +} + +/** {@link redactEmbeddedCredentials} at a walk depth (see {@link embeddedCredentialAt}). */ +function redactEmbeddedAt(value: string, depth: number, headerish = false): string { + if (embeddedCredentialAt(value, depth, headerish) === undefined) return value; + const out = redactEmbeddedOnce(value, depth, headerish); + // The invariant the read door rests on: what it serves carries no + // credential. A string the rewrite cannot clear is withheld whole. + return embeddedCredentialAt(out, depth, headerish) === undefined ? out : ''; +} + +/** + * {@link embeddedCredentialOf}'s inverse for the read path: the string with + * every embedded credential removed and everything else kept — the URL's + * username and host (`user@host`), the non-credential segments, pairs, query + * and fragment parameters, a header line's name, a JSON-encoded value's + * non-credential members. Dropped, not masked: a mask would round-trip back + * as a literal new credential. Returns the input unchanged when it carries + * none; what it returns never carries one ({@link embeddedCredentialOf} + * answers `undefined` for it) — a string the rewrite cannot clear comes back + * empty. + */ +export function redactEmbeddedCredentials(value: string, options?: EmbeddedCredentialOptions): string { + return redactEmbeddedAt(value, 0, options?.headerish === true); +} + +// --------------------------------------------------------------------------- +// The one walk both doors read +// --------------------------------------------------------------------------- + +/** A position in a contractless driver's `config` that holds credential material. */ +export interface ContractlessCredentialFinding { + /** Segments from the config root; an array element's index is its decimal string. */ + path: readonly string[]; + /** + * `named` — the value sits under a credential position (a credential-shaped + * key, a leaf inside a credential-shaped object, a header pair's or tuple's + * value, binary data there or carrying an embedded credential) and is + * withheld whole; `embedded` — a string carrying a credential + * ({@link embeddedCredentialOf}), rewritten without it; `depth` — a subtree + * too deep to judge, and `opaque` — a `Map` or `Set`, whose entries the walk + * does not read: both doors treat these as credential material rather than + * skip them. + */ + kind: 'named' | 'embedded' | 'depth' | 'opaque'; + /** The value found there, as stored. */ + value: unknown; + /** For `embedded`, what was found, for the author. */ + what?: string; + /** + * For `embedded`: `true` when the string sits under a header-ish key and was + * read as a single-line `Name: value` header line too — hand it to + * {@link embeddedCredentialOf} as {@link EmbeddedCredentialOptions.headerish} + * to judge the string the same way. + */ + headerish?: true; +} + +/** Deeper than this, a subtree is a {@link ContractlessCredentialFinding} of kind `depth`. */ +export const CONTRACTLESS_CREDENTIAL_WALK_DEPTH = 16; + +/** The keys that label a `{ name, value }` pair; EVERY one present is judged (`{ key: 'Authorization', value }`). */ +const PAIR_LABEL_KEYS = ['name', 'key', 'header', 'headerName'] as const; + +/** Is a value under a credential position credential material? Booleans, `null` and empties are not. */ +function holdsCredentialValue(value: unknown): boolean { + if (typeof value === 'string') return value !== ''; + if (typeof value === 'number' || typeof value === 'bigint') return true; + if (isBinary(value)) return binaryLength(value) > 0; + if (Array.isArray(value)) { + return value.some((element) => (element && typeof element === 'object' && !isBinary(element) ? false : holdsCredentialValue(element))); + } + return false; +} + +/** A `Buffer`, a typed array, a `DataView` or an `ArrayBuffer` — bytes, judged as ONE value. */ +const isBinary = (value: unknown): value is ArrayBufferView | ArrayBuffer => + !!value && typeof value === 'object' && (ArrayBuffer.isView(value) || value instanceof ArrayBuffer); + +const binaryLength = (value: ArrayBufferView | ArrayBuffer): number => value.byteLength; + +/** The bytes as UTF-8 text, for the embedded-credential judgment. */ +function binaryText(value: ArrayBufferView | ArrayBuffer): string { + const bytes = value instanceof ArrayBuffer ? new Uint8Array(value) : new Uint8Array(value.buffer, value.byteOffset, value.byteLength); + return new TextDecoder('utf-8', { fatal: false }).decode(bytes); +} + +/** + * Does a string under a bare `key` look like key material rather than a name? + * {@link looksLikeSecretValue}, or — wider, for a bare `key` only — at least + * 16 characters of hexadecimal holding a letter (`deadbeefcafebabe`), or + * digit-free text whose case flips like random text rather than camel case or + * a path — 30% to 70% of its letters upper-case and at least four + * lower-to-upper steps — that is either base64 (at least 16 characters of + * `A`–`Z`, `a`–`z`, `+`, `/`, up to two `=` of padding, a length divisible by + * four, and a `+`, a `/` or padding) or at least 24 letters (`-` and `_` + * allowed). + * `email`, `customerEmailAddress` and `XMLHttpRequestURLBuilder` do not. + */ +function looksLikeBareKeyMaterial(value: string): boolean { + if (looksLikeSecretValue(value)) return true; + if (value.length < 16) return false; + if (/^[0-9a-f]+$/i.test(value) && /[a-f]/i.test(value)) return true; + // Random text flips case like coin tosses; camel case and paths do not. + let upper = 0; + let letters = 0; + let lowerToUpper = 0; + let prev = ''; + for (const c of value) { + const isUpper = c >= 'A' && c <= 'Z'; + if (isUpper || (c >= 'a' && c <= 'z')) letters += 1; + if (isUpper) { + upper += 1; + if (prev >= 'a' && prev <= 'z') lowerToUpper += 1; + } + prev = c; + } + const ratio = letters === 0 ? 0 : upper / letters; + if (ratio < 0.3 || ratio > 0.7 || lowerToUpper < 4) return false; + if (/^[A-Za-z+/]+={0,2}$/.test(value) && value.length % 4 === 0 && /[+/=]/.test(value)) return true; + return value.length >= 24 && /^[A-Za-z_-]+$/.test(value); +} + +/** Does a value under a bare `key` look like key material rather than a name (`{ key: 'email' }`)? */ +function bareKeyHoldsSecret(value: unknown): boolean { + if (typeof value === 'string') return looksLikeBareKeyMaterial(value); + if (isBinary(value)) return binaryLength(value) > 0; + if (Array.isArray(value)) return value.some((element) => typeof element === 'string' && looksLikeBareKeyMaterial(element)); + return false; +} + +/** Where a node sits, for the judgment of its own key and leaves. */ +interface WalkContext { + /** Inside an object reached under a credential-shaped key — never reset by a descriptor key on the way down. */ + enclosing: boolean; + /** The key leaf values are judged by: an array element's is its array's key. */ + leafKey: string | undefined; + /** The key of the object (or of the array holding the object) the node sits in. */ + holderKey: string | undefined; + /** The node's object is an ELEMENT of a list (`headers: [{ key, value }]`), not a map (`headers: { key }`). */ + element: boolean; +} + +/** + * The positions of a list that hold a header's VALUE: + * - a `[name, value, …]` tuple whose name is credential-shaped, inside a list + * (two or more elements) or directly under a header-ish key (an odd number + * of elements, or two) — every element after the name; + * - a flat `[name, value, name, value]` list directly under a header-ish key + * (the raw-headers form) — the element after each credential-shaped name at + * an even position. + */ +function headerValuePositions(key: string | undefined, list: readonly unknown[]): Set { + const marked = new Set(); + const headerish = key !== undefined && isHeaderishKey(key); + if (key !== undefined && !headerish) return marked; + const first = list[0]; + // Directly under a header-ish key, an even-length list reads as pairs + // (`rawHeaders`), an odd-length one as a tuple. + const tuple = headerish ? list.length % 2 === 1 : list.length >= 2; + if (tuple && typeof first === 'string' && isCredentialShapedConfigKey(first)) { + for (let i = 1; i < list.length; i += 1) marked.add(i); + } + if (headerish) { + for (let i = 0; i + 1 < list.length; i += 2) { + const name = list[i]; + if (typeof name === 'string' && isCredentialShapedConfigKey(name)) marked.add(i + 1); + } + } + return marked; +} + +/** Every finding in a value, the root being an object or an array, starting at `depth`. */ +function collectFindings(root: unknown, depth: number): ContractlessCredentialFinding[] { + const out: ContractlessCredentialFinding[] = []; + + const visit = (key: string | undefined, value: unknown, path: string[], ctx: WalkContext, at: number): void => { + const leafKey = key ?? ctx.leafKey; + let named = key !== undefined && isCredentialShapedConfigKey(key); + // A bare `key` is credential-shaped only where it can mean key material: + // inside a credential-shaped or TLS holder, in a header MAP (`headers: + // { key: … }`, the header named `key` — not a list element, which is a + // pair's label: `headers: [{ key: 'Authorization' }]`), or holding a value + // that looks like key material — never `{ key: 'email' }`. + if (named && isBareKeyName(key as string) && !ctx.enclosing) { + const holder = ctx.holderKey; + named = (holder !== undefined + && (isCredentialShapedConfigKey(holder) || (isHeaderishKey(holder) && !ctx.element) || isKeyMaterialHolder(holder))) + || bareKeyHoldsSecret(value); + } + // The descriptor exemption applies to LEAF values only (a string, a + // number, bytes, a list of them); an object below a descriptor key keeps + // the enclosing credential context. + const leafCredential = named || (ctx.enclosing && !(leafKey !== undefined && isDescriptorLeafKey(leafKey))); + if (value === null || value === undefined || typeof value === 'boolean') return; + if (typeof value === 'string') { + if (leafCredential) { + if (value !== '') out.push({ path, kind: 'named', value }); + return; + } + const headerish = isHeaderishKey(leafKey) || isHeaderishKey(ctx.holderKey); + const what = embeddedCredentialAt(value, at, headerish); + if (what) out.push({ path, kind: 'embedded', value, what, ...(headerish ? { headerish: true as const } : {}) }); + return; + } + if (typeof value !== 'object') { + if (leafCredential && holdsCredentialValue(value)) out.push({ path, kind: 'named', value }); + return; + } + if (isBinary(value)) { + const judged = leafCredential + || binaryLength(value) > MAX_JUDGED_STRING_LENGTH + || embeddedCredentialAt(binaryText(value), at, isHeaderishKey(leafKey) || isHeaderishKey(ctx.holderKey)) !== undefined; + if (leafCredential ? binaryLength(value) > 0 : judged) { + out.push({ path, kind: 'named', value }); + } + return; + } + if (at > CONTRACTLESS_CREDENTIAL_WALK_DEPTH) { + out.push({ path, kind: 'depth', value }); + return; + } + if (value instanceof Map || value instanceof Set) { + out.push({ path, kind: 'opaque', value }); + return; + } + const enclosing = ctx.enclosing || named; + const holderKey = key ?? ctx.holderKey; + if (Array.isArray(value)) { + if (leafCredential && holdsCredentialValue(value)) { + out.push({ path, kind: 'named', value }); + return; + } + const marked = headerValuePositions(key, value); + value.forEach((element, index) => { + const elementPath = [...path, String(index)]; + if (marked.has(index)) { + if (holdsCredentialValue([element]) || (element && typeof element === 'object')) { + out.push({ path: elementPath, kind: 'named', value: element }); + } + return; + } + visit(undefined, element, elementPath, { enclosing, leafKey, holderKey, element: false }, at + 1); + }); + return; + } + walkObject(value as Record, path, { enclosing, leafKey: undefined, holderKey, element: key === undefined }, at + 1); + }; + + const walkObject = (node: Record, path: string[], ctx: WalkContext, at: number): void => { + const labels = PAIR_LABEL_KEYS.filter((k) => typeof node[k] === 'string'); + const isPair = labels.length > 0 && 'value' in node; + const pairCredential = isPair && labels.some((k) => isCredentialShapedConfigKey(node[k] as string)); + for (const [key, value] of Object.entries(node)) { + const childPath = [...path, key]; + if (isPair && (labels as readonly string[]).includes(key)) { + // A label names the pair; it is judged only for an embedded credential. + const headerish = isHeaderishKey(ctx.holderKey); + const what = embeddedCredentialAt(value as string, at, headerish); + if (what) out.push({ path: childPath, kind: 'embedded', value, what, ...(headerish ? { headerish: true as const } : {}) }); + continue; + } + if (isPair && key === 'value' && pairCredential) { + if (holdsCredentialValue(value) || (value && typeof value === 'object')) out.push({ path: childPath, kind: 'named', value }); + continue; + } + visit(key, value, childPath, ctx, at); + } + }; + + const top: WalkContext = { enclosing: false, leafKey: undefined, holderKey: undefined, element: false }; + if (Array.isArray(root)) visit(undefined, root, [], top, depth); + else if (root && typeof root === 'object' && !isBinary(root) && !(root instanceof Map) && !(root instanceof Set)) { + walkObject(root as Record, [], top, depth); + } + return out; +} + +/** + * Every position in a contractless driver's `config` that holds credential + * material — the ONE judgment the write door refuses and the read door + * withholds: + * + * - a value under a credential-shaped key ({@link isCredentialShapedConfigKey}): + * a non-empty string, a number, non-empty bytes (a `Buffer` or typed array, + * judged as ONE value), or an array holding a non-empty primitive + * (`apiKeys: ['k1', 'k2']`). A boolean is a flag, never a secret. A bare + * `key` counts only inside a credential-shaped, header-ish or TLS holder + * ({@link isKeyMaterialHolder}), or holding a value that looks like key + * material ({@link looksLikeBareKeyMaterial}); + * - inside an object reached under a credential-shaped key (`credentials`, + * `auth`), every leaf except a descriptor or identity (`type`, `clientId`, + * `user`, …) — an OBJECT below a descriptor key stays inside that context; + * - the `value` of a `{ name, value }` pair whose label — ANY of `name`, + * `key`, `header`, `headerName` — is credential-shaped: a headers list + * carrying `Authorization`, `X-API-Key` or `Cookie`; + * - every element after the name of a `[name, value, …]` tuple whose name is + * credential-shaped, inside a list or directly under a header-ish key, and + * the element after each credential-shaped name at an even position of a + * flat list directly under a header-ish key (`rawHeaders`); + * - a string carrying an embedded credential ({@link embeddedCredentialOf}), + * wherever it sits — array elements and pair labels included — and bytes + * whose UTF-8 text carries one; + * - a subtree past {@link CONTRACTLESS_CREDENTIAL_WALK_DEPTH}, and a `Map` or + * `Set` anywhere, judged whole. + * + * The walk enters the object elements of arrays (`servers: [{ host, password }]`) + * and judges them by key; plain row data with no credential-shaped key is not + * a finding. + */ +export function findContractlessCredentials(config: unknown): ContractlessCredentialFinding[] { + if (!config || typeof config !== 'object' || Array.isArray(config)) return []; + return collectFindings(config, 0); +} + +/** + * Withhold `findings` from `root` IN PLACE (the caller owns `root`): a string + * finding of kind `embedded` is rewritten without its credential, every other + * finding is dropped — a key deleted, an array element spliced only from the + * END of its array (so no sibling shifts and a write-path inverse restores each + * withheld element at its own index), nulled when siblings follow it. Returns + * `root`. + */ +function withholdFindings(root: T, findings: readonly ContractlessCredentialFinding[], depth: number): T { + const parentOf = (path: readonly string[]): unknown => { + let node: unknown = root; + for (const segment of path.slice(0, -1)) { + if (!node || typeof node !== 'object') return undefined; + node = (node as Record)[segment]; + } + return node; + }; + for (const finding of findings) { + if (finding.kind !== 'embedded') continue; + const parent = parentOf(finding.path) as Record | undefined; + const leaf = finding.path[finding.path.length - 1] as string; + if (parent && typeof parent[leaf] === 'string') { + parent[leaf] = redactEmbeddedAt(parent[leaf] as string, depth + Math.max(0, finding.path.length - 1), finding.headerish === true); + } + } + // Drops deepest-first and, inside one array, highest index first — so a + // drop never shifts a position still to be visited. + const drops = findings + .filter((finding) => finding.kind !== 'embedded') + .sort((a, b) => { + if (a.path.length !== b.path.length) return b.path.length - a.path.length; + return Number(b.path[b.path.length - 1]) - Number(a.path[a.path.length - 1]) || 0; + }); + for (const finding of drops) { + const parent = parentOf(finding.path); + const leaf = finding.path[finding.path.length - 1] as string; + if (Array.isArray(parent)) { + if (Number(leaf) === parent.length - 1) parent.splice(Number(leaf), 1); + else parent[Number(leaf)] = null; + } else if (parent && typeof parent === 'object' && !isBinary(parent)) { + delete (parent as Record)[leaf]; + } + } + return root; +} + +/** + * The read half of the contractless-driver judgment: `config` with every + * position {@link findContractlessCredentials} reports withheld — the SAME + * walk the write door refuses by — and those positions, sorted. A value under + * a credential position, a subtree too deep to judge, and a `Map` or `Set` are + * dropped (an array element is spliced from the end of its array, or nulled + * when siblings follow it); a string with an embedded credential is rewritten + * without it ({@link redactEmbeddedCredentials}). Every position is reported, + * array indices included, so a write-path inverse can restore exactly what was + * withheld. The projection is judged again until it holds no finding — what + * is served is what an untouched Save hands the write door — and one that has + * not settled after {@link MAX_WITHHOLD_PASSES} passes is withheld whole. + * Pure: the input is never mutated, and returned by reference when nothing is + * withheld. + */ +export function withholdContractlessCredentials(config: Record): { + config: Record; + paths: (readonly string[])[]; +} { + let findings = findContractlessCredentials(config); + if (findings.length === 0) return { config, paths: [] }; + let out = structuredClone(config); + const seen = new Map(); + // What is served must itself hold no finding — it is what an untouched Save + // hands the write door. A withheld position can leave a sibling that reads + // as credential material on its own (a pair's `key` label without its + // `value`, directly under a header-ish key), so the projection is judged + // again until it is clean; each pass only removes, and a projection that + // will not settle is withheld whole. + for (let pass = 0; findings.length > 0; pass += 1) { + for (const finding of findings) seen.set(JSON.stringify(finding.path), finding.path); + if (pass === MAX_WITHHOLD_PASSES) { + for (const key of Object.keys(out)) seen.set(JSON.stringify([key]), [key]); + out = {}; + break; + } + out = withholdFindings(out, findings, 0); + findings = findContractlessCredentials(out); + } + const paths = [...seen.values()].sort((a, b) => (a.join('.') < b.join('.') ? -1 : a.join('.') > b.join('.') ? 1 : 0)); + return { config: out, paths }; +} + +/** Past this many passes, a projection that still holds a finding is withheld whole (see {@link withholdContractlessCredentials}). */ +const MAX_WITHHOLD_PASSES = 8; diff --git a/packages/spec/src/data/driver/index.ts b/packages/spec/src/data/driver/index.ts index eca601d1fbb..0c8f6eda020 100644 --- a/packages/spec/src/data/driver/index.ts +++ b/packages/spec/src/data/driver/index.ts @@ -12,6 +12,7 @@ */ export * from './common.zod'; +export * from './contractless-credentials'; export * from './config-registry.zod'; export * from './memory.zod'; export * from './mongo.zod'; diff --git a/scripts/adr-anchors/packages__spec__src__data__datasource.zod.ts.json b/scripts/adr-anchors/packages__spec__src__data__datasource.zod.ts.json new file mode 100644 index 00000000000..a933fc7bb11 --- /dev/null +++ b/scripts/adr-anchors/packages__spec__src__data__datasource.zod.ts.json @@ -0,0 +1,8 @@ +{ + "file": "packages/spec/src/data/datasource.zod.ts", + "adrs": [ + "ADR-0015", + "ADR-0062" + ], + "invariant": "Credentials never appear in metadata artefacts (ADR-0015 section 10), for a driver the platform ships no config contract for as much as for one it does: such a datasource's config shape stays unjudged, but a credential-shaped key or an embedded credential (URL userinfo password, credential query parameter, a Password= connection-string segment) is refused at publish by the same name predicate the read-path redactor uses. The remedy is the bound secret (external.credentialsRef), which ADR-0062 D3 resolves at connect and hands to the driver factory; routing the value into the secret store silently is not done, because nothing says which key a contractless driver's factory reads its credential from." +}