Repository navigation
Restore S3 carver wiring and reject the unimplemented local carver - #1095
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restore S3 carver wiring and reject the unimplemented
localcarverProblem
File carving has three carver types (
--carver-type=db|s3|local), but onlydbworked:carves.CreateCarverS3.carvers3was declared but never set, so with--carver-type=s3every carve block failed withs3 carver not initialized. Before that,InitateBlockdereferenced the nil carver.nil, so archiving or downloading an S3 carve dereferenced a nilh.Carves.S3.localnever worked.CreateBlockhad nolocalcase in any version, so every block failed withunknown carver, andcarver.local.carvesDirwas never read. History showslocalwas only ever added to the valid-types maps.carvesDirwas osctrl-admin's archive staging folder, which went away when osctrl-admin was removed.carver:section crashed startup. The carver was built fromflagParams.Carver.Typebefore the service config was resolved, so a YAML file without that section caused a nil-pointer crash.CreateCarverS3always installed static credentials, even empty ones. That breaks the instance/task-role setup the sample config recommends.Changes
New
carves.CreateFileCarvesFromConfig: the single place where the carver is built from config. Both osctrl-tls and osctrl-api call it afterserviceConfigMgr.Resolve, so carver settings saved in the DB apply. Startup fails with a clear error when:local;s3andcarver.s3.bucketorcarver.s3.regionis missing;accessKey/secretAccessKeyis set.A missing
carver:section defaults todb.config.ValidateCarverType: shared type check, also used when osctrl-tls loads its YAML. Thelocalerror reads:carver type "local" is not supported; use "db" (blocks stored in the database) or "s3".CreateCarverS3: uses static keys only when they're configured; otherwise it uses the default AWS credential chain (env, shared config,credential_process, IAM Roles Anywhere, web identity, instance/task role). This matches the S3 logger fix in S3 logger: use the default AWS credential chain when no static keys are configured #1094.InitateBlock/Archive: returns3 carver not initializedinstead of crashing when the carver iss3but no S3 client exists. Removed the deadlocalbranch inArchive.Docs: removed
local(and the undocumentednone) from the sampletls.yml/api.ymland from the service-config UI help. Kept--carver-local-dir/CARVER_LOCAL_DIRand theLocalCarverstruct so existing command lines and YAML still parse; the flag's help now says it's unused.Why reject
localinstead of implementing itIt has never worked, and making it work means osctrl-tls and osctrl-api must share a filesystem: TLS writes the blocks and API archives and serves them. The usual deployment runs them on different hosts or containers.
dbalready covers keeping data local, ands3covers external storage.Security / operational impact
localconfigs: any deployment withcarver.type: local(orCARVER_TYPE=local) now refuses to start instead of silently failing every carve. The fix is to switch todbors3. The same applies to alocalcarver type saved in the service-config DB.carver.type: s3and missing bucket or region now fail at startup instead of failing per block.s3:PutObjectands3:GetObjecton the bucket's objects.UploadPartCopy, so every block except the last must be at least 5 MiB on real S3. Environments using the S3 carver needcarver_block_sizeof 5 MiB or more.Validation
localaccepted).pkg/carves/carver_test.go:cmd/tls/carver_test.go,cmd/api/carver_test.go: YAML load → service parameters → carver, fordb,s3, incompletes3andlocal.pkg/config/validation_test.go:localrejected when osctrl-tls validates its YAML.go test ./...,go vet ./...,gofmt -l,golangci-lint run ./...(0 issues),make openapi-check: all pass, on top ofdevelopat 81d10eb.npm run check/ Vitest (nonode_modulesin this worktree). The frontend change is two help strings inServiceConfigPage.tsx.