Skip to content

ebpftracer: report missing BPF tracing program types clearly - #367

Merged
def merged 1 commit into
coroot:mainfrom
KR-Ravindra:fix/bpf-tracing-program-type-check
Sep 24, 2026
Merged

def merged 1 commit into
coroot:mainfrom
KR-Ravindra:fix/bpf-tracing-program-type-check

Conversation

@KR-Ravindra

@KR-Ravindra KR-Ravindra commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem

On kernels built without CONFIG_BPF_EVENTS (for example NVIDIA JetPack 5 / L4T 5.10.120-tegra, where CONFIG_KPROBES=n and CONFIG_UPROBE_EVENTS=n), the agent passes every pre-flight check (tracefs present, kernel version >= 5.1, collection spec loads) and then exits with:

failed to load collection: program sched_process_exit: load program: invalid argument

Nothing in the message points at the kernel configuration, the program name varies between runs (it is whichever program cilium/ebpf loaded first), and the *ebpf.VerifierError branch prints nothing because this is not a verifier error.

Root cause

ebpf.NewCollectionWithOptions in ebpftracer/tracer.go receives a bare EINVAL from bpf(BPF_PROG_LOAD). With CONFIG_BPF_EVENTS off, BPF_PROG_TYPE_TRACEPOINT and BPF_PROG_TYPE_KPROBE are compiled out of the kernel and find_prog_type() returns -EINVAL. Every program in the agent's collection is a tracepoint/, kprobe/ or uprobe/ program (uprobes are BPF_PROG_TYPE_KPROBE), so nothing can load, but the agent has no check that names this condition.

Fix

  • ebpftracer/tracer.go: in (*Tracer).ebpf, right before loading the collection, probe ebpf.TracePoint and ebpf.Kprobe with features.HaveProgramType from the already-required github.com/cilium/ebpf module. On a conclusive ebpf.ErrNotSupported the agent fails with:

    kernel does not support BPF TracePoint programs (CONFIG_BPF_EVENTS is not set?): not supported
    

    Any other probe result (for example EPERM under kernel lockdown) is ignored so the real collection load surfaces the same error it does today. Six added lines, inlined as suggested in review; no helper, no extra comments, no tests.

  • README.md: one line stating that the kernel must be built with CONFIG_BPF_EVENTS=y.

No new dependencies; go.mod/go.sum are unchanged.

How tested

Cross-compiled for linux/amd64: gofmt -l . clean, go vet ./ebpftracer, go build ./ebpftracer and go test -c ./ebpftracer all OK. I have not been able to run the changed binary on the affected Tegra kernel from this environment, so the end-to-end message on real hardware is worth a check by anyone who has one.

Links

This change was prepared with an AI agent operated by KR-Ravindra, who reviewed and tested it.

@KR-Ravindra
KR-Ravindra force-pushed the fix/bpf-tracing-program-type-check branch from 0129a76 to 3f9d9d5 Compare September 8, 2026 06:53
@KR-Ravindra

Copy link
Copy Markdown
Contributor Author

Self-review before marking ready. The agent now probes the tracing program types it needs before loading and fails with a message naming the likely kernel option instead of a bare EINVAL; only a conclusive not-supported result short-circuits, so lockdown and permission errors still take the existing path. Unit test pins the exact message; vet, tests and build pass in a golang:1.24 container with libsystemd-dev.

@KR-Ravindra
KR-Ravindra marked this pull request as ready for review September 8, 2026 06:58
@KR-Ravindra

Copy link
Copy Markdown
Contributor Author

Round 1 self-review.

Checked features.HaveProgramType in cilium/ebpf v0.20.0: probeProgram maps EINVAL/E2BIG from BPF_PROG_LOAD to ebpf.ErrNotSupported and returns anything else (for example EPERM) unchanged, so the conclusive/inconclusive split in checkProgramTypes matches the library. On a kernel with CONFIG_BPF_EVENTS=y both probes return nil and the load path is unchanged. go vet ./ebpftracer/, go mod tidy (no diff) and TestCheckProgramTypes pass here. Marking ready for review.

@def

def commented Sep 21, 2026

Copy link
Copy Markdown
Member

@KR-Ravindra, in general, I like the idea of logging something more meaningful than the current fatal error. But can we keep the code changes to a minimum and just inline something like this directly into the .ebpf method?

for _, pt := range []ebpf.ProgramType{ebpf.TracePoint, ebpf.Kprobe} {
	if err := features.HaveProgramType(pt); errors.Is(err, ebpf.ErrNotSupported) {
		return fmt.Errorf("kernel does not support BPF %s programs (CONFIG_BPF_EVENTS is not set?): %w", pt, ebpf.ErrNotSupported)
	}
}

I also don't think we need additional code comments or tests for this.

On kernels built without CONFIG_BPF_EVENTS (e.g. NVIDIA JetPack 5 / L4T
5.10 with CONFIG_KPROBES=n), BPF_PROG_TYPE_TRACEPOINT and
BPF_PROG_TYPE_KPROBE are compiled out and bpf(BPF_PROG_LOAD) fails with a
bare EINVAL. The agent then exits with

  failed to load collection: program sched_process_exit: load program: invalid argument

where the program name is whichever program happened to load first, and
nothing points at the kernel configuration.

Probe TracePoint and Kprobe support with cilium/ebpf/features before
loading the collection and fail with an error naming the missing kernel
option. Only a conclusive ebpf.ErrNotSupported is reported; any other
probe result falls through to the real collection load so existing error
paths are unchanged.

Document the CONFIG_BPF_EVENTS requirement in the README.
@KR-Ravindra
KR-Ravindra force-pushed the fix/bpf-tracing-program-type-check branch from 3f9d9d5 to f76895d Compare September 21, 2026 16:20
@KR-Ravindra

Copy link
Copy Markdown
Contributor Author

@def done in f76895d: the check is now the six-line loop inlined in (*Tracer).ebpf right before NewCollectionWithOptions, and the helper, its comments and the test file are gone. I kept the one-line README note about CONFIG_BPF_EVENTS; happy to drop it too if you prefer. Could you take another look?

Drafted with AI assistance and checked against the code before posting.

@def
def merged commit f8a2790 into coroot:main Sep 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unhelpful fatal error on kernels built without BPF tracing program types: failed to load collection: program <name>: load program: invalid argument

2 participants