[9/10] libs/libc/elf: Load the libraries a module names in DT_NEEDED

[casaroli]

Summary

[8/10] #20131 is merged, so an FDPIC module loads and runs. A module that names a shared library in DT_NEEDED still has nothing to load it with, and this loads it and binds the module's imports against it.

libelf_insert() does the loading, which is what dlopen() calls anyway: the library lands in the module registry like anything else, its exports come back through libelf_getsymbol(), which is the call dlsym() uses, and a library named by two modules is loaded once. A bare name is looked for along LD_LIBRARY_PATH, where dlopen() looks for it. Undefined symbols resolve against the globally registered symbols first, then the modules this one depends on, then the table exec() supplied. Nothing here calls into dlfcn, because this loader is also the kernel's module loader, which has none.

Six things had to be fixed to make it work, none of which a build shows.

reldata was a file-scope global. Opening a library from inside libelf_relocatedyn() makes that function reentrant, so the nested load overwrote the outer one's relocation offsets and the module resumed binding with the library's DT_REL. It is per call now.

A cross-object call needs the callee's data base, not the caller's. A symbol resolved from an FDPIC library comes back as a descriptor, and R_ARM_FUNCDESC_VALUE was treating it as a code address and pairing it with the importing module's GOT. It copies both words now, so the library runs with its own.

An object with no imports has no PLT and so no DT_PLTGOT, but it still has a GOT and still has to be entered with it. Without the fallback its descriptors carried a data base of zero and the library read its globals through a null pointer.

R_ARM_FUNCDESC, a pointer to a descriptor, wrapped a library's descriptor in a second one. It now stores the library's descriptor as it is.

The flag that says a resolved value is a descriptor was set only for an import and never cleared, so the next relocation against a symbol of the module itself took that symbol for a descriptor too. It is cleared there.

libelf_symname() was static, and reading a DT_NEEDED name needs it.

Impact

Nothing happens without CONFIG_LIBC_DLFCN: a module with DT_NEEDED is refused there, as before, since there is no way to load what it asks for.

With CONFIG_ARCH_ADDRENV a module with DT_NEEDED is refused too, as before: the program runs in its own address space, which a library libelf_insert() loads cannot reach. dlopen() is still a stub in kernel builds, so neither path exists there yet.

A DT_NEEDED library is one shared instance, its data included, because the loader returns the object already in the registry. A module started with exec() is different: that path loads the module afresh each time, so two running instances have separate data while sharing one copy of the text.

Each library becomes one of the module's dependencies[], and the dependency holds it in place of the reference libelf_insert() took. A library loaded only for DT_NEEDED is kept by the modules that depend on it, and libelf_undepend() unloads it with the last of them; one that dlopen() or insmod also opened stays until that reference goes too. CONFIG_LIBC_ELF_MAXDEPEND bounds how many libraries one module may name, which is what it already meant.

Testing

mps3-an547:picostest builds three ways on master: with CONFIG_FDPIC off, with it on, and with it on plus CONFIG_LIBC_DLFCN, which is the path this patch adds. With CONFIG_FDPIC on, the applications the configuration carries are FDPIC objects (OS/ABI: ARM FDPIC).

mps2-an500:xipfs with CONFIG_FDPIC under QEMU, xipfs_test fdpic, with the module fixtures rebuilt by the tree's module flags (apache/nuttx-apps#3762) and #20388: 34 passed, 0 failed. Without the last two fixes the C++ library tests crash.

qemu-armv8a:knsh (CONFIG_ARCH_ADDRENV) links with -Werror; in a flat build elf_bind.o is unchanged by the ADDRENV guard.

tools/checkpatch.sh -c -u -m -g passes.

Review

Everything below this in the series is merged, so it is one commit on master.

[github-actions]

MemBrowse Memory Report

s698pm-dkit

• Code: .text +3,584 B (+1.0%, 372,240 B)
• Data: .bss -32 B (-0.3%, 11,819 B)
  No memory changes detected for:
• arduino-mega2560
• esp32-devkitc
• hifive1-revb
• mirtoo
• qemu-armv8a
• qemu-intel64
• rx65n-rsk2mb
• stm32-nucleo-f103rb

[xiaoxiang781216]

but still can reach as per task list, so let's remove LIBELF_NEEDED and use CONFIG_LIBC_ELF_MAXDEPEND directly

[casaroli]

Without this guard qemu-armv8a:knsh does not link. We could fix ld-kernel.script for qemu-armv8a so that it links, but DT_NEEDED would still not work: libelf_insert() loads the library into kernel memory, which the task's address environment does not map.

Ideally the library is loaded only once and mapped into each task that needs it. That needs reference counting across tasks and a much larger design change, so I would like to do it in follow-up PRs. The other option is to load the library into the task's own address space, but then a library used by several tasks is copied into every one of them.