Support littlefs as the CIRCUITPY filesystem

[tannewt]

Add a supervisor-level filesystem API (supervisor_vfs_*) that works on both FAT and littlefs mounts so workflows, BLE file transfer, the web workflow and settings don't care which filesystem is active. The root filesystem storage becomes a supervisor_vfs_t union of fs_user_mount_t (FAT) and mp_obj_vfs_lfs2_t (littlefs); the root is mounted as littlefs when CIRCUITPY_FILESYSTEM_LITTLEFS is set.

• extmod: export mp_obj_vfs_lfs2_t and add mp_vfs_lfs2_mount_supervisor() so the supervisor can mount littlefs before the VM and GC run, and make the littlefs VFS honor the supervisor write-protection flags so storage.remount() applies to it like FAT.
• supervisor: map FatFS and littlefs errors to shared supervisor_fs_err_t codes, carry mtimes as ns past 1970 and stamp littlefs mtime attributes and FAT timestamps from the same value.
• workflow/USB/BLE/web code switch from FRESULT to supervisor_fs_err_t and use the new API, removing direct FatFS access.
• zephyr-cp: pick FAT or littlefs per board from the partition layout (a littlefs_partition devicetree node selects littlefs), add a native_sim_lfs test harness, fix the flash write cache stale-read when the target block is in the currently cached page, and provide mp_hal_time_ns().

[dhalbert]

I was looking at how micropython supports this, and avoids calling f_read(), etc. It seems like it uses the StreamIO mp_stream_p_t operations, so it can work with any kind of filesystem. That's what's done in extmod/modjson.c, etc. For opening a file, mounting, etc., there are operations in extmod/vfs.c that we use too.

So do you need the extra API for file operations in fileystem.c? Could you use those existing other operations and types?

I also see leftover FAT operations in a couple of places. I found these by hand, and there may be more:

• a bunch in ports/espressif/boards/mixgo_ce_serial/board.c
• f_read() in shared-bindings/synthio/__init__.c

[bablokb]

Yes, others are:

grep -Elr "f_read|f_lseek|f_rewind" shared*
shared-bindings/synthio/__init__.c
shared-module/audiocore/WaveFile.c
shared-module/displayio/OnDiskBitmap.c
shared-module/gifio/OnDiskGif.c
shared-module/lvfontio/OnDiskFont.c

[tannewt]

So do you need the extra API for file operations in fileystem.c? Could you use those existing other operations and types?

I introduced a new way because the stream API expects the MP VM to be running for raising exceptions and allocating. Added a comment about this in the header.

I don't care about FATFS use on a per-board basis because that won't change. Boards know what filesystem they are using.

Yes, others are:

grep -Elr "f_read|f_lseek|f_rewind" shared*
shared-bindings/synthio/__init__.c
shared-module/audiocore/WaveFile.c
shared-module/displayio/OnDiskBitmap.c
shared-module/gifio/OnDiskGif.c
shared-module/lvfontio/OnDiskFont.c

Did you run this command on this PR? I think I got most of them. I'd only missed synthio but it still checked it was FATFS first.

[dhalbert]

Did you run this command on this PR? I think I got most of them. I'd only missed synthio but it still checked it was FATFS first.

I ran @bablokb's grep on your branch, and now only shared-module/audiocore/WaveFile.c is left as having FAT-specific code.

[tannewt]

Ok, I had my clanker update wavefile. It would have errored early when given littlefs but now it should work.

[dhalbert]

OK, looks good!