Skip to content

Add support for Windows #162

Open
Open

Description

@lstocchi

Following my chat with Colin, I am opening this issue to track the exploration of enabling Windows for bcvk.
This issue serves as a high-level overview of the initial POC and a discussion point for the direction we want to take.

During DevTools week I have implemented a CLI POC that allows the project to work on Windows using Hyper-V.

  • POC branch (experimental): Link
  • main branch (a bit more polished/rebased on upstream): Link

the POC is very limited and only implements the to_disk and hyperv run commands. They are very far from the upstream implementation. My intent was more to put my hands over rust. It's not ready for testing as it is tightly coupled with my local setup. It requires podman to have a machine on WSL, macadam to be installed and that the bootc image to have the hyperv-daemons package in it.

This issue will be used as an Epic to track sub-issues that provides a minimal support on Windows.

@cgwalters if you prefer to not have any epic you can close it. I opened this as it was the only way i thought to give a quick overview why i was targeting windows.

Were you envisioning to have the windows implementation here or keep a separate project as it could be implemented slower than the linux version?
FYI in the bcvk_win main branch i tried to add my commits over bcvk.

Which is the minimal support bcvk should provide on Windows? Bootc VHD/VHDX image creation + running a permanent Hyper-v VM with that image? I'll open an issue for each command to separate discussions and avoid confusion.

Activity

  1. cgwalters commented on Dec 1, 2025

    @cgwalters
    Collaborator

    Were you envisioning to have the windows implementation here or keep a separate project as it could be implemented slower than the linux version?

    A complex question because I think roughly 80% of the code would not be shared, right?

    That said...I guess my lean is it'd make sense to try to share that remaining 20%.

    So one of the first commits there basically turns the entire project into a "stub" effectively on non-linux. I think we can do this a little bit better. Maybe one thing that we could do is have crates/bcvk_linux and crates/bcvk_windows for example.

    We'd have then crates/bcvk_cli which would hold the shared bits (perhaps just some core clap structs?).

    A prep commit could basically git mv crates/kit crates/bcvk_linux and then create the cli crate?


    Which is the minimal support bcvk should provide on Windows? Bootc VHD/VHDX image creation + running a permanent Hyper-v VM with that image?

    I would really like to replicate the bcvk ephemeral core primitive because that means that generating disk images is just a special case of a more general tool.

    See #21 - my understanding here is that Windows is probably similar.

  2. tnk4on commented on May 26, 2026

    @tnk4on
    Contributor

    Windows Hyper-V backend: progress update

    I've been working on a Windows implementation of bcvk ephemeral using Hyper-V Gen2 VMs. The development branch is at tnk4on/bcvk:wip/windows-pr. This follows the approach established in the macOS PR #259.

    Architecture

    Ephemeral VMs run as Hyper-V Gen2 VMs. The block device (rootfs) is provided via Hyper-V standard vsock (AF_HYPERV) — no patches or custom drivers required. bcvk works with any standard Podman installation, with no modifications to Podman itself. Podman machine can use either the WSL2 or Hyper-V provider.

    The data flow:

    podman machine (nbdkit EROFS) ←[vsock]← Host relay (caching) →[vsock]→ ephemeral VM (ublk/nbd)
    
    • Host-initiated vsock: Windows Host connects TO both VMs via AF_HYPERV sockets. During development we observed a ~10x throughput asymmetry depending on connection direction (~1 GB/s Host-initiated vs ~120 MB/s Guest-initiated), likely due to VMBus ring buffer handling
    • NBD caching relay: prefetches 256MB boot regions + on-demand cache for subsequent reads
    • ublk (io_uring) on Fedora, NBD fallback on RHEL/CentOS (io_uring disabled by default on EL10)
    Boot time & I/O benchmarks (click to expand)

    Boot time (seconds)

    Image already pulled to podman machine. "Cache miss" = boot VHDX rebuilt from scratch, "Cache hit" = boot VHDX reused from local cache. Measured with podman machine Hyper-V provider.

    Image Device Cache miss Cache hit
    Fedora latest (44) ublk 60.9 42.0
    RHEL 9 NBD 54.5 36.7
    RHEL 10 NBD 59.5 43.5
    RHEL 10 EUS 10.0 NBD 56.2 45.3
    CentOS Stream 10 NBD 57.4 41.0

    dd (MB/s, cached, iflag=direct)

    podman machine: WSL2 provider:

    Image Device Full disk 1GB 256MB
    Fedora latest (44) ublk 696 669 666
    RHEL 9 NBD 481 477 445
    RHEL 10 NBD 512 468 474
    RHEL 10 EUS 10.0 NBD 550 428 427
    CentOS Stream 10 NBD 525 491 483

    podman machine: Hyper-V provider:

    Image Device Full disk 1GB 256MB
    Fedora latest (44) ublk 672 721 661
    RHEL 9 NBD 534 509 466
    RHEL 10 NBD 496 474 495
    RHEL 10 EUS 10.0 NBD 536 557 499
    CentOS Stream 10 NBD 505 502 530

    fio (cached, 10s average)

    Sequential Read 1M (MiB/s):

    Image Device WSL 1job WSL 4jobs HV 1job HV 4jobs
    Fedora latest (44) ublk 622 685 613 668
    RHEL 9 NBD 477 834 510 813
    RHEL 10 NBD 479 813 485 824
    RHEL 10 EUS 10.0 NBD 488 804 451 794
    CentOS Stream 10 NBD 468 815 445 805

    Random Read 4K (KiB/s):

    Image Device WSL 1job HV 1job
    Fedora latest (44) ublk 5638 5446
    RHEL 9 NBD 4470 4408
    RHEL 10 NBD 4452 4367
    RHEL 10 EUS 10.0 NBD 4435 4342
    CentOS Stream 10 NBD 4483 4353

    Random Read 128K (MiB/s):

    Image Device WSL HV
    Fedora latest (44) ublk 102 110
    RHEL 9 NBD 115 115
    RHEL 10 NBD 118 113
    RHEL 10 EUS 10.0 NBD 116 113
    CentOS Stream 10 NBD 115 113

    Current status

    • Ephemeral mode: nearly complete. Tested on 5 images × 2 podman machine providers (WSL2, Hyper-V).
    • Persistent VM mode: TBD
  3. lstocchi commented on May 26, 2026

    @lstocchi
    Author

    It's been a while since I looked into this, but this is cool!

    An update about me: after talking with Colin, my goal was to see if I could leverage virtio-fs on Windows somehow, using bcvk to bridge the rootfs to the ephemeral VM similarly to how it's done on Linux. So, i've been working on a custom VMM for Windows, but then i was asked to port it on libkrun so i'm quite stuck atm.
    Once it will be merged i wanted to give it a try here and see if i could bypass Hyper-V constraints.

    That said, your NBD approach looks great. I think it was the only viable acceptable solution with Hyper-v. Do you need to start bcvk as admin to set up the hvsock in the Registry? Do you just need one for starting ephemeral VMs on different images at the same time?

  4. tnk4on commented on Jun 9, 2026

    @tnk4on
    Contributor

    @lstocchi Thanks for the comment!

    To answer your questions:

    Do you need to start bcvk as admin to set up the hvsock in the Registry?

    Admin is needed on first run — same mechanism as podman's hyperv-prep. The hvsock service GUID must be registered in the Windows registry (HKEY_LOCAL_MACHINE\...\GuestCommunicationServices). bcvk uses its own port range, separate from podman's entries, so it needs its own registration. Once registered, subsequent runs don't require admin. The NBD server (bcvk-nbd) itself runs inside the podman machine VM as a systemd transient service, and a host-side vsock relay bridges the NBD traffic between the podman machine and the ephemeral Hyper-V VM.

    Do you just need one for starting ephemeral VMs on different images at the same time?

    Each ephemeral VM gets its own NBD server instance (one per image), but they all share the same podman machine. Concurrent VMs on different images work — each bcvk-nbd unit gets a unique vsock port, and the vsock relay handles multiplexing. This has been tested with both WSL2 and Hyper-V podman machine providers.

    Since my last update, I've added significant improvements (see wip/windows-pr):

    • Replaced nbdkit + C plugin with a standalone Rust NBD server (bcvk-nbd crate, same as #259 )
    • Added persistent VM support (bcvk vm run/list/ssh/stop/start/rm)
    • Added to-disk support (VHDX with SHA256 cache + differencing disks)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions