Skip to content

logging_subprocess deadlocks on Windows when child stdout/stderr exceeds pipe buffer #519

Closed

Description

@chirag127

Problem

logging_subprocess in github_backup/github_backup.py opens child processes with stdout=PIPE, stderr=PIPE but never drains those pipes on Windows, causing git clone / git fetch of any repo producing more than a few KB of output to hang indefinitely.

Steps

  1. On Windows, run github-backup <user> --all -o backup/ against any account with a repo whose clone produces substantial stderr progress output.
  2. Observe process hangs at the first large git clone or git fetch.

Expected

Process completes; child output is either drained silently or logged.

Actual

The parent's while child.poll() is None: check_io() loop spins forever because check_io() returns immediately on win32 (github_backup/github_backup.py:102-103) while the child's PIPE buffers fill and block the git subprocess.

Environment

  • github-backup 0.63.0 (github_backup/__init__.py:1)
  • Windows, Python 3.10+ (per pyproject.toml)

Cite: github_backup/github_backup.py:91-114. Fix idea: drain stdout/stderr with threads on Windows, or pass stdout=None, stderr=None when sys.platform == "win32" so the child inherits the parent's handles instead of buffering into a never-read PIPE.

Thanks for maintaining josegonzalez/python-github-backup!

Activity

  1. Iamrodos commented on Jul 4, 2026

    @Iamrodos
    Contributor

    Good find, reproduced on Windows 11 / Python 3.12. It hangs once the child writes more than ~8KB to stderr, so any real clone or fetch with progress output triggers it. Quiet operations like an up-to-date incremental fetch can slip under the buffer, which is why it can look intermittent.

    The regression came in with v0.44.0 (7437e3a), which added the win32 early-return to stop select() crashing on Windows but left the pipes undrained, turning the crash into a hang.

    PR #520 submitted using your first suggestion, reader threads draining both pipes. Tested on Windows with a real git clone --progress and it now completes with the subprocess output logged, which Windows never had before.

    I suspect @josegonzalez will pick this up and merge it before too long.

  2. added a commit that references this issue on Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions