[Windows] os.device_encoding(fd) should support any console fd #87587

Activity

1. eryksun commented on Mar 6, 2021

   

   eryksun

   on Mar 6, 2021

   ContributorAuthor

   More actions

   In Windows, os.device_encoding() is hard coded to map file descriptor 0 and descriptors 1 and 2 respectively to the console's input and output code page if isatty(fd) is true. But isatty() is true for any character device, such as "NUL". Also any fd might be a console, by way of dup(), dup2() -- or open() with the device names "CON", "CONIN$", and "CONOUT$".

   The correct device encoding of a console file is needed for use with os.read() and os.write(). It's also necessary for io.TextIOWrapper() if PYTHONLEGACYWINDOWSSTDIO is set.

   _Py_device_encoding() in Python/fileutils.c should use _get_osfhandle() to get the OS handle, and, if it's a character-device file, determine the code page to return, if any, depending on whether it's an input or output console file. For example:

   PyObject *
   _Py_device_encoding(int fd)
   {
   #if defined(MS_WINDOWS)
       HANDLE handle;
       DWORD temp;
       UINT cp = 0;
   
       _Py_BEGIN_SUPPRESS_IPH
           handle = (HANDLE)_get_osfhandle(fd);
       _Py_END_SUPPRESS_IPH
       if (handle == INVALID_HANDLE_VALUE ||
               GetFileType(handle) != FILE_TYPE_CHAR)
           Py_RETURN_NONE;
   
       Py_BEGIN_ALLOW_THREADS
       /* GetConsoleMode requires a console handle. */
       if (!GetConsoleMode(handle, &temp)) {
           /* Assume access denied implies output. */
           if (GetLastError() == ERROR_ACCESS_DENIED)
               cp = GetConsoleOutputCP();
       } else {
           if (GetNumberOfConsoleInputEvents(handle, &temp)) {
               cp = GetConsoleCP();
           } else {
               cp = GetConsoleOutputCP();
           }
       }
       Py_END_ALLOW_THREADS
   
       if (cp == CP_UTF8) {
           return PyUnicode_FromString("UTF-8");
       } else if (cp != 0) {
           return PyUnicode_FromFormat("cp%u", (unsigned int)cp);
       } else {
           Py_RETURN_NONE;
       }
   #else
       if (isatty(fd)) {
           return _Py_GetLocaleEncodingObject();
       } else {
           Py_RETURN_NONE;
       }
   #endif  /* defined(MS_WINDOWS) */
   }
2. eryksun

   added

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   OS-windows

   topic-IO

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Mar 6, 2021
3. ezio-melotti

   transferred this issue fromon Apr 10, 2022
4. eryksun

   changed the title [-]os.device_encoding(fd) should support any console fd in Windows[/-] [+][Windows] os.device_encoding(fd) should support any console fd[/+] on Aug 8, 2022
5. serhiy-storchaka

   added a commit that references this issue on Aug 29, 2026

   gh-87587: Fix os.device_encoding() for any console on Windows (GH-155410

   ...

   b4aea41
6. serhiy-storchaka

   closed this as completedon Aug 29, 2026
7. serhiy-storchaka

   added 2 commits that reference this issue on Aug 29, 2026

   [3.13] gh-87587: Fix os.device_encoding() for any console on Windows (G…

   ...

   d9b7390

   [3.14] gh-87587: Fix os.device_encoding() for any console on Windows (G…

   ...

   7755fce
8. clin1234

   added a commit that references this issue on Sep 12, 2026

   pythongh-87587: Fix os.device_encoding() for any console on Windows (p…

   ...

   5c0923e
9. serhiy-storchaka

   added a commit that references this issue on Oct 10, 2026

   [3.15] gh-87587: Fix os.device_encoding() for any console on Windows (G…

   ...

   d21c501