Skip to content

Stop treating non-file document URIs as filesystem paths - #982

Open
ZayanKhan-12 wants to merge 2 commits into
palantir:developfrom
ZayanKhan-12:fix/non-file-uris
Open

ZayanKhan-12 wants to merge 2 commits into
palantir:developfrom
ZayanKhan-12:fix/non-file-uris

Conversation

@ZayanKhan-12

Copy link
Copy Markdown

Description

Partially addresses #199.

Editors that keep buffers in memory send URIs with no file behind them — Monaco's inmemory://dummy.py from the original report, and VS Code's untitled: and vscode-notebook-cell:. The server assumed every document had a file, derived a path, and then used that path for filesystem work.

The specific trigger is that inmemory://dummy.py puts the name in the URI authority, not the path, so urlparse gives netloc='dummy.py', path=''. The UNC branch in to_fs_path is gated on scheme == 'file', so the authority was dropped and the function returned the empty string:

inmemory://dummy.py    -> to_fs_path ''       -> filename ''  dot_path ''
untitled:Untitled-1    -> to_fs_path 'Untitled-1'

Three things followed from that empty path. The one worth highlighting is the second:

  1. Jedi received path='' rather than no path.
  2. os.path.dirname('') normalizes to '.', so asking for completions with use_document_path=True put the server's own working directory on Jedi's module search path. Completions in an in-memory buffer could resolve against whatever happened to sit next to the running process. Verified by capturing the sys_path handed to jedi.Project.
  3. Reading a document that had not been opened yet fell through to io.open(''), raising FileNotFoundError: '' — an error naming a path the client never sent.

Changes

to_fs_path keeps the authority for non-file schemes, so inmemory://dummy.py resolves to /dummy.py and the document has a stable identity for module naming and diagnostics.

This deliberately differs from vscode-uri, which drops the authority and returns the empty path. Since the module docstring points at vscode-uri and a reader may well diff the two, the reason is recorded in a comment next to the branch. File URIs, UNC paths and Windows drive letters are untouched:

file:///tmp/a.py       -> /tmp/a.py        (unchanged)
file://share/c$/a.py   -> //share/c$/a.py  (unchanged)
file:///C:/far/boo     -> c:/far/boo       (unchanged)
inmemory://dummy.py    -> /dummy.py        (was '')

Adds uris.is_file_uri() and Document.is_file_backed so callers can tell a path-shaped identifier from an actual location, then uses it to skip the sys.path extension for non-file documents and to raise a clear error instead of reading an unrelated file.

Scope

This covers in-memory analysis of a single document, which is what @gatesn asked about in the thread ("purely in-memory analysis, e.g. only things passed by didOpen").

It does not implement what @ColinTenaguillo asked for in 2022 — importing between two in-memory documents. That needs Jedi to see unsaved buffers as importable modules, which means a custom project or importer rather than a URI fix, and I did not want to bundle it into this change. Flagging it so the issue is not closed as fully resolved on the strength of this PR.

Tests

16 new tests across test/test_uris.py and test/test_document.py: the URI translations above, is_file_uri across schemes, the document properties, the sys.path behaviour for both document kinds, the error when an unopened non-file document is read, and a completion inside an in-memory document.

I checked the tests actually hold the line by re-breaking each fix in turn: reverting the authority handling fails 3, restoring the sys.path injection fails 1, removing the source guard fails 1.

Verification

Run on Python 3.8 with jedi 0.17.2, matching the CI matrix.

A clean checkout of develop does not produce a clean run, so I recorded a baseline first and compared:

baseline with this change
pytest test/ 12 failed, 99 passed, 8 skipped 12 failed, 115 passed, 8 skipped
failing test names identical set
pycodestyle pyls test clean clean
pyflakes pyls test 1 pre-existing (_utils.py) unchanged

The 12 failures are pre-existing and track linter versions that moved on since 2020 (flake8, pydocstyle, pylint, numpy hover). None are touched by this change.

pylint pyls test goes from 84 messages to 92. The eight added are 6 × redundant-u-string-prefix and 2 × consider-using-f-string, both of which the Python 2.7 support in this repo requires and both already present in the baseline. I fixed the one genuinely new message I introduced, a wrong-import-order caused by test shadowing a stdlib name.

Two environment pins were needed to get the suite running at all, neither related to this change: setuptools<70 (pkg_resources was removed) and pylint<3 (pylint.epylint was removed). Both are captured in the CLAUDE.md added in the second commit, which is independent — happy to drop that commit if it is not wanted.

zk-khan and others added 2 commits September 16, 2026 16:18
Editors that keep buffers in memory send URIs such as Monaco's
inmemory://dummy.py or VS Code's untitled:. Those have no file behind them, but
the server assumed every document did, so it derived a path and then used it for
filesystem work.

The name in inmemory://dummy.py lands in the URI authority rather than the path,
so to_fs_path returned the empty string for it. That left the document with an
empty path, filename and dot name, and three consequences followed:

- os.path.dirname('') normalizes to '.', so requesting completions with
  use_document_path put the server's own working directory on Jedi's module
  search path. Completions could resolve against whatever happened to be next to
  the running process.
- Jedi received path='' rather than no path at all.
- Reading a document that had not been opened yet fell through to
  io.open(''), raising FileNotFoundError naming a path the client never sent.

to_fs_path now keeps the authority for non-file schemes, so inmemory://dummy.py
resolves to /dummy.py and the document has a stable identity for module naming
and diagnostics. This deliberately differs from vscode-uri, which drops the
authority, and the reason is recorded next to the branch. File URIs, UNC paths
and Windows drive letters are unchanged.

Adds uris.is_file_uri and Document.is_file_backed so callers can tell an
identifier from a location, and uses it to skip the sys.path extension and to
raise a clear error instead of reading an unrelated file.

This covers in-memory analysis of a single document. Resolving imports between
two in-memory documents needs Jedi to see them as modules and is not addressed.

Partially addresses palantir#199

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Records the things that are easy to get wrong here: the Python 2.7 and
jedi<0.18 constraints that rule out a modern interpreter, the two extra pins
needed for a working environment, the pre-existing test and lint failures on a
clean checkout, and the distinction between a URI that has a file behind it and
one that does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@palantirtech

Copy link
Copy Markdown
Member

Thanks for your interest in palantir/python-language-server, @ZayanKhan-12! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

ZayanKhan-12 pushed a commit to ZayanKhan-12/python-language-server that referenced this pull request Sep 16, 2026
Stop treating non-file document URIs as filesystem paths (palantir#199)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants