Type inference 2.0

[hvitved]

This PR makes a significant overhaul of our QL based implementation of type inference for Rust (hence the tacky PR title). At a high level, a lot of code is moved from the Rust codebase to the shared type inference library (in preparation for unified/Swift), and there is now a very clear distinction between bottom-up type inference and top-down (contextual) type inference.

Before this PR

• The shared type inference library contained just the core functionality for propagating type information through for example function calls, but all logic for mapping AST nodes to types was done outside of the library.
• Rust type inference allowed for types to propagate bidirectionally (as in classical constrained-based implementations), but because of QL's monotonic nature, this could often result in combinatorial explosions. To circumvent such explosions, advanced logic existed for inferring types with certainty, and this logic also tried to infer certain types for calls. Another measure put in place to prevent explosions was LUB coercions, with ad hoc logic for propagating type information between LUB siblings (such as the two branches of a conditional expression).

Shared logic for mapping AST node to types

We introduce a new Make3 parameterization layer to the shared type inference library, which takes as input a definition of AST nodes, including common concepts such as calls and callables, as well as language-specific typing rules, and constructs the inferType predicate for recursively inferring the types of AST nodes.

The input signature of Make3 is deliberately similar to that of the shared CFG library, and it may be possible to align them at some point.

The shared library takes care of typing of many standard constructs such as calls and field accesses, and also has logic for contextual typing and typing of closures.

Bottom-up vs top-down inference

Perhaps the most important change is that we now distinguish between bottom-up type inference (the default) and top-down type inference. For example, in order to infer the type of a conditional expression,if cond { e1 } else { e2 }, we propagate type information from either of the branches e1 and e2 into the conditional expression (for simplicity, we do not attempt to calculate least-upper-bound types or similar). This corresponds to the two bottom-up type inference rules:

            e1: T
------------------------------- (cond-then)
if cond { e1 } else { e2 } : T

            e2: T
------------------------------- (cond-else)
if cond { e1 } else { e2 } : T

Now, if we have a conditional expression like

if cond { 42i64 } else { Default::default() }

where the type of Default::default() needs to be inferred from the context, we

1. conclude that the conditional has type i64, using the cond-then rule,
2. assign Default::default() the special UnknownType (the shared library has logic for identifying calls where (parts of) the return type needs to be inferred from the context), and
3. since the else branch has UnknownType, we apply the cond-else rule backwards to infer that Default::default() has type i64.

Note that UnknownType can propagate bottom-up like any other type, which is needed in cases like for example

let x = if cond { Default::default() } else { Default::default() };
let y : i64 = x;

where the UknownType will propagate upwards using two bottom-up steps, and the contextual inference will then propagate the i64 type backwards using two reversed steps.

Reversal of bottom-up steps happens inside the ContextualTyping::inferTypeContextualCand0 predicate, and contextual propagation into a node n at type path path is only allowed when n has UnknownType at some prefix of path, and furthermore if path is non-empty, then it must be compatible with an already inferred type (contextually or not). The latter part means that the Rust-specific typing rule for *e expressions, when e has a raw pointer type, can be handled by a single bottom-up rule (the first disjunct of stepLanguageSpecific) instead of two rules in the old implementation.

Simplified and shared certain type inference

The logic for inferring types with certainty has been moved inside the shared library (Make3::Certain), but we no longer attempt to infer certain type inference for calls. This simplifies the implementation significantly, but without resulting in combinatorial explosions because the revised handling of contextual inference is much less prone to explosions.

Improved and shared handling of closure typing

Closures typically need to have their parameter types inferred from the context in which they are used. There are two ways for type information to flow contextually into a closure parameter: (A) either by knowing the types of arguments, or (B) by knowing the return type. While we could assign closure parameters the UnknownType, this would mean that they could also have their type inferred from the closure body, which we want to avoid (as it can result in combinatorial explosions).

Case A

let c = |x| (x, false);
let r = c(0);

1. c is assigned the type Fn(UnknownType) -> ...,
2. since 0 has type i32, we can infer the c has type Fn(i32) -> ..., and
3. using contextual inference, we conclude that x has type i32.

Case B

let c = |x| (x, false);
let r: i32 = c(Default::default()).0;

1. x is assigned a special pseudo type T_x,
2. infer that the return type of c is (T_x, bool) and hence that c has type Fn(...) -> (T_x, bool),
3. this enables us to detect that contextual inference is needed, so we also assign c the type Fn(...) -> (UnknownType, bool),
4. infer that c(Default::default()).0 must have UnknownType,
5. infer, using contextual inference, that c has type Fn(...) -> (i32, bool), and finally
6. since c also has type Fn(...) -> (T_x, bool), we conclude that x has type i32 and hence that c has type Fn(i32) -> (i32, bool).

Improved and shared handling of type arguments and type qualifiers

Type qualifiers and type arguments are now distinguished, so for example in Foo::<A>::bar::<B>(...), Foo::<A> is the type qualifier and B is the only explicit type argument (we used to also consider A a type argument). This means that type arguments only need positional matching, and hence TypeArgumentPosition is no longer needed.

Other minor changes

• We no longer have a dedicated NeverType for ! typed expressions; instead we simply use UnknownType to indicate that the actual type must be inferred from the context.
• async return types are now also taken into account for closures.
• Various predicates in FunctionOverloading.qll have changed a column type from TypeParameter to TypeParamTypeParameter; the reason is that all associated types, which are also modeled as type parameters, are functionally determined from the Self type, which we already check.

Note for the reviewer

As usual, commit-by-commit reviewing is encouraged. The second commit (which compiles and works) moves a bunch of logic around in the Rust implementation, which is then removed in the subsequent commit (which doesn't compile). I found that doing it like this resulted in a cleaner diff on the last commit, and it also makes it more clear some of the parts that are now handled by shared code.

Impact

• Around 900 lines of code is removed from the Rust specific implementation.
• From DCA:
  • Percentage of calls with call target increases by 3.9 % point from 84.9 % to 88.8 % (and, as a result, number of alerts increases as well).
  • Nodes With Type At Length Limit, which measures type inference explosions, decreases by almost 80 % from 219,692 to 46,302.
  • Analysis time is mostly unchanged.
• From QA:
  • 13 stable progressions and 13 stable regressions.
  • Average run_queries timings unchanged.
  • More query results.

Future work

• Handle more AST constructs in the shared library, such as patterns (should be relatively straightforward).
• Shared logic for type-based overload resolution; logic currently exists for Rust, and perhaps some of this logic can be shared.

[Copilot]

Pull request overview

Refactors Rust type inference around the shared library’s new bottom-up and contextual inference architecture.

Changes:

• Adds shared AST inference, contextual typing, closure handling, and diagnostics.
• Reimplements Rust inference through the shared Make3 interface.
• Updates Rust tests and consistency expectations.

Show a summary per file

File	Description
shared/util/codeql/util/UnboundList.qll	Adds list append helper.
shared/typeinference/codeql/typeinference/internal/TypeInference.qll	Implements shared inference framework.
rust/ql/test/library-tests/type-inference/type-inference.ql	Uses shared type-test support.
rust/ql/test/library-tests/type-inference/pattern_matching.rs	Updates inference expectations.
rust/ql/test/library-tests/type-inference/overloading.rs	Records contextual inference regression.
rust/ql/test/library-tests/type-inference/main.rs	Updates coverage and expectations.
rust/ql/test/library-tests/type-inference/dereference.rs	Exercises inferred generic arguments.
rust/ql/test/library-tests/type-inference/CONSISTENCY/PathResolutionConsistency.expected	Updates generated consistency output.
rust/ql/test/library-tests/type-inference/closure.rs	Updates closure expectations.
rust/ql/test/library-tests/dataflow/sources/web_frameworks/CONSISTENCY/TypeInferenceConsistency.expected	Updates generated consistency output.
rust/ql/test/library-tests/dataflow/models/CONSISTENCY/PathResolutionConsistency.expected	Updates generated consistency output.
rust/ql/lib/codeql/rust/internal/typeinference/TypeMention.qll	Adds contextual and constructor type mentions.
rust/ql/lib/codeql/rust/internal/typeinference/TypeInferenceConsistency.qll	Adopts shared consistency checks.
rust/ql/lib/codeql/rust/internal/typeinference/TypeInference.qll	Adapts Rust inference to Make3.
rust/ql/lib/codeql/rust/internal/typeinference/Type.qll	Introduces generalized pseudo-types.
rust/ql/lib/codeql/rust/internal/typeinference/FunctionType.qll	Generalizes pseudo-type filtering.
rust/ql/lib/codeql/rust/internal/typeinference/BlanketImplementation.qll	Generalizes pseudo-type filtering.
rust/ql/lib/codeql/rust/internal/CachedStages.qll	Uses the shared inference cache stage.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (1)

shared/typeinference/codeql/typeinference/internal/TypeInference.qll:3345

• Remove the duplicated article.

       * Holds if the the textual representation `repr` should be used for `n` in

• Files reviewed: 17/19 changed files
• Comments generated: 4
• Review effort level: Balanced

[paldepind]

We no longer have a dedicated NeverType for ! typed expressions; instead we simply use UnknownType to indicate that the actual type must be inferred from the context.

It's not clear to me why ! requires contextual inference? Could we add a test that demonstrates the need?

Percentage of calls with call target increases by 3.9 % point from 84.9 % to 88.8 % (and, as a result, number of alerts increases as well).

That's a really nice improvement! Do we know why the increase is this large? Is it due to the changes for closures?

[paldepind]

Maybe move the blank line up before arr3 to make it clear that the last three lines are connected?

[hvitved]

The space after arr3 is needed since otherwise the inline expectation framework thinks that // $ type=arr3@[;]<TArray>:i32 is also assigning a name to pin_array 🤦 But I'll put in an extra linebreak.

[paldepind]

Why is this now missing?

[hvitved]

Previously, we would push the return type annotation usize onto x, but now we only do that when explicit contextual information is needed.

[paldepind]

Pedantically speaking e is an element an not a singleton list.

Suggested change

	* Gets the list obtained by appending the singleton list `e`
	* after `prefix`.
	* Gets the list obtained by appending the element `e` after `prefix`.

Perhaps we should also change the doc for cons to say: "Gets the list obtained by prepending the element e onto suffix"?

[paldepind]

Suggested change

	* type inference, for example a local variable, `const`, or `static` in Rust.
	* type inference, for example a local variable, `const` item, or `static` item in Rust.

Seems a little clearer to me since these keywords have multiple uses in Rust. For instance, static can also be used for lifetimes.

[paldepind]

Suggested change

	* Gets the type at `path` of the entity that contains this declaration, if any.
	* Gets the type mention of the entity that contains this declaration, if any.

[paldepind]

We could also do this adjustment on the type mentions in return position in async functions in Rust. Then we could remove this predicate.

[hvitved]

I thought about doing that, but then also thought it would be a bit weird to have explicitly mentioned types that resolve to something different from what is actually mentioned. If we had an AST node for the async modifier we could have used that as a representative, but sadly we don't.

[paldepind]

The doc states this as an implication, but since we reverse the steps for contextual types, it must in fact be a biimplication.

Subjectively I'd still prefer the typeEqual name we had before. It still represents the idea that two nodes have equal types (for some prefixes) but with an added directionality that n1 should be below n2 in the AST.

With that name the doc could be something like the following:

Holds if the type tree of n1 at prefix1 should be equal to the type tree of n2 at prefix2 and n1 is below n2 in the AST.

Type information always flow right-to-left/bottom-up through type equalities. When contextual type inference is needed, type information additionally flows left-to-right/top-down through type equalities.

[hvitved]

How about if I make it clear in the QL doc that it is sometimes reversed as well? I don't think we want to say that n1 is necessarily below n2 in the AST; while it is true for expressions, the opposite is true for patterns.

[paldepind]

I guess in the future we'd want to guard this, as there's many languages where logical operators doesn't necessarily evaluate to booleans.

[paldepind]

Suggested change

	* let y : i64 = x;
	* let y: i64 = x;

[paldepind]

Suggested change

	// `x : Vec<_>` in Rust
	// `x: Vec<_>` in Rust