Skip to content

Add CPU and CUDA autocast scopes backed by LibTorch - #1582

Open
lunosia wants to merge 1 commit into
dotnet:mainfrom
lunosia:feature/bf16-autocast
Open

lunosia wants to merge 1 commit into
dotnet:mainfrom
lunosia:feature/bf16-autocast

Conversation

@lunosia

@lunosia lunosia commented Sep 29, 2026

Copy link
Copy Markdown

Summary

  • Expose torch.autocast(DeviceType, ScalarType) and is_autocast_enabled for CPU and CUDA through LibTorch.
  • Preserve thread-local enabled state and dtype across nested scopes. Clear cached casts when the dtype changes so BF16 and FP16 scopes can nest safely.
  • Add CPU and CUDA tests for output dtype, FP32 parameters and gradients, optimizer updates, nesting, and thread-affine disposal. The Linux CUDA test project resolves its runtime package directly.
  • On Windows with LibTorch 2.10, remove whole-archive only from the torch.lib and torch_cpu.lib DLL import libraries. Without this, the native build fails with duplicate import descriptors (LNK2005/LNK1169).

Validation

  • Windows x64 / Visual Studio 2022: native and managed build succeeded; net8.0 CPU suite 1332 passed, 30 skipped; net472 test project compiled.
  • Ubuntu x64 / RTX 3090 / CUDA 12.8 runtime: CPU suite 1329 passed, 33 skipped; CUDA runtime suite 1324 passed, 26 skipped.
  • With CUDA hidden, the two CUDA autocast tests fail explicitly instead of reporting a false pass.
  • A downstream C# language-model project completed BF16 autocast short training and checkpoint resume on the RTX 3090.

Scope and related work

  • This exposes BF16 and FP16 autocast scopes. It does not add a GradScaler for FP16 training.
  • Related to Autocast #1235, which explores broader AMP support. This PR keeps the API and tests focused on the existing LibTorch autocast implementation.

@lunosia

lunosia commented Sep 29, 2026

Copy link
Copy Markdown
Author

@dotnet-policy-service agree

@lunosia
lunosia marked this pull request as ready for review September 29, 2026 22:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant