Repository navigation
feat(chatterbox): support Turbo and Nano - #12664
Draft
localai-org-maint-bot wants to merge 4 commits into
Draft
localai-org-maint-bot wants to merge 4 commits into
localai-org-maint-bot wants to merge 4 commits into
Conversation
Select the model at load time and filter incompatible generation controls. Keep the original model as the default and load Turbo lazily. Add CPU-only contract tests for selection, errors, and speech requests. Assisted-by: Codex:gpt-6
Parse integers before floats for static options and request parameters. Turbo and Nano require an integer top_k value during generation. Assisted-by: Codex:gpt-6
Pin the dependency fork with isolated Turbo and Nano inference. Match its Transformers requirement across build profiles and add pyloudnorm. Run variant tests in CI and document configuration and sampler limits. Assisted-by: Codex:gpt-6
Keep float controls as floats when values omit the decimal point. Only top_k uses integer conversion for its strict sampler contract. Assisted-by: Codex:gpt-6
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add
turbo:trueandnano:truemodel options to the Chatterbox backend. The original model remains the default. Reject conflicting selectors and keep load-time selectors out of speech parameters. Turbo and Nano ignore unsupported CFG, exaggeration, and min-p controls; integer sampler options such astop_kretain their type.Fixes #12475.
Notes for Reviewers
Depends on mudler/chatterbox#1. The installer pins its commit
b38c317b03834ef35828ae96c72ac8929b002a05, which adds isolated Turbo/Nano inference while preserving the fork's existing optimized models. All hardware profiles use the dependency's Transformers version; the shared requirements addpyloudnorm.This is a draft pending dependency review and real model loading/audio validation. The environment lacks Torch and model weights. No backend image build or CPU/GPU audio synthesis was run. Configuration examples are included; this change does not add model-gallery entries.
Validation:
python3 -m unittest discover -s backend/python/chatterbox -p test_variants.py -v: 11 tests pass. Tests demonstrated failures before each fix, including integer and floating-point sampler regressions.python3 -m unittest discover -s backend/python/common -p temp_utils_test.py -v: 3 tests pass.bash -n backend/python/chatterbox/install.shandgit diff --checkpass.make -C backend/python/chatterbox test-unittarget runs the variant tests in lint CI. Local tests used a cached Python runtime directly because this environment has nomake.Signed commits
Assisted-byattribution.