Implement vLLM FSDP LoRA hot-swapping integration - #10

Open

jacobthebanana wants to merge 92 commits into

jjt/lora-vllm-hotswap

Commits on Feb 26, 2024

Implemented baseline LoRA peft for one Nvidia GPU.
jacobthebanana
committed
904d1e1

Commits on Feb 27, 2024

Added support for saving lora adapters.
jacobthebanana
committed
2ace67e

Commits on Feb 29, 2024

Commits on Mar 6, 2024

Commits on Mar 11, 2024

Implemented LoRA benchmarking logic for worker.
jacobthebanana
committed
3eea331
model_utils: Refactored get_lora_model to reduce interface width. (this method no longer wraps load_model_and_tokenizer)
jacobthebanana
committed
906e4f3
test_modelling: moved text output to data/.
jacobthebanana
committed
0c41535
added example yaml config for lora benchmarking.
jacobthebanana
committed
f24d2fa
launch_benchmark: marked qos flag as optional.
jacobthebanana
committed
7d27d90
launch_benchmark: added option to limit number of jobs launched.
jacobthebanana
committed
d22ea85
launch_benchmark: implemented torch profiler integration.
jacobthebanana
committed
84b953a
Merged changes from low CPU memory usage feature (#6) into jjt/lora-benchmarking

adil-a
authored and
jacobthebanana
committed
e1cda07
Revised launch_benchmark.py to use new profiling path.
jacobthebanana
committed
48f61d9
Enabled automatic creation of data/trace folder.
jacobthebanana
committed
9876ebe
Added instructions for profiling tools.
jacobthebanana
committed
5330871
Merge remote-tracking branch 'origin/master' into jjt/lora-baseline
jacobthebanana
committed
17e24bd
Cleaned up duplicate imports from merge.
jacobthebanana
committed
9982791
Cleaned up duplicate imports from merge.
jacobthebanana
committed
9a76e80
Cleaned up parse_benchmark.py
jacobthebanana
committed
ffa7067
Integrated LoRA logic into llama_example.py.
jacobthebanana
committed
bd893e1
Moved lora_configs into train_parameters in config yaml. Adjusted docs/config.md accordingly.
jacobthebanana
committed
c2f346f

Commits on Mar 13, 2024

Commits on Mar 15, 2024

Commits on Apr 9, 2024

Implemented non-LoRA profiling and benchmarking.
jacobthebanana
committed
eb3721a

Commits on Apr 11, 2024

Various static typechecking and formatting fixes.
jacobthebanana
committed
37f5dec

Commits on Apr 15, 2024

Commits on Apr 16, 2024

Commits on Apr 17, 2024

Commits on Apr 19, 2024

Merge commit 'afbc061' from jjt/lora-baseline into jjt/lora-vllm-hotswap
jacobthebanana
committed
7bc6f89

Commits on Apr 22, 2024

Commits on Apr 25, 2024

Commits on May 7, 2024

vllm integration [WIP]: Implemented lora hotswap.
jacobthebanana
committed
b697dc0

Commits on May 9, 2024

Commits on May 10, 2024

Commits on May 23, 2024

Commits on May 24, 2024

Commits on Jun 18, 2024