Home
Commands
vLLM with quantization
vLLM with quantization
AI
python -m vllm.entrypoints.openai.api_server --model <model> --quantization awq
Copy command