Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
vllm openai接口需要添加模型名称
  • Loading branch information
yijia2413 authored Feb 27, 2024
1 parent e4ec830 commit b15b735
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Now, Qwen1.5 is supported by multiple inference frameworks. Here we demonstrate
### vLLM
We advise you to use `vLLM>=0.3.0` to build OpenAI-compatible API service. Start the server with a chat model, e.g. `Qwen1.5-7B-Chat`:
```shell
python -m vllm.entrypoints.openai.api_server --model Qwen/Qwen1.5-7B-Chat
python -m vllm.entrypoints.openai.api_server --served-model-name Qwen1.5-7B-Chat --model Qwen/Qwen1.5-7B-Chat
```

Then use the chat API as demonstrated below:
Expand All @@ -139,7 +139,7 @@ Then use the chat API as demonstrated below:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen1.5-7B-Chat",
"model": "Qwen1.5-7B-Chat",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me something about large language models."}
Expand All @@ -158,7 +158,7 @@ client = OpenAI(
)

chat_response = client.chat.completions.create(
model="Qwen/Qwen1.5-7B-Chat",
model="Qwen1.5-7B-Chat",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me something about large language models."},
Expand Down

0 comments on commit b15b735

Please sign in to comment.