Update openai-style api in README.md
This commit is contained in:
parent
3444756a17
commit
de8ab5b136
22
README.md
22
README.md
@ -225,6 +225,28 @@ curl -X POST "http://127.0.0.1:8000" \
|
|||||||
"time":"2023-03-23 21:38:40"
|
"time":"2023-03-23 21:38:40"
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
感谢 [@hiyouga]() 实现了 OpenAI 格式的流式 API 部署,可以作为任意基于 ChatGPT的应用的后端,比如 [ChatGPT-Next-Web](https://github.com/Yidadaa/ChatGPT-Next-Web)。可以通过运行仓库中的[openai_api.py](openai_api.py) 进行部署:
|
||||||
|
```shell
|
||||||
|
python openai_api.py
|
||||||
|
```
|
||||||
|
进行 API 调用的示例代码为
|
||||||
|
```python
|
||||||
|
import openai
|
||||||
|
if __name__ == "__main__":
|
||||||
|
openai.api_base = "http://localhost:8000/v1"
|
||||||
|
openai.api_key = "none"
|
||||||
|
for chunk in openai.ChatCompletion.create(
|
||||||
|
model="chatglm2-6b",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "你好"}
|
||||||
|
],
|
||||||
|
stream=True
|
||||||
|
):
|
||||||
|
if hasattr(chunk.choices[0].delta, "content"):
|
||||||
|
print(chunk.choices[0].delta.content, end="", flush=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## 低成本部署
|
## 低成本部署
|
||||||
|
|
||||||
### 模型量化
|
### 模型量化
|
||||||
|
Loading…
Reference in New Issue
Block a user