SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 07:40:12 +01:00
Code Issues Projects Releases Packages Activity

Add timeout parameter for better support of openailike LLM tools on local computer (like LM Studio). (#1858)
Some checks failed
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled

Browse source 
feat(llm): Improve settings of the OpenAILike LLM
This commit is contained in:
jcbonnet-fwd 2024-05-10 16:44:08 +02:00 • committed by GitHub
parent 966af4771d
commit 45df99feb7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 11 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
settings-vllm.yaml
View file

@ -3,6 +3,9 @@ server:
llm:
mode: openailike
max_new_tokens: 512
tokenizer: mistralai/Mistral-7B-Instruct-v0.2
temperature: 0.1
embedding:
mode: huggingface
@ -15,3 +18,4 @@ openai:
api_base: http://localhost:8000/v1
api_key: EMPTY
model: facebook/opt-125m
request_timeout: 600.0