SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 10:45:42 +01:00
Code Issues Projects Releases Packages Activity

Add timeout parameter for better support of openailike LLM tools on local computer (like LM Studio). (#1858)
Some checks failed
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled

Browse source 
feat(llm): Improve settings of the OpenAILike LLM
This commit is contained in:
jcbonnet-fwd 2024-05-10 16:44:08 +02:00 • committed by GitHub
parent 966af4771d
commit 45df99feb7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 11 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
private_gpt/components/llm/llm_component.py
View file

@ -123,6 +123,9 @@ class LLMComponent:
max_new_tokens=settings.llm.max_new_tokens,
messages_to_prompt=prompt_style.messages_to_prompt,
completion_to_prompt=prompt_style.completion_to_prompt,
tokenizer=settings.llm.tokenizer,
timeout=openai_settings.request_timeout,
reuse_client=False,
)
case "ollama":
try:

4
private_gpt/settings/settings.py
View file

@ -205,6 +205,10 @@ class OpenAISettings(BaseModel):
"gpt-3.5-turbo",
description="OpenAI Model to use. Example: 'gpt-4'.",
)
request_timeout: float = Field(
120.0,
description="Time elapsed until openailike server times out the request. Default is 120s. Format is float. ",
)
class OllamaSettings(BaseModel):