SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 23:22:57 +01:00
Code Issues Projects Releases Packages Activity

added llama4, openchat prompt template, fixed a bug in ui.py that caused a test fail

Browse source
This commit is contained in:
Behrad 2025-04-18 14:37:59 +02:00
parent b7ee43788d
commit af8f2b4187
3 changed files with 124 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

28
private_gpt/settings/settings.py
View file

@ -136,19 +136,21 @@ class LLMSettings(BaseModel):
0.1,
description="The temperature of the model. Increasing the temperature will make the model answer more creatively. A value of 0.1 would be more factual.",
)
prompt_style: Literal["default", "llama2", "llama3", "tag", "mistral", "chatml"] = (
Field(
"llama2",
description=(
"The prompt style to use for the chat engine. "
"If `default` - use the default prompt style from the llama_index. It should look like `role: message`.\n"
"If `llama2` - use the llama2 prompt style from the llama_index. Based on `<s>`, `[INST]` and `<<SYS>>`.\n"
"If `llama3` - use the llama3 prompt style from the llama_index."
"If `tag` - use the `tag` prompt style. It should look like `<|role|>: message`. \n"
"If `mistral` - use the `mistral prompt style. It shoudl look like <s>[INST] {System Prompt} [/INST]</s>[INST] { UserInstructions } [/INST]"
"`llama2` is the historic behaviour. `default` might work better with your custom models."
),
)
prompt_style: Literal[
"default", "llama2", "llama3", "llama4", "tag", "mistral", "chatml", "openchat"
] = Field(
"llama2",
description=(
"The prompt style to use for the chat engine. "
"If `default` - use the default prompt style from the llama_index. It should look like `role: message`.\n"
"If `llama2` - use the llama2 prompt style from the llama_index. Based on `<s>`, `[INST]` and `<<SYS>>`.\n"
"If `llama3` - use the llama3 prompt style from the llama_index.\n"
"If `llama4` - use the llama3 prompt style from the llama_index.\n"
"If `tag` - use the `tag` prompt style. It should look like `<|role|>: message`. \n"
"If `mistral` - use the `mistral prompt style. It shoudl look like <s>[INST] {System Prompt} [/INST]</s>[INST] { UserInstructions } [/INST]\n"
"If `openchat` - use the openchat prompt style. Based on `GPT4 Correct User` and `GPT4 Correct Assistant:`\n"
"`llama2` is the historic behaviour. `default` might work better with your custom models."
),
)