mirror of
https://github.com/zylon-ai/private-gpt.git
synced 2025-12-22 23:22:57 +01:00
docs: remove list-llm pages
This commit is contained in:
parent
ce0748af85
commit
b7d2941cb1
2 changed files with 0 additions and 130 deletions
|
|
@ -84,14 +84,6 @@ navigation:
|
||||||
contents:
|
contents:
|
||||||
- page: Summarize
|
- page: Summarize
|
||||||
path: ./docs/pages/recipes/summarize.mdx
|
path: ./docs/pages/recipes/summarize.mdx
|
||||||
# Small code snippet or example of usage to help users
|
|
||||||
- tab: tested-combinations
|
|
||||||
layout:
|
|
||||||
- section: Choice of LLM
|
|
||||||
contents:
|
|
||||||
# TODO: add combinations
|
|
||||||
- page: List of LLMs
|
|
||||||
path: ./docs/pages/tested-combinations/list-llm.mdx
|
|
||||||
# More advanced usage of PrivateGPT, by API
|
# More advanced usage of PrivateGPT, by API
|
||||||
- tab: api-reference
|
- tab: api-reference
|
||||||
layout:
|
layout:
|
||||||
|
|
|
||||||
|
|
@ -1,122 +0,0 @@
|
||||||
# List of working LLM
|
|
||||||
|
|
||||||
**Do you have any working combination of LLM and embeddings?**
|
|
||||||
|
|
||||||
Please open a PR to add it to the list, and come on our Discord to tell us about it!
|
|
||||||
|
|
||||||
## Prompt style
|
|
||||||
|
|
||||||
LLMs might have been trained with different prompt styles.
|
|
||||||
The prompt style is the way the prompt is written, and how the system message is injected in the prompt.
|
|
||||||
|
|
||||||
For example, `llama2` looks like this:
|
|
||||||
```text
|
|
||||||
<s>[INST] <<SYS>>
|
|
||||||
{{ system_prompt }}
|
|
||||||
<</SYS>>
|
|
||||||
|
|
||||||
{{ user_message }} [/INST]
|
|
||||||
```
|
|
||||||
|
|
||||||
While `default` (the `llama_index` default) looks like this:
|
|
||||||
```text
|
|
||||||
system: {{ system_prompt }}
|
|
||||||
user: {{ user_message }}
|
|
||||||
assistant: {{ assistant_message }}
|
|
||||||
```
|
|
||||||
|
|
||||||
The "`tag`" style looks like this:
|
|
||||||
|
|
||||||
```text
|
|
||||||
<|system|>: {{ system_prompt }}
|
|
||||||
<|user|>: {{ user_message }}
|
|
||||||
<|assistant|>: {{ assistant_message }}
|
|
||||||
```
|
|
||||||
|
|
||||||
The "`mistral`" style looks like this:
|
|
||||||
|
|
||||||
```text
|
|
||||||
<s>[INST] You are an AI assistant. [/INST]</s>[INST] Hello, how are you doing? [/INST]
|
|
||||||
```
|
|
||||||
|
|
||||||
The "`chatml`" style looks like this:
|
|
||||||
```text
|
|
||||||
<|im_start|>system
|
|
||||||
{{ system_prompt }}<|im_end|>
|
|
||||||
<|im_start|>user"
|
|
||||||
{{ user_message }}<|im_end|>
|
|
||||||
<|im_start|>assistant
|
|
||||||
{{ assistant_message }}
|
|
||||||
```
|
|
||||||
|
|
||||||
Some LLMs will not understand these prompt styles, and will not work (returning nothing).
|
|
||||||
You can try to change the prompt style to `default` (or `tag`) in the settings, and it will
|
|
||||||
change the way the messages are formatted to be passed to the LLM.
|
|
||||||
|
|
||||||
## Example of configuration
|
|
||||||
|
|
||||||
You might want to change the prompt depending on the language and model you are using.
|
|
||||||
|
|
||||||
### English, with instructions
|
|
||||||
|
|
||||||
`settings-en.yaml`:
|
|
||||||
```yml
|
|
||||||
local:
|
|
||||||
llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.1-GGUF
|
|
||||||
llm_hf_model_file: mistral-7b-instruct-v0.1.Q4_K_M.gguf
|
|
||||||
embedding_hf_model_name: BAAI/bge-small-en-v1.5
|
|
||||||
prompt_style: "llama2"
|
|
||||||
```
|
|
||||||
|
|
||||||
### French, with instructions
|
|
||||||
|
|
||||||
`settings-fr.yaml`:
|
|
||||||
```yml
|
|
||||||
local:
|
|
||||||
llm_hf_repo_id: TheBloke/Vigogne-2-7B-Instruct-GGUF
|
|
||||||
llm_hf_model_file: vigogne-2-7b-instruct.Q4_K_M.gguf
|
|
||||||
embedding_hf_model_name: dangvantuan/sentence-camembert-base
|
|
||||||
prompt_style: "default"
|
|
||||||
# prompt_style: "tag" # also works
|
|
||||||
# The default system prompt is injected only when the `prompt_style` != default, and there are no system message in the discussion
|
|
||||||
# default_system_prompt: Vous êtes un assistant IA qui répond à la question posée à la fin en utilisant le contexte suivant. Si vous ne connaissez pas la réponse, dites simplement que vous ne savez pas, n'essayez pas d'inventer une réponse. Veuillez répondre exclusivement en français.
|
|
||||||
```
|
|
||||||
|
|
||||||
You might want to change the prompt as the one above might not directly answer your question.
|
|
||||||
You can read online about how to write a good prompt, but in a nutshell, make it (extremely) directive.
|
|
||||||
|
|
||||||
You can try and troubleshot your prompt by writing multiline requests in the UI, while
|
|
||||||
writing your interaction with the model, for example:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Tu es un programmeur senior qui programme en python et utilise le framework fastapi. Ecrit moi un serveur qui retourne "hello world".
|
|
||||||
```
|
|
||||||
|
|
||||||
Another example:
|
|
||||||
```text
|
|
||||||
Context: None
|
|
||||||
Situation: tu es au milieu d'un champ.
|
|
||||||
Tache: va a la rivière, en bas du champ.
|
|
||||||
Décrit comment aller a la rivière.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Optimised Models
|
|
||||||
GodziLLa2-70B LLM (English, rank 2 on HuggingFace OpenLLM Leaderboard), bge large Embedding Model (rank 1 on HuggingFace MTEB Leaderboard)
|
|
||||||
`settings-optimised.yaml`:
|
|
||||||
```yml
|
|
||||||
local:
|
|
||||||
llm_hf_repo_id: TheBloke/GodziLLa2-70B-GGUF
|
|
||||||
llm_hf_model_file: godzilla2-70b.Q4_K_M.gguf
|
|
||||||
embedding_hf_model_name: BAAI/bge-large-en
|
|
||||||
prompt_style: "llama2"
|
|
||||||
```
|
|
||||||
### German speaking model
|
|
||||||
`settings-de.yaml`:
|
|
||||||
```yml
|
|
||||||
local:
|
|
||||||
llm_hf_repo_id: TheBloke/em_german_leo_mistral-GGUF
|
|
||||||
llm_hf_model_file: em_german_leo_mistral.Q4_K_M.gguf
|
|
||||||
embedding_hf_model_name: T-Systems-onsite/german-roberta-sentence-transformer-v2
|
|
||||||
#llama, default or tag
|
|
||||||
prompt_style: "default"
|
|
||||||
```
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue