diff --git a/fern/docs.yml b/fern/docs.yml index 3421c3b..e0a5c42 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -84,14 +84,6 @@ navigation: contents: - page: Summarize path: ./docs/pages/recipes/summarize.mdx - # Small code snippet or example of usage to help users - - tab: tested-combinations - layout: - - section: Choice of LLM - contents: - # TODO: add combinations - - page: List of LLMs - path: ./docs/pages/tested-combinations/list-llm.mdx # More advanced usage of PrivateGPT, by API - tab: api-reference layout: diff --git a/fern/docs/pages/tested-combinations/list-llm.mdx b/fern/docs/pages/tested-combinations/list-llm.mdx deleted file mode 100644 index 103867a..0000000 --- a/fern/docs/pages/tested-combinations/list-llm.mdx +++ /dev/null @@ -1,122 +0,0 @@ -# List of working LLM - -**Do you have any working combination of LLM and embeddings?** - -Please open a PR to add it to the list, and come on our Discord to tell us about it! - -## Prompt style - -LLMs might have been trained with different prompt styles. -The prompt style is the way the prompt is written, and how the system message is injected in the prompt. - -For example, `llama2` looks like this: -```text -[INST] <> -{{ system_prompt }} -<> - -{{ user_message }} [/INST] -``` - -While `default` (the `llama_index` default) looks like this: -```text -system: {{ system_prompt }} -user: {{ user_message }} -assistant: {{ assistant_message }} -``` - -The "`tag`" style looks like this: - -```text -<|system|>: {{ system_prompt }} -<|user|>: {{ user_message }} -<|assistant|>: {{ assistant_message }} -``` - -The "`mistral`" style looks like this: - -```text -[INST] You are an AI assistant. [/INST][INST] Hello, how are you doing? [/INST] -``` - -The "`chatml`" style looks like this: -```text -<|im_start|>system -{{ system_prompt }}<|im_end|> -<|im_start|>user" -{{ user_message }}<|im_end|> -<|im_start|>assistant -{{ assistant_message }} -``` - -Some LLMs will not understand these prompt styles, and will not work (returning nothing). -You can try to change the prompt style to `default` (or `tag`) in the settings, and it will -change the way the messages are formatted to be passed to the LLM. - -## Example of configuration - -You might want to change the prompt depending on the language and model you are using. - -### English, with instructions - -`settings-en.yaml`: -```yml -local: - llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.1-GGUF - llm_hf_model_file: mistral-7b-instruct-v0.1.Q4_K_M.gguf - embedding_hf_model_name: BAAI/bge-small-en-v1.5 - prompt_style: "llama2" -``` - -### French, with instructions - -`settings-fr.yaml`: -```yml -local: - llm_hf_repo_id: TheBloke/Vigogne-2-7B-Instruct-GGUF - llm_hf_model_file: vigogne-2-7b-instruct.Q4_K_M.gguf - embedding_hf_model_name: dangvantuan/sentence-camembert-base - prompt_style: "default" - # prompt_style: "tag" # also works - # The default system prompt is injected only when the `prompt_style` != default, and there are no system message in the discussion - # default_system_prompt: Vous êtes un assistant IA qui répond à la question posée à la fin en utilisant le contexte suivant. Si vous ne connaissez pas la réponse, dites simplement que vous ne savez pas, n'essayez pas d'inventer une réponse. Veuillez répondre exclusivement en français. -``` - -You might want to change the prompt as the one above might not directly answer your question. -You can read online about how to write a good prompt, but in a nutshell, make it (extremely) directive. - -You can try and troubleshot your prompt by writing multiline requests in the UI, while -writing your interaction with the model, for example: - -```text -Tu es un programmeur senior qui programme en python et utilise le framework fastapi. Ecrit moi un serveur qui retourne "hello world". -``` - -Another example: -```text -Context: None -Situation: tu es au milieu d'un champ. -Tache: va a la rivière, en bas du champ. -Décrit comment aller a la rivière. -``` - -### Optimised Models -GodziLLa2-70B LLM (English, rank 2 on HuggingFace OpenLLM Leaderboard), bge large Embedding Model (rank 1 on HuggingFace MTEB Leaderboard) -`settings-optimised.yaml`: -```yml -local: - llm_hf_repo_id: TheBloke/GodziLLa2-70B-GGUF - llm_hf_model_file: godzilla2-70b.Q4_K_M.gguf - embedding_hf_model_name: BAAI/bge-large-en - prompt_style: "llama2" -``` -### German speaking model -`settings-de.yaml`: -```yml -local: - llm_hf_repo_id: TheBloke/em_german_leo_mistral-GGUF - llm_hf_model_file: em_german_leo_mistral.Q4_K_M.gguf - embedding_hf_model_name: T-Systems-onsite/german-roberta-sentence-transformer-v2 - #llama, default or tag - prompt_style: "default" -```