SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 17:05:41 +01:00
Code Issues Projects Releases Packages Activity

feat: change ollama default model to llama3.1

Browse source
This commit is contained in:
Javier Martinez 2024-07-29 17:14:46 +02:00
parent d080969407
commit e2b319db92
No known key found for this signature in database
6 changed files with 7 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

4
fern/docs/pages/installation/installation.mdx
View file

@ -135,14 +135,14 @@ Now, start Ollama service (it will start a local inference server, serving both
ollama serve ollama serve
``` ```
Install the models to be used, the default settings-ollama.yaml is configured to user mistral 7b LLM (~4GB) and nomic-embed-text Embeddings (~275MB) Install the models to be used, the default settings-ollama.yaml is configured to user llama3.1 8b LLM (~4GB) and nomic-embed-text Embeddings (~275MB)
By default, PGPT will automatically pull models as needed. This behavior can be changed by modifying the `ollama.autopull_models` property. By default, PGPT will automatically pull models as needed. This behavior can be changed by modifying the `ollama.autopull_models` property.
In any case, if you want to manually pull models, run the following commands: In any case, if you want to manually pull models, run the following commands:
```bash ```bash
ollama pull mistral ollama pull llama3.1
ollama pull nomic-embed-text ollama pull nomic-embed-text
``` ```

2
fern/docs/pages/installation/troubleshooting.mdx
View file

@ -24,7 +24,7 @@ PrivateGPT uses the `AutoTokenizer` library to tokenize input text accurately. I
In your `settings.yaml` file, specify the model you want to use: In your `settings.yaml` file, specify the model you want to use:
```yaml ```yaml
llm: llm:
tokenizer: mistralai/Mistral-7B-Instruct-v0.2 tokenizer: meta-llama/Meta-Llama-3.1-8B-Instruct
``` ```
2. **Set Access Token for Gated Models:** 2. **Set Access Token for Gated Models:**
If you are using a gated model, ensure the `access_token` is set as mentioned in the previous section. If you are using a gated model, ensure the `access_token` is set as mentioned in the previous section.

2
settings-docker.yaml
View file

@ -20,7 +20,7 @@ sagemaker:
embedding_endpoint_name: ${PGPT_SAGEMAKER_EMBEDDING_ENDPOINT_NAME:} embedding_endpoint_name: ${PGPT_SAGEMAKER_EMBEDDING_ENDPOINT_NAME:}
ollama: ollama:
llm_model: ${PGPT_OLLAMA_LLM_MODEL:mistral} llm_model: ${PGPT_OLLAMA_LLM_MODEL:llama3.1}
embedding_model: ${PGPT_OLLAMA_EMBEDDING_MODEL:nomic-embed-text} embedding_model: ${PGPT_OLLAMA_EMBEDDING_MODEL:nomic-embed-text}
api_base: ${PGPT_OLLAMA_API_BASE:http://ollama:11434} api_base: ${PGPT_OLLAMA_API_BASE:http://ollama:11434}
embedding_api_base: ${PGPT_OLLAMA_EMBEDDING_API_BASE:http://ollama:11434} embedding_api_base: ${PGPT_OLLAMA_EMBEDDING_API_BASE:http://ollama:11434}

2
settings-ollama-pg.yaml
View file

@ -14,7 +14,7 @@ embedding:
embed_dim: 768 embed_dim: 768
ollama: ollama:
llm_model: mistral llm_model: llama3.1
embedding_model: nomic-embed-text embedding_model: nomic-embed-text
api_base: http://localhost:11434 api_base: http://localhost:11434

2
settings-ollama.yaml
View file

@ -11,7 +11,7 @@ embedding:
mode: ollama mode: ollama
ollama: ollama:
llm_model: mistral llm_model: llama3.1
embedding_model: nomic-embed-text embedding_model: nomic-embed-text
api_base: http://localhost:11434 api_base: http://localhost:11434
embedding_api_base: http://localhost:11434 # change if your embedding model runs on another ollama embedding_api_base: http://localhost:11434 # change if your embedding model runs on another ollama

2
settings.yaml
View file

@ -111,7 +111,7 @@ openai:
embedding_api_key: ${OPENAI_API_KEY:} embedding_api_key: ${OPENAI_API_KEY:}
ollama: ollama:
llm_model: llama2 llm_model: llama3.1
embedding_model: nomic-embed-text embedding_model: nomic-embed-text
api_base: http://localhost:11434 api_base: http://localhost:11434
embedding_api_base: http://localhost:11434 # change if your embedding model runs on another ollama embedding_api_base: http://localhost:11434 # change if your embedding model runs on another ollama