SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 07:40:12 +01:00
Code Issues Projects Releases Packages Activity

feat: prompt_style applied to all LLMs + extra LLM params. (#1835)
Some checks are pending
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions

Browse source 
* Updated prompt_style to be moved to the main LLM setting since all LLMs from llama_index can utilize this.  I also included temperature, context window size, max_tokens, max_new_tokens into the openailike to help ensure the settings are consistent from the other implementations.

* Removed prompt_style from llamacpp entirely

* Fixed settings-local.yaml to include prompt_style in the LLM settings instead of llamacpp.
This commit is contained in:
icsy7867 2024-04-30 03:53:10 -04:00 • committed by GitHub
parent c1802e7cf0
commit e21bf20c10
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 22 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

4
settings-local.yaml
View file

@ -8,9 +8,9 @@ llm:
max_new_tokens: 512
context_window: 3900
tokenizer: mistralai/Mistral-7B-Instruct-v0.2
prompt_style: "mistral"
llamacpp:
prompt_style: "mistral"
llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.2-GGUF
llm_hf_model_file: mistral-7b-instruct-v0.2.Q4_K_M.gguf
@ -24,4 +24,4 @@ vectorstore:
database: qdrant
qdrant:
path: local_data/private_gpt/qdrant
path: local_data/private_gpt/qdrant