Somashekar B R
62985df555
resolved dependecies
2024-09-26 20:24:06 +05:30
Javier Martinez
5851b02378
feat: update llama-index + dependencies ( #2092 )
...
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled
* chore: update libraries
* fix: mypy
* chore: more updates
* fix: mypy/black
* chore: fix docker warnings
* fix: mypy
* fix: black
2024-09-26 16:29:52 +02:00
Somashekar B R
cecec30ca8
fixed test black error
2024-09-24 23:37:03 +05:30
Somashekar B R
9c3590e555
Added embedded model option for fireworks \n Added documentation for fireworks
2024-09-24 23:27:40 +05:30
Somashekar B R
519c48b70b
Merge branch 'main' of github.com:somashekhar161/private-gpt into feat/fireworks-integration
2024-09-24 22:38:45 +05:30
J
fa3c30661d
fix: Add default mode option to settings ( #2078 )
...
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled
* Add default mode option to settings
* Revise default_mode to Literal (enum) and add to settings.yaml
* Revise to pass make check/test
* Default mode: RAG
---------
Co-authored-by: Jason <jason@sowinsight.solutions>
2024-09-24 08:33:02 +02:00
Liam Dowd
f9182b3a86
feat: Adding MistralAI mode ( #2065 )
...
* Adding MistralAI mode
* Update embedding_component.py
* Update ui.py
* Update settings.py
* Update embedding_component.py
* Update settings.py
* Update settings.py
* Update settings-mistral.yaml
* Update llm_component.py
* Update settings-mistral.yaml
* Update settings.py
* Update settings.py
* Update ui.py
* Update embedding_component.py
* Delete settings-mistral.yaml
---------
Co-authored-by: SkiingIsFun123 <101684827+SkiingIsFun123@users.noreply.github.com>
Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-09-24 08:31:30 +02:00
Somashekar B R
2d5865e773
FEAT: Added fixed UI mode name
2024-09-20 14:49:02 +05:30
Somashekar B R
a38675b844
FEAT: Added Fireworks Integration
2024-09-20 14:15:34 +05:30
Somashekar B R
f1ad995124
FEAT: Added Fireworks Integration
2024-09-20 14:13:09 +05:30
Javier Martinez
54659588b5
fix: nomic embeddings ( #2030 )
...
* fix: allow to configure trust_remote_code
based on: https://github.com/zylon-ai/private-gpt/issues/1893#issuecomment-2118629391
* fix: nomic hf embeddings
2024-08-01 09:43:30 +02:00
Javier Martinez
8119842ae6
feat(recipe): add our first recipe Summarize ( #2028 )
...
publish docs / publish-docs (push) Waiting to run
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions
* feat: add summary recipe
* test: add summary tests
* docs: move all recipes docs
* docs: add recipes and summarize doc
* docs: update openapi reference
* refactor: split method in two method (summary)
* feat: add initial summarize ui
* feat: add mode explanation
* fix: mypy
* feat: allow to configure async property in summarize
* refactor: move modes to enum and update mode explanations
* docs: fix url
* docs: remove list-llm pages
* docs: remove double header
* fix: summary description
2024-07-31 16:53:27 +02:00
Javier Martinez
e54a8fe043
fix: prevent to ingest local files (by default) ( #2010 )
...
* feat: prevent to local ingestion (by default) and add white-list
* docs: add local ingestion warning
* docs: add missing comment
* fix: update exception error
* fix: black
2024-07-31 14:33:46 +02:00
Robert Hirsch
d080969407
added llama3 prompt ( #1962 )
...
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions
* added llama3 prompt
* more fixes to pass tests; changed type VectorStore -> BasePydanticVectorStore, see https://github.com/run-llama/llama_index/blob/main/CHANGELOG.md#2024-05-14
* fix: new llama3 prompt
---------
Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-29 17:28:00 +02:00
Javier Martinez
20bad17c98
feat(llm): autopull ollama models ( #2019 )
...
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions
publish docs / publish-docs (push) Has been cancelled
* chore: update ollama (llm)
* feat: allow to autopull ollama models
* fix: mypy
* chore: install always ollama client
* refactor: check connection and pull ollama method to utils
* docs: update ollama config with autopulling info
2024-07-29 13:25:42 +02:00
Jackson
43cc31f740
feat(vectordb): Milvus vector db Integration ( #1996 )
...
* integrate Milvus into Private GPT
* adjust milvus settings
* update doc info and reformat
* adjust milvus initialization
* adjust import error
* mionr update
* adjust format
* adjust the db storing path
* update doc
2024-07-18 10:55:45 +02:00
Proger666
2612928839
feat(vectorstore): Add clickhouse support as vectore store ( #1883 )
...
* Added ClickHouse vector sotre support
* port fix
* updated lock file
* fix: mypy
* fix: mypy
---------
Co-authored-by: Valery Denisov <valerydenisov@double.cloud>
Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 16:18:22 +02:00
uw4
fc13368bc7
feat(llm): Support for Google Gemini LLMs and Embeddings ( #1965 )
...
publish docs / publish-docs (push) Waiting to run
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions
* Support for Google Gemini LLMs and Embeddings
Initial support for Gemini, enables usage of Google LLMs and embedding models (see settings-gemini.yaml)
Install via
poetry install --extras "llms-gemini embeddings-gemini"
Notes:
* had to bump llama-index-core to later version that supports Gemini
* poetry --no-update did not work: Gemini/llama_index seem to require more (transient) updates to make it work...
* fix: crash when gemini is not selected
* docs: add gemini llm
---------
Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 11:47:36 +02:00
Yevhenii Semendiak
3b3e96ad6c
Allow parameterizing OpenAI embeddings component (api_base, key, model) ( #1920 )
...
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled
* Allow parameterizing OpenAI embeddings component (api_base, key, model)
* Update settings
* Update description
2024-05-17 09:52:50 +02:00
jcbonnet-fwd
45df99feb7
Add timeout parameter for better support of openailike LLM tools on local computer (like LM Studio). ( #1858 )
...
release-please / release-please (push) Has been cancelled
tests / setup (push) Has been cancelled
tests / ${{ matrix.quality-command }} (black) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (mypy) (push) Has been cancelled
tests / ${{ matrix.quality-command }} (ruff) (push) Has been cancelled
tests / test (push) Has been cancelled
tests / all_checks_passed (push) Has been cancelled
feat(llm): Improve settings of the OpenAILike LLM
2024-05-10 16:44:08 +02:00
icsy7867
e21bf20c10
feat: prompt_style applied to all LLMs + extra LLM params. ( #1835 )
...
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions
* Updated prompt_style to be moved to the main LLM setting since all LLMs from llama_index can utilize this. I also included temperature, context window size, max_tokens, max_new_tokens into the openailike to help ensure the settings are consistent from the other implementations.
* Removed prompt_style from llamacpp entirely
* Fixed settings-local.yaml to include prompt_style in the LLM settings instead of llamacpp.
2024-04-30 09:53:10 +02:00
imartinez
49ef729abc
Allow passing HF access token to download tokenizer. Fallback to default tokenizer.
2024-04-19 15:38:25 +02:00
imartinez
f469b4619d
Add required Ollama setting
2024-04-02 18:27:57 +02:00
Robin Boone
b3b0140e24
feat(llm): Ollama LLM-Embeddings decouple + longer keep_alive settings ( #1800 )
2024-04-02 16:23:10 +02:00
machatschek
83adc12a8e
feat(RAG): Introduce SentenceTransformer Reranker ( #1810 )
2024-04-02 10:29:51 +02:00
icsy7867
087cb0b7b7
feat(rag): expose similarity_top_k and similarity_score to settings ( #1771 )
...
* Added RAG settings to settings.py, vector_store and chat_service to add similarity_top_k and similarity_score
* Updated settings in vector and chat service per Ivans request
* Updated code for mypy
2024-03-20 22:25:26 +01:00
Iván Martínez
6f6c785dac
feat(llm): Ollama timeout setting ( #1773 )
...
* added request_timeout to ollama, default set to 30.0 in settings.yaml and settings-ollama.yaml
* Update settings-ollama.yaml
* Update settings.yaml
* updated settings.py and tidied up settings-ollama-yaml
* feat(UI): Faster startup and document listing (#1763 )
* fix(ingest): update script label (#1770 )
huggingface -> Hugging Face
* Fix lint errors
---------
Co-authored-by: Stephen Gresham <steve@gresham.id.au>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-03-20 21:33:46 +01:00
Brett England
134fc54d7d
feat(ingest): Created a faster ingestion mode - pipeline ( #1750 )
...
* Unify pgvector and postgres connection settings
* Remove local changes
* Update file pgvector->postgres
* postgresql should be postgres
* Adding pipeline ingestion mode
* disable hugging face parallelism. Continue on file to doc transform failure
* Semaphore to limit docq async workers. ETA reporting
2024-03-19 21:24:46 +01:00
Otto L
1efac6a3fe
feat(llm - embed): Add support for Azure OpenAI ( #1698 )
...
* Add support for Azure OpenAI
* fix: wrong default api_version
Should be dashes instead of underscores.
see: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
* fix: code styling
applied "make check" changes
* refactor: extend documentation
* mention azopenai as available option and extras
* add recommended section
* include settings-azopenai.yaml configuration file
* fix: documentation
2024-03-15 16:49:50 +01:00
Brett England
63de7e4930
feat: unify settings for vector and nodestore connections to PostgreSQL ( #1730 )
...
* Unify pgvector and postgres connection settings
* Remove local changes
* Update file pgvector->postgres
2024-03-15 09:55:17 +01:00
Brett England
68b3a34b03
feat(nodestore): add Postgres for the doc and index store ( #1706 )
...
* Adding Postgres for the doc and index store
* Adding documentation. Rename postgres database local->simple. Postgres storage dependencies
* Update documentation for postgres storage
* Renaming feature to nodestore
* update docstore -> nodestore in doc
* missed some docstore changes in doc
* Updated poetry.lock
* Formatting updates to pass ruff/black checks
* Correction to unreachable code!
* Format adjustment to pass black test
* Adjust extra inclusion name for vector pg
* extra dep change for pg vector
* storage-postgres -> storage-nodestore-postgres
* Hash change on poetry lock
2024-03-14 17:12:33 +01:00
icsy7867
02dc83e8e9
feat(llm): adds serveral settings for llamacpp and ollama ( #1703 )
2024-03-11 22:51:05 +01:00
Iván Martínez
45f05711eb
feat: Upgrade to LlamaIndex to 0.10 ( #1663 )
...
* Extract optional dependencies
* Separate local mode into llms-llama-cpp and embeddings-huggingface for clarity
* Support Ollama embeddings
* Upgrade to llamaindex 0.10.14. Remove legacy use of ServiceContext in ContextChatEngine
* Fix vector retriever filters
2024-03-06 17:51:30 +01:00
TQ
cd40e3982b
feat(Vector): support pgvector ( #1624 )
2024-02-20 15:29:26 +01:00
Iván Martínez
aa13afde07
feat(UI): Select file to Query or Delete + Delete ALL ( #1612 )
...
---------
Co-authored-by: Robin Boone <rboone@sofics.com>
2024-02-16 17:36:09 +01:00
Ygal Blum
6bbec79583
feat(llm): Add support for Ollama LLM ( #1526 )
2024-02-09 15:50:50 +01:00
CognitiveTech
e326126d0d
feat: add mistral + chatml prompts ( #1426 )
2024-01-16 22:51:14 +01:00
Matthew Hill
2d27a9f956
feat(llm): Add openailike llm mode ( #1447 )
...
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.
Implements #1424
2023-12-26 10:26:08 +01:00
Iván Martínez
4780540870
feat(settings): Configurable context_window and tokenizer ( #1437 )
2023-12-21 14:49:35 +01:00
3ly-13
145f3ec9f4
feat(ui): Allows User to Set System Prompt via "Additional Options" in Chat Interface ( #1353 )
2023-12-10 19:45:14 +01:00
3ly-13
a072a40a7c
Allow setting OpenAI model in settings ( #1386 )
...
feat(settings): Allow setting openai model to be used. Default to GPT 3.5
2023-12-09 20:13:00 +01:00
Louis Melchior
a3ed14c58f
feat(llm): drop default_system_prompt ( #1385 )
...
As discussed on Discord, the decision has been made to remove the system prompts by default, to better segregate the API and the UI usages.
A concurrent PR (#1353 ) is enabling the dynamic setting of a system prompt in the UI.
Therefore, if UI users want to use a custom system prompt, they can specify one directly in the UI.
If the API users want to use a custom prompt, they can pass it directly into their messages that they are passing to the API.
In the highlight of the two use case above, it becomes clear that default system_prompt does not need to exist.
2023-12-08 23:13:51 +01:00
lopagela
56af625d71
Fix the parallel ingestion mode, and make it available through conf ( #1336 )
...
* Fix the parallel ingestion mode, and make it available through conf
Also updated the documentation to show how to configure the ingest mode.
* PR feedback: redirect to documentation
2023-11-30 11:41:55 +01:00
Gianni Acquisto
9c192ddd73
Added max_new_tokens as a config option to llm yaml block ( #1317 )
...
* added max_new_tokens as a configuration option to the llm block in settings
* Update fern/docs/pages/manual/settings.mdx
Co-authored-by: lopagela <lpglm@orange.fr>
* Update private_gpt/settings/settings.py
Add default value for max_new_tokens = 256
Co-authored-by: lopagela <lpglm@orange.fr>
* Addressed location of docs comment
* reformatting from running 'make check'
* remove default config value from settings.yaml
---------
Co-authored-by: lopagela <lpglm@orange.fr>
2023-11-26 19:17:29 +01:00
lopagela
bafdd3baf1
Ingestion Speedup Multiple strategy ( #1309 )
2023-11-25 20:12:09 +01:00
Iván Martínez
944c43bfa8
Multi language support - fern debug ( #1307 )
...
---------
Co-authored-by: Louis <lpglm@orange.fr>
Co-authored-by: LeMoussel <cnhx27@gmail.com>
2023-11-25 14:34:23 +01:00
Iván Martínez
2a417d2f61
Fix/qdrant support ( #1253 )
...
* Disable check same thread by default to enable disk-based Qdrant local client to work
2023-11-16 13:29:17 +01:00
Anush
03d1ae6d70
feat: Qdrant support ( #1228 )
...
* feat: Qdrant support
* Update private_gpt/components/vector_store/vector_store_component.py
2023-11-13 21:23:26 +01:00
Iván Martínez
86fc4781d8
Fix openai setting literal ( #1221 )
2023-11-12 22:29:26 +01:00
Pablo Orgaz
022bd718e3
fix: Remove global state ( #1216 )
...
* Remove all global settings state
* chore: remove autogenerated class
* chore: cleanup
* chore: merge conflicts
2023-11-12 22:20:36 +01:00
