SystemSh0cker/private-gpt
1
0
Fork 0
mirror of https://github.com/zylon-ai/private-gpt.git synced 2025-12-22 07:40:12 +01:00
Code Issues Projects Releases Packages Activity

feat(llm): Support for Google Gemini LLMs and Embeddings (#1965)
Some checks are pending
publish docs / publish-docs (push) Waiting to run
release-please / release-please (push) Waiting to run
tests / setup (push) Waiting to run
tests / ${{ matrix.quality-command }} (black) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (mypy) (push) Blocked by required conditions
tests / ${{ matrix.quality-command }} (ruff) (push) Blocked by required conditions
tests / test (push) Blocked by required conditions
tests / all_checks_passed (push) Blocked by required conditions

Browse source 
* Support for Google Gemini LLMs and Embeddings

Initial support for Gemini, enables usage of Google LLMs and embedding models (see settings-gemini.yaml)

Install via
poetry install --extras "llms-gemini embeddings-gemini"

Notes:
* had to bump llama-index-core to later version that supports Gemini
* poetry --no-update did not work: Gemini/llama_index seem to require more (transient) updates to make it work...

* fix: crash when gemini is not selected

* docs: add gemini llm

---------

Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
This commit is contained in:
uw4 2024-07-08 11:47:36 +02:00 • committed by GitHub
parent 19a7c065ef
commit fc13368bc7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 382 additions and 59 deletions
Download patch file Download diff file Expand all files Collapse all files

26
private_gpt/settings/settings.py
View file

@ -82,7 +82,14 @@ class DataSettings(BaseModel):
class LLMSettings(BaseModel):
mode: Literal[
"llamacpp", "openai", "openailike", "azopenai", "sagemaker", "mock", "ollama"
"llamacpp",
"openai",
"openailike",
"azopenai",
"sagemaker",
"mock",
"ollama",
"gemini",
]
max_new_tokens: int = Field(
256,
@ -157,7 +164,9 @@ class HuggingFaceSettings(BaseModel):
class EmbeddingSettings(BaseModel):
mode: Literal["huggingface", "openai", "azopenai", "sagemaker", "ollama", "mock"]
mode: Literal[
"huggingface", "openai", "azopenai", "sagemaker", "ollama", "mock", "gemini"
]
ingest_mode: Literal["simple", "batch", "parallel", "pipeline"] = Field(
"simple",
description=(
@ -220,6 +229,18 @@ class OpenAISettings(BaseModel):
)
class GeminiSettings(BaseModel):
api_key: str
model: str = Field(
"models/gemini-pro",
description="Google Model to use. Example: 'models/gemini-pro'.",
)
embedding_model: str = Field(
"models/embedding-001",
description="Google Embedding Model to use. Example: 'models/embedding-001'.",
)
class OllamaSettings(BaseModel):
api_base: str = Field(
"http://localhost:11434",
@ -426,6 +447,7 @@ class Settings(BaseModel):
huggingface: HuggingFaceSettings
sagemaker: SagemakerSettings
openai: OpenAISettings
gemini: GeminiSettings
ollama: OllamaSettings
azopenai: AzureOpenAISettings
vectorstore: VectorstoreSettings