Next version of PrivateGPT (#1077)

* Dockerize private-gpt

* Use port 8001 for local development

* Add setup script

* Add CUDA Dockerfile

* Create README.md

* Make the API use OpenAI response format

* Truncate prompt

* refactor: add models and __pycache__ to .gitignore

* Better naming

* Update readme

* Move models ignore to it's folder

* Add scaffolding

* Apply formatting

* Fix tests

* Working sagemaker custom llm

* Fix linting

* Fix linting

* Enable streaming

* Allow all 3.11 python versions

* Use llama 2 prompt format and fix completion

* Restructure (#3)

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>

* Fix Dockerfile

* Use a specific build stage

* Cleanup

* Add FastAPI skeleton

* Cleanup openai package

* Fix DI and tests

* Split tests and tests with coverage

* Remove old scaffolding

* Add settings logic (#4)

* Add settings logic

* Add settings for sagemaker

---------

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>

* Local LLM (#5)

* Add settings logic

* Add settings for sagemaker

* Add settings-local-example.yaml

* Delete terraform files

* Refactor tests to use fixtures

* Join deltas

* Add local model support

---------

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>

* Update README.md

* Fix tests

* Version bump

* Enable simple llamaindex observability (#6)

* Enable simple llamaindex observability

* Improve code through linting

* Update README.md

* Move to async (#7)

* Migrate implementation to use asyncio

* Formatting

* Cleanup

* Linting

---------

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>

* Query Docs and gradio UI

* Remove unnecessary files

* Git ignore chromadb folder

* Async migration + DI Cleanup

* Fix tests

* Add integration test

* Use fastapi responses

* Retrieval service with partial implementation

* Cleanup

* Run formatter

* Fix types

* Fetch nodes asynchronously

* Install local dependencies in tests

* Install ui dependencies in tests

* Install dependencies for llama-cpp

* Fix sudo

* Attempt to fix cuda issues

* Attempt to fix cuda issues

* Try to reclaim some space from ubuntu machine

* Retrieval with context

* Fix lint and imports

* Fix mypy

* Make retrieval API a POST

* Make Completions body a dataclass

* Fix LLM chat message order

* Add Query Chunks to Gradio UI

* Improve rag query prompt

* Rollback CI Changes

* Move to sync code

* Using Llamaindex abstraction for query retrieval

* Fix types

* Default to CONDENSED chat mode for contextualized chat

* Rename route function

* Add Chat endpoint

* Remove webhooks

* Add IntelliJ run config to gitignore

* .gitignore applied

* Sync chat completion

* Refactor total

* Typo in context_files.py

* Add embeddings component and service

* Remove wrong dataclass from IngestService

* Filter by context file id implementation

* Fix typing

* Implement context_filter and separate from the bool use_context in the API

* Change chunks api to avoid conceptual class of the context concept

* Deprecate completions and fix tests

* Remove remaining dataclasses

* Use embedding component in ingest service

* Fix ingestion to have multipart and local upload

* Fix ingestion API

* Add chunk tests

* Add configurable paths

* Cleaning up

* Add more docs

* IngestResponse includes a list of IngestedDocs

* Use IngestedDoc in the Chunk document reference

* Rename ingest routes to ingest_router.py

* Fix test working directory for intellij

* Set testpaths for pytest

* Remove unused as_chat_engine

* Add .fleet ide to gitignore

* Make LLM and Embedding model configurable

* Fix imports and checks

* Let local_data folder exist empty in the repository

* Don't use certain metadata in LLM

* Remove long lines

* Fix windows installation

* Typos

* Update poetry.lock

* Add TODO for linux

* Script and first version of docs

* No jekill build

* Fix relative url to openapi json

* Change default docs values

* Move chromadb dependency to the general group

* Fix tests to use separate local_data

* Create CNAME

* Update CNAME

* Fix openapi.json relative path

* PrivateGPT logo

* WIP OpenAPI documentation metadata

* Add ingest script (#11)

* Add ingest script

* Fix broken name refactor

* Add ingest docs and Makefile script

* Linting

* Move transformers to main dependency

* Move torch to main dependencies

* Don't load HuggingFaceEmbedding in tests

* Fix lint

---------

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>

* Rename file to camel_case

* Commit settings-local.yaml

* Move documentation to public docs

* Fix docker image for linux

* Installation and Running the Server documentation

* Move back to docs folder, as it is the only supported by github pages

* Delete CNAME

* Create CNAME

* Delete CNAME

* Create CNAME

* Improved API documentation

* Fix lint

* Completions documentation

* Updated openapi scheme

* Ingestion API doc

* Minor doc changes

* Updated openapi scheme

* Chunks API documentation

* Embeddings and Health API, and homogeneous responses

* Revamp README with new skeleton of content

* More docs

* PrivateGPT logo

* Improve UI

* Update ingestion docu

* Update README with new sections

* Use context window in the retriever

* Gradio Documentation

* Add logo to UI

* Include Contributing and Community sections to README

* Update links to resources in the README

* Small README.md updates

* Wrap lines of README.md

* Don't put health under /v1

* Add copy button to Chat

* Architecture documentation

* Updated openapi.json

* Updated openapi.json

* Updated openapi.json

* Change UI label

* Update documentation

* Add releases link to README.md

* Gradio avatar and stop debug

* Readme update

* Clean old files

* Remove unused terraform checks

* Update twitter link.

* Disable minimum coverage

* Clean install message in README.md

---------

Co-authored-by: Pablo Orgaz <pablo@Pablos-MacBook-Pro.local>
Co-authored-by: Iván Martínez <ivanmartit@gmail.com>
Co-authored-by: RubenGuerrero <ruben.guerrero@boopos.com>
Co-authored-by: Daniel Gallego Vico <daniel.gallego@bq.com>

scripts/__init__.py

@@ -0,0 +1 @@
+"""PrivateGPT scripts."""

scripts/extract_openapi.py

@@ -0,0 +1,33 @@
+import argparse
+import json
+import sys
+import yaml
+from uvicorn.importer import import_from_string
+
+parser = argparse.ArgumentParser(prog="extract_openapi.py")
+parser.add_argument("app", help='App import string. Eg. "main:app"', default="main:app")
+parser.add_argument("--app-dir", help="Directory containing the app", default=None)
+parser.add_argument(
+    "--out", help="Output file ending in .json or .yaml", default="openapi.yaml"
+)
+
+if __name__ == "__main__":
+    args = parser.parse_args()
+
+    if args.app_dir is not None:
+        print(f"adding {args.app_dir} to sys.path")
+        sys.path.insert(0, args.app_dir)
+
+    print(f"importing app from {args.app}")
+    app = import_from_string(args.app)
+    openapi = app.openapi()
+    version = openapi.get("openapi", "unknown version")
+
+    print(f"writing openapi spec v{version}")
+    with open(args.out, "w") as f:
+        if args.out.endswith(".json"):
+            json.dump(openapi, f, indent=2)
+        else:
+            yaml.dump(openapi, f, sort_keys=False)
+
+    print(f"spec written to {args.out}")

scripts/ingest_folder.py

@@ -0,0 +1,45 @@
+import argparse
+import sys
+from pathlib import Path
+
+from private_gpt.di import root_injector
+from private_gpt.server.ingest.ingest_service import IngestService
+from private_gpt.server.ingest.ingest_watcher import IngestWatcher
+
+ingest_service = root_injector.get(IngestService)
+
+
+parser = argparse.ArgumentParser(prog="ingest_folder.py")
+parser.add_argument("folder", help="Folder to ingest")
+parser.add_argument(
+    "--watch",
+    help="Watch for changes",
+    action=argparse.BooleanOptionalAction,
+    default=False,
+)
+args = parser.parse_args()
+
+
+def _recursive_ingest_folder(folder_path: Path) -> None:
+    for file_path in folder_path.iterdir():
+        if file_path.is_file():
+            _do_ingest(file_path)
+        elif file_path.is_dir():
+            _recursive_ingest_folder(file_path)
+
+
+def _do_ingest(changed_path: Path) -> None:
+    if changed_path.exists():
+        print(f"\nIngesting {changed_path}")
+        ingest_service.ingest(changed_path.name, changed_path)
+
+
+path = Path(args.folder)
+if not path.exists():
+    raise ValueError(f"Path {args.folder} does not exist")
+
+_recursive_ingest_folder(path)
+if args.watch:
+    print(f"Watching {args.folder} for changes, press Ctrl+C to stop...")
+    watcher = IngestWatcher(args.folder, _do_ingest)
+    watcher.start()

scripts/setup

@@ -0,0 +1,30 @@
+#!/usr/bin/env python3
+import os
+
+from huggingface_hub import hf_hub_download, snapshot_download
+
+from private_gpt.paths import models_path, models_cache_path
+from private_gpt.settings.settings import settings
+
+os.makedirs(models_path, exist_ok=True)
+embedding_path = models_path / "embedding"
+
+print(f"Downloading embedding {settings.local.embedding_hf_model_name}")
+snapshot_download(
+    repo_id=settings.local.embedding_hf_model_name,
+    cache_dir=models_cache_path,
+    local_dir=embedding_path,
+)
+print("Embedding model downloaded!")
+print("Downloading models for local execution...")
+
+# Download LLM and create a symlink to the model file
+hf_hub_download(
+    repo_id=settings.local.llm_hf_repo_id,
+    filename=settings.local.llm_hf_model_file,
+    cache_dir=models_cache_path,
+    local_dir=models_path,
+)
+
+print("LLM model downloaded!")
+print("Setup done")