abi

secret-llama

2,672

// summary

Secret Llama is an entirely in-browser chatbot that allows users to run open-source models like Llama 3 and Mistral locally. Because the application operates directly within the browser, all conversation data remains private and no server installation is required. The platform provides a user-friendly interface that functions offline while leveraging WebGPU technology for performance.

// technical analysis

Secret Llama is a browser-based LLM chatbot designed to provide a fully private, offline-capable AI experience by leveraging the WebGPU-powered web-llm inference engine. By executing models entirely within the user's browser, it eliminates the need for server-side infrastructure and ensures that sensitive conversation data never leaves the local machine. This architecture prioritizes user privacy and accessibility, though it necessitates a modern browser with WebGPU support and sufficient system RAM to handle the specific model sizes.

// key highlights

Operates entirely within the browser, ensuring that all user conversation data remains local and private.

Functions completely offline, removing the dependency on external servers or internet connectivity after the initial load.

Utilizes the web-llm inference engine to provide high-performance model execution directly on the client's hardware.

Supports a variety of open-source models including Llama 3, Mistral, and TinyLlama to offer flexible performance options.

Provides a user-friendly interface comparable to ChatGPT, making advanced open-source LLMs accessible to non-technical users.

// use cases

Running private LLMs entirely within a web browser

Executing open-source models like Llama 3 and Mistral offline

Providing a ChatGPT-like interface without server-side data processing

// getting started

To begin using Secret Llama, simply visit the hosted website in a WebGPU-compatible browser like Chrome or Edge. If you wish to modify or build the project locally, clone the repository, run 'yarn' to install dependencies, and use 'yarn dev' to launch the development environment with live reload.