Merge pull request #87 from ItzCrazyKns/develop/1.4.0

Develop/1.4.0

README.md

@@ -10,8 +10,11 @@
 - [Installation](#installation)
   - [Getting Started with Docker (Recommended)](#getting-started-with-docker-recommended)
   - [Non-Docker Installation](#non-docker-installation)
+  - [Ollama connection errors](#ollama-connection-errors)
+- [One-Click Deployment](#one-click-deployment)
 - [Upcoming Features](#upcoming-features)
 - [Support Us](#support-us)
+  - [Donations](#donations)
 - [Contribution](#contribution)
 - [Help and Support](#help-and-support)
 
@@ -21,6 +24,8 @@ Perplexica is an open-source AI-powered searching tool or an AI-powered search e
 
 Using SearxNG to stay current and fully open source, Perplexica ensures you always get the most up-to-date information without compromising your privacy.
 
+Want to know more about its architecture and how it works? You can read it [here](https://github.com/ItzCrazyKns/Perplexica/tree/master/docs/architecture/README.md).
+
 ## Preview
 
 ![video-preview](.assets/perplexica-preview.gif)
@@ -38,7 +43,7 @@ Using SearxNG to stay current and fully open source, Perplexica ensures you alwa
   - **YouTube Search Mode:** Finds YouTube videos based on the search query.
   - **Wolfram Alpha Search Mode:** Answers queries that need calculations or data analysis using Wolfram Alpha.
   - **Reddit Search Mode:** Searches Reddit for discussions and opinions related to the query.
-- **Current Information:** Some search tools might give you outdated info because they use data from crawling bots and convert them into embeddings and store them in a index. Unlike them, Perplexica uses SearxNG, a metasearch engine to get the results and rerank and get the most relevent source out of it, ensuring you always get the latest information without the overhead of daily data updates.
+- **Current Information:** Some search tools might give you outdated info because they use data from crawling bots and convert them into embeddings and store them in a index. Unlike them, Perplexica uses SearxNG, a metasearch engine to get the results and rerank and get the most relevant source out of it, ensuring you always get the latest information without the overhead of daily data updates.
 
 It has many more features like image and video search. Some of the planned features are mentioned in [upcoming features](#upcoming-features).
 
@@ -59,13 +64,11 @@ There are mainly 2 ways of installing Perplexica - With Docker, Without Docker.
 
 4. Rename the `sample.config.toml` file to `config.toml`. For Docker setups, you need only fill in the following fields:
 
-   - `CHAT_MODEL`: The name of the LLM to use. Like `llama3:latest` (using Ollama), `gpt-3.5-turbo` (using OpenAI), etc.
-   - `CHAT_MODEL_PROVIDER`: The chat model provider, either `openai` or `ollama`. Depending upon which provider you use you would have to fill in the following fields:
+   - `OPENAI`: Your OpenAI API key. **You only need to fill this if you wish to use OpenAI's models**.
+   - `OLLAMA`: Your Ollama API URL. You should enter it as `http://host.docker.internal:PORT_NUMBER`. If you installed Ollama on port 11434, use `http://host.docker.internal:11434`. For other ports, adjust accordingly. **You need to fill this if you wish to use Ollama's models instead of OpenAI's**.
+   - `GROQ`: Your Groq API key. **You only need to fill this if you wish to use Groq's hosted models**
 
-     - `OPENAI`: Your OpenAI API key. **You only need to fill this if you wish to use OpenAI's models**.
-     - `OLLAMA`: Your Ollama API URL. You should enter it as `http://host.docker.internal:PORT_NUMBER`. If you installed Ollama on port 11434, use `http://host.docker.internal:11434`. For other ports, adjust accordingly. **You need to fill this if you wish to use Ollama's models instead of OpenAI's**.
-
-       **Note**: You can change these and use different models after running Perplexica as well from the settings page.
+     **Note**: You can change these after starting Perplexica from the settings dialog.
 
    - `SIMILARITY_MEASURE`: The similarity measure to use (This is filled by default; you can leave it as is if you are unsure about it.)
 
@@ -89,6 +92,20 @@ There are mainly 2 ways of installing Perplexica - With Docker, Without Docker.
 
 **Note**: Using Docker is recommended as it simplifies the setup process, especially for managing environment variables and dependencies.
 
+### Ollama connection errors
+
+If you're facing an Ollama connection error, it is often related to the backend not being able to connect to Ollama's API. How can you fix it? You can fix it by updating your Ollama API URL in the settings menu to the following:
+
+On Windows: `http://host.docker.internal:11434`<br>
+On Mac: `http://host.docker.internal:11434`<br>
+On Linux: `http://private_ip_of_computer_hosting_ollama:11434`
+
+You need to edit the ports accordingly.
+
+## One-Click Deployment
+
+[![Deploy to RepoCloud](https://d16t0pc4846x52.cloudfront.net/deploylobe.svg)](https://repocloud.io/details/?app_id=267)
+
 ## Upcoming Features
 
 - [ ] Finalizing Copilot Mode
@@ -99,7 +116,15 @@ There are mainly 2 ways of installing Perplexica - With Docker, Without Docker.
 
 ## Support Us
 
-If you find Perplexica useful, consider giving us a star on GitHub. This helps more people discover Perplexica and supports the development of new features. Your support is appreciated.
+If you find Perplexica useful, consider giving us a star on GitHub. This helps more people discover Perplexica and supports the development of new features. Your support is greatly appreciated.
+
+### Donations
+
+We also accept donations to help sustain our project. If you would like to contribute, you can use the following button to make a donation in cryptocurrency. Thank you for your support!
+
+<a href="https://nowpayments.io/donation?api_key=RFFKJH1-GRR4DQG-HFV1DZP-00G6MMK&source=lk_donation&medium=referral" target="_blank">
+  <img src="https://nowpayments.io/images/embeds/donation-button-white.svg" alt="Crypto donation button by NOWPayments">
+</a>
 
 ## Contribution
 

backend.dockerfile

@@ -1,4 +1,4 @@
-FROM node:alpine
+FROM node:buster-slim
 
 ARG SEARXNG_API_URL
 

docker-compose.yaml

@@ -3,8 +3,6 @@ services:
     build:
       context: .
       dockerfile: searxng.dockerfile
-    expose:
-      - 4000
     ports:
       - 4000:8080
     networks:
@@ -18,8 +16,6 @@ services:
         - SEARXNG_API_URL=http://searxng:8080
     depends_on:
       - searxng
-    expose:
-      - 3001
     ports:
       - 3001:3001
     networks:
@@ -34,12 +30,10 @@ services:
         - NEXT_PUBLIC_WS_URL=ws://127.0.0.1:3001
     depends_on:
       - perplexica-backend
-    expose:
-      - 3000
     ports:
       - 3000:3000
     networks:
       - perplexica-network
 
 networks:
-  perplexica-network:
+  perplexica-network:

docs/architecture/README.md

@@ -0,0 +1,11 @@
+## Perplexica's Architecture
+
+Perplexica's architecture consists of the following key components:
+
+1. **User Interface**: A web-based interface that allows users to interact with Perplexica for searching images, videos, and much more.
+2. **Agent/Chains**: These components predict Perplexica's next actions, understand user queries, and decide whether a web search is necessary.
+3. **SearXNG**: A metadata search engine used by Perplexica to search the web for sources.
+4. **LLMs (Large Language Models)**: Utilized by agents and chains for tasks like understanding content, writing responses, and citing sources. Examples include Claude, GPTs, etc.
+5. **Embedding Models**: To improve the accuracy of search results, embedding models re-rank the results using similarity search algorithms such as cosine similarity and dot product distance.
+
+For a more detailed explanation of how these components work together, see [WORKING.md](https://github.com/ItzCrazyKns/Perplexica/tree/master/docs/architecture/WORKING.md).

docs/architecture/WORKING.md

@@ -0,0 +1,19 @@
+## How does Perplexica work?
+
+Curious about how Perplexica works? Don't worry, we'll cover it here. Before we begin, make sure you've read about the architecture of Perplexica to ensure you understand what it's made up of. Haven't read it? You can read it [here](https://github.com/ItzCrazyKns/Perplexica/tree/master/docs/architecture/README.md).
+
+We'll understand how Perplexica works by taking an example of a scenario where a user asks: "How does an A.C. work?". We'll break down the process into steps to make it easier to understand. The steps are as follows:
+
+1. The message is sent via WS to the backend server where it invokes the chain. The chain will depend on your focus mode. For this example, let's assume we use the "webSearch" focus mode.
+2. The chain is now invoked; first, the message is passed to another chain where it first predicts (using the chat history and the question) whether there is a need for sources or searching the web. If there is, it will generate a query (in accordance with the chat history) for searching the web that we'll take up later. If not, the chain will end there, and then the answer generator chain, also known as the response generator, will be started.
+3. The query returned by the first chain is passed to SearXNG to search the web for information.
+4. After the information is retrieved, it is based on keyword-based search. We then convert the information into embeddings and the query as well, then we perform a similarity search to find the most relevant sources to answer the query.
+5. After all this is done, the sources are passed to the response generator. This chain takes all the chat history, the query, and the sources. It generates a response that is streamed to the UI.
+
+### How are the answers cited?
+
+The LLMs are prompted to do so. We've prompted them so well that they cite the answers themselves, and using some UI magic, we display it to the user.
+
+### Image and Video Search
+
+Image and video searches are conducted in a similar manner. A query is always generated first, then we search the web for images and videos that match the query. These results are then returned to the user.

package.json

@@ -1,12 +1,12 @@
 {
   "name": "perplexica-backend",
-  "version": "1.0.0",
+  "version": "1.4.0",
   "license": "MIT",
   "author": "ItzCrazyKns",
   "scripts": {
     "start": "node dist/app.js",
     "build": "tsc",
-    "dev": "nodemon src/app.ts",
+    "dev": "nodemon src/app.ts" , 
     "format": "prettier . --check",
     "format:write": "prettier . --write"
   },
@@ -14,6 +14,7 @@
     "@types/cors": "^2.8.17",
     "@types/express": "^4.17.21",
     "@types/readable-stream": "^4.0.11",
+    "nodemon": "^3.1.0",
     "prettier": "^3.2.5",
     "ts-node": "^10.9.2",
     "typescript": "^5.4.3"
@@ -21,6 +22,7 @@
   "dependencies": {
     "@iarna/toml": "^2.2.5",
     "@langchain/openai": "^0.0.25",
+    "@xenova/transformers": "^2.17.1",
     "axios": "^1.6.8",
     "compute-cosine-similarity": "^1.1.0",
     "compute-dot": "^1.1.0",

sample.config.toml

@@ -1,11 +1,10 @@
 [GENERAL]
 PORT = 3001 # Port to run the server on
 SIMILARITY_MEASURE = "cosine" # "cosine" or "dot"
-CHAT_MODEL_PROVIDER = "openai" # "openai" or "ollama"
-CHAT_MODEL = "gpt-3.5-turbo" # Name of the model to use
 
 [API_KEYS]
 OPENAI = "" # OpenAI API key - sk-1234567890abcdef1234567890abcdef
+GROQ = "" # Groq API key - gsk_1234567890abcdef1234567890abcdef
 
 [API_ENDPOINTS]
 SEARXNG = "http://localhost:32768" # SearxNG API URL

src/config.ts

@@ -8,11 +8,10 @@ interface Config {
   GENERAL: {
     PORT: number;
     SIMILARITY_MEASURE: string;
-    CHAT_MODEL_PROVIDER: string;
-    CHAT_MODEL: string;
   };
   API_KEYS: {
     OPENAI: string;
+    GROQ: string;
   };
   API_ENDPOINTS: {
     SEARXNG: string;
@@ -34,13 +33,10 @@ export const getPort = () => loadConfig().GENERAL.PORT;
 export const getSimilarityMeasure = () =>
   loadConfig().GENERAL.SIMILARITY_MEASURE;
 
-export const getChatModelProvider = () =>
-  loadConfig().GENERAL.CHAT_MODEL_PROVIDER;
-
-export const getChatModel = () => loadConfig().GENERAL.CHAT_MODEL;
-
 export const getOpenaiApiKey = () => loadConfig().API_KEYS.OPENAI;
 
+export const getGroqApiKey = () => loadConfig().API_KEYS.GROQ;
+
 export const getSearxngApiEndpoint = () => loadConfig().API_ENDPOINTS.SEARXNG;
 
 export const getOllamaApiEndpoint = () => loadConfig().API_ENDPOINTS.OLLAMA;
@@ -49,21 +45,19 @@ export const updateConfig = (config: RecursivePartial<Config>) => {
   const currentConfig = loadConfig();
 
   for (const key in currentConfig) {
-    /* if (currentConfig[key] && !config[key]) {
-      config[key] = currentConfig[key];
-    } */
+    if (!config[key]) config[key] = {};
 
-    if (currentConfig[key] && typeof currentConfig[key] === 'object') {
+    if (typeof currentConfig[key] === 'object' && currentConfig[key] !== null) {
       for (const nestedKey in currentConfig[key]) {
         if (
-          currentConfig[key][nestedKey] &&
           !config[key][nestedKey] &&
+          currentConfig[key][nestedKey] &&
           config[key][nestedKey] !== ''
         ) {
           config[key][nestedKey] = currentConfig[key][nestedKey];
         }
       }
-    } else if (currentConfig[key] && !config[key] && config[key] !== '') {
+    } else if (currentConfig[key] && config[key] !== '') {
       config[key] = currentConfig[key];
     }
   }

src/lib/huggingfaceTransformer.ts

@@ -0,0 +1,82 @@
+import { Embeddings, type EmbeddingsParams } from '@langchain/core/embeddings';
+import { chunkArray } from '@langchain/core/utils/chunk_array';
+
+export interface HuggingFaceTransformersEmbeddingsParams
+  extends EmbeddingsParams {
+  modelName: string;
+
+  model: string;
+
+  timeout?: number;
+
+  batchSize?: number;
+
+  stripNewLines?: boolean;
+}
+
+export class HuggingFaceTransformersEmbeddings
+  extends Embeddings
+  implements HuggingFaceTransformersEmbeddingsParams
+{
+  modelName = 'Xenova/all-MiniLM-L6-v2';
+
+  model = 'Xenova/all-MiniLM-L6-v2';
+
+  batchSize = 512;
+
+  stripNewLines = true;
+
+  timeout?: number;
+
+  private pipelinePromise: Promise<any>;
+
+  constructor(fields?: Partial<HuggingFaceTransformersEmbeddingsParams>) {
+    super(fields ?? {});
+
+    this.modelName = fields?.model ?? fields?.modelName ?? this.model;
+    this.model = this.modelName;
+    this.stripNewLines = fields?.stripNewLines ?? this.stripNewLines;
+    this.timeout = fields?.timeout;
+  }
+
+  async embedDocuments(texts: string[]): Promise<number[][]> {
+    const batches = chunkArray(
+      this.stripNewLines ? texts.map((t) => t.replace(/\n/g, ' ')) : texts,
+      this.batchSize,
+    );
+
+    const batchRequests = batches.map((batch) => this.runEmbedding(batch));
+    const batchResponses = await Promise.all(batchRequests);
+    const embeddings: number[][] = [];
+
+    for (let i = 0; i < batchResponses.length; i += 1) {
+      const batchResponse = batchResponses[i];
+      for (let j = 0; j < batchResponse.length; j += 1) {
+        embeddings.push(batchResponse[j]);
+      }
+    }
+
+    return embeddings;
+  }
+
+  async embedQuery(text: string): Promise<number[]> {
+    const data = await this.runEmbedding([
+      this.stripNewLines ? text.replace(/\n/g, ' ') : text,
+    ]);
+    return data[0];
+  }
+
+  private async runEmbedding(texts: string[]) {
+    const { pipeline } = await import('@xenova/transformers');
+
+    const pipe = await (this.pipelinePromise ??= pipeline(
+      'feature-extraction',
+      this.model,
+    ));
+
+    return this.caller.call(async () => {
+      const output = await pipe(texts, { pooling: 'mean', normalize: true });
+      return output.tolist();
+    });
+  }
+}

src/lib/providers.ts

@@ -1,11 +1,17 @@
 import { ChatOpenAI, OpenAIEmbeddings } from '@langchain/openai';
 import { ChatOllama } from '@langchain/community/chat_models/ollama';
 import { OllamaEmbeddings } from '@langchain/community/embeddings/ollama';
-import { getOllamaApiEndpoint, getOpenaiApiKey } from '../config';
+import { HuggingFaceTransformersEmbeddings } from './huggingfaceTransformer';
+import {
+  getGroqApiKey,
+  getOllamaApiEndpoint,
+  getOpenaiApiKey,
+} from '../config';
 import logger from '../utils/logger';
 
-export const getAvailableProviders = async () => {
+export const getAvailableChatModelProviders = async () => {
   const openAIApiKey = getOpenaiApiKey();
+  const groqApiKey = getGroqApiKey();
   const ollamaEndpoint = getOllamaApiEndpoint();
 
   const models = {};
@@ -13,34 +19,83 @@ export const getAvailableProviders = async () => {
   if (openAIApiKey) {
     try {
       models['openai'] = {
-        'gpt-3.5-turbo': new ChatOpenAI({
+        'GPT-3.5 turbo': new ChatOpenAI({
           openAIApiKey,
           modelName: 'gpt-3.5-turbo',
           temperature: 0.7,
         }),
-        'gpt-4': new ChatOpenAI({
+        'GPT-4': new ChatOpenAI({
           openAIApiKey,
           modelName: 'gpt-4',
           temperature: 0.7,
         }),
-        'gpt-4-turbo': new ChatOpenAI({
+        'GPT-4 turbo': new ChatOpenAI({
           openAIApiKey,
           modelName: 'gpt-4-turbo',
           temperature: 0.7,
         }),
-        embeddings: new OpenAIEmbeddings({
-          openAIApiKey,
-          modelName: 'text-embedding-3-large',
-        }),
       };
     } catch (err) {
       logger.error(`Error loading OpenAI models: ${err}`);
     }
   }
 
+  if (groqApiKey) {
+    try {
+      models['groq'] = {
+        'LLaMA3 8b': new ChatOpenAI(
+          {
+            openAIApiKey: groqApiKey,
+            modelName: 'llama3-8b-8192',
+            temperature: 0.7,
+          },
+          {
+            baseURL: 'https://api.groq.com/openai/v1',
+          },
+        ),
+        'LLaMA3 70b': new ChatOpenAI(
+          {
+            openAIApiKey: groqApiKey,
+            modelName: 'llama3-70b-8192',
+            temperature: 0.7,
+          },
+          {
+            baseURL: 'https://api.groq.com/openai/v1',
+          },
+        ),
+        'Mixtral 8x7b': new ChatOpenAI(
+          {
+            openAIApiKey: groqApiKey,
+            modelName: 'mixtral-8x7b-32768',
+            temperature: 0.7,
+          },
+          {
+            baseURL: 'https://api.groq.com/openai/v1',
+          },
+        ),
+        'Gemma 7b': new ChatOpenAI(
+          {
+            openAIApiKey: groqApiKey,
+            modelName: 'gemma-7b-it',
+            temperature: 0.7,
+          },
+          {
+            baseURL: 'https://api.groq.com/openai/v1',
+          },
+        ),
+      };
+    } catch (err) {
+      logger.error(`Error loading Groq models: ${err}`);
+    }
+  }
+
   if (ollamaEndpoint) {
     try {
-      const response = await fetch(`${ollamaEndpoint}/api/tags`);
+      const response = await fetch(`${ollamaEndpoint}/api/tags`, {
+        headers: {
+          'Content-Type': 'application/json',
+        },
+      });
 
       const { models: ollamaModels } = (await response.json()) as any;
 
@@ -52,17 +107,77 @@ export const getAvailableProviders = async () => {
         });
         return acc;
       }, {});
-
-      if (Object.keys(models['ollama']).length > 0) {
-        models['ollama']['embeddings'] = new OllamaEmbeddings({
-          baseUrl: ollamaEndpoint,
-          model: models['ollama'][Object.keys(models['ollama'])[0]].model,
-        });
-      }
     } catch (err) {
       logger.error(`Error loading Ollama models: ${err}`);
     }
   }
 
+  models['custom_openai'] = {};
+
+  return models;
+};
+
+export const getAvailableEmbeddingModelProviders = async () => {
+  const openAIApiKey = getOpenaiApiKey();
+  const ollamaEndpoint = getOllamaApiEndpoint();
+
+  const models = {};
+
+  if (openAIApiKey) {
+    try {
+      models['openai'] = {
+        'Text embedding 3 small': new OpenAIEmbeddings({
+          openAIApiKey,
+          modelName: 'text-embedding-3-small',
+        }),
+        'Text embedding 3 large': new OpenAIEmbeddings({
+          openAIApiKey,
+          modelName: 'text-embedding-3-large',
+        }),
+      };
+    } catch (err) {
+      logger.error(`Error loading OpenAI embeddings: ${err}`);
+    }
+  }
+
+  if (ollamaEndpoint) {
+    try {
+      const response = await fetch(`${ollamaEndpoint}/api/tags`, {
+        headers: {
+          'Content-Type': 'application/json',
+        },
+      });
+
+      const { models: ollamaModels } = (await response.json()) as any;
+
+      models['ollama'] = ollamaModels.reduce((acc, model) => {
+        acc[model.model] = new OllamaEmbeddings({
+          baseUrl: ollamaEndpoint,
+          model: model.model,
+        });
+        return acc;
+      }, {});
+
+    } catch (err) {
+      logger.error(`Error loading Ollama embeddings: ${err}`);
+    }
+  }
+
+  try {
+    models['local'] = {
+      'BGE Small': new HuggingFaceTransformersEmbeddings({
+        modelName: 'Xenova/bge-small-en-v1.5',
+      }),
+      'GTE Small': new HuggingFaceTransformersEmbeddings({
+        modelName: 'Xenova/gte-small',
+      }),
+      'Bert Multilingual': new HuggingFaceTransformersEmbeddings({
+        modelName: 'Xenova/bert-base-multilingual-uncased'
+      }),
+    };
+  } catch(err) {
+    logger.error(`Error loading local embeddings: ${err}`); 
+  }
+
   return models;
 };

src/routes/config.ts

@@ -1,8 +1,10 @@
 import express from 'express';
-import { getAvailableProviders } from '../lib/providers';
 import {
-  getChatModel,
-  getChatModelProvider,
+  getAvailableChatModelProviders,
+  getAvailableEmbeddingModelProviders,
+} from '../lib/providers';
+import {
+  getGroqApiKey,
   getOllamaApiEndpoint,
   getOpenaiApiKey,
   updateConfig,
@@ -13,23 +15,29 @@ const router = express.Router();
 router.get('/', async (_, res) => {
   const config = {};
 
-  const providers = await getAvailableProviders();
+  const [chatModelProviders, embeddingModelProviders] = await Promise.all([
+    getAvailableChatModelProviders(),
+    getAvailableEmbeddingModelProviders(),
+  ]);
 
-  for (const provider in providers) {
-    delete providers[provider]['embeddings'];
+  config['chatModelProviders'] = {};
+  config['embeddingModelProviders'] = {};
+
+  for (const provider in chatModelProviders) {
+    config['chatModelProviders'][provider] = Object.keys(
+      chatModelProviders[provider],
+    );
   }
 
-  config['providers'] = {};
-
-  for (const provider in providers) {
-    config['providers'][provider] = Object.keys(providers[provider]);
+  for (const provider in embeddingModelProviders) {
+    config['embeddingModelProviders'][provider] = Object.keys(
+      embeddingModelProviders[provider],
+    );
   }
 
-  config['selectedProvider'] = getChatModelProvider();
-  config['selectedChatModel'] = getChatModel();
-
-  config['openeaiApiKey'] = getOpenaiApiKey();
+  config['openaiApiKey'] = getOpenaiApiKey();
   config['ollamaApiUrl'] = getOllamaApiEndpoint();
+  config['groqApiKey'] = getGroqApiKey();
 
   res.status(200).json(config);
 });
@@ -38,12 +46,9 @@ router.post('/', async (req, res) => {
   const config = req.body;
 
   const updatedConfig = {
-    GENERAL: {
-      CHAT_MODEL_PROVIDER: config.selectedProvider,
-      CHAT_MODEL: config.selectedChatModel,
-    },
     API_KEYS: {
-      OPENAI: config.openeaiApiKey,
+      OPENAI: config.openaiApiKey,
+      GROQ: config.groqApiKey,
     },
     API_ENDPOINTS: {
       OLLAMA: config.ollamaApiUrl,

src/routes/images.ts

@@ -1,8 +1,7 @@
 import express from 'express';
 import handleImageSearch from '../agents/imageSearchAgent';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
-import { getAvailableProviders } from '../lib/providers';
-import { getChatModel, getChatModelProvider } from '../config';
+import { getAvailableChatModelProviders } from '../lib/providers';
 import { HumanMessage, AIMessage } from '@langchain/core/messages';
 import logger from '../utils/logger';
 
@@ -10,7 +9,7 @@ const router = express.Router();
 
 router.post('/', async (req, res) => {
   try {
-    let { query, chat_history } = req.body;
+    let { query, chat_history, chat_model_provider, chat_model } = req.body;
 
     chat_history = chat_history.map((msg: any) => {
       if (msg.role === 'user') {
@@ -20,14 +19,14 @@ router.post('/', async (req, res) => {
       }
     });
 
-    const models = await getAvailableProviders();
-    const provider = getChatModelProvider();
-    const chatModel = getChatModel();
+    const chatModels = await getAvailableChatModelProviders();
+    const provider = chat_model_provider || Object.keys(chatModels)[0];
+    const chatModel = chat_model || Object.keys(chatModels[provider])[0];
 
     let llm: BaseChatModel | undefined;
 
-    if (models[provider] && models[provider][chatModel]) {
-      llm = models[provider][chatModel] as BaseChatModel | undefined;
+    if (chatModels[provider] && chatModels[provider][chatModel]) {
+      llm = chatModels[provider][chatModel] as BaseChatModel | undefined;
     }
 
     if (!llm) {

src/routes/index.ts

@@ -2,11 +2,13 @@ import express from 'express';
 import imagesRouter from './images';
 import videosRouter from './videos';
 import configRouter from './config';
+import modelsRouter from './models';
 
 const router = express.Router();
 
 router.use('/images', imagesRouter);
 router.use('/videos', videosRouter);
 router.use('/config', configRouter);
+router.use('/models', modelsRouter);
 
 export default router;

src/routes/models.ts

@@ -0,0 +1,24 @@
+import express from 'express';
+import logger from '../utils/logger';
+import {
+  getAvailableChatModelProviders,
+  getAvailableEmbeddingModelProviders,
+} from '../lib/providers';
+
+const router = express.Router();
+
+router.get('/', async (req, res) => {
+  try {
+    const [chatModelProviders, embeddingModelProviders] = await Promise.all([
+      getAvailableChatModelProviders(),
+      getAvailableEmbeddingModelProviders(),
+    ]);
+
+    res.status(200).json({ chatModelProviders, embeddingModelProviders });
+  } catch (err) {
+    res.status(500).json({ message: 'An error has occurred.' });
+    logger.error(err.message);
+  }
+});
+
+export default router;

src/routes/videos.ts

@@ -1,7 +1,6 @@
 import express from 'express';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
-import { getAvailableProviders } from '../lib/providers';
-import { getChatModel, getChatModelProvider } from '../config';
+import { getAvailableChatModelProviders } from '../lib/providers';
 import { HumanMessage, AIMessage } from '@langchain/core/messages';
 import logger from '../utils/logger';
 import handleVideoSearch from '../agents/videoSearchAgent';
@@ -10,7 +9,7 @@ const router = express.Router();
 
 router.post('/', async (req, res) => {
   try {
-    let { query, chat_history } = req.body;
+    let { query, chat_history, chat_model_provider, chat_model } = req.body;
 
     chat_history = chat_history.map((msg: any) => {
       if (msg.role === 'user') {
@@ -20,14 +19,14 @@ router.post('/', async (req, res) => {
       }
     });
 
-    const models = await getAvailableProviders();
-    const provider = getChatModelProvider();
-    const chatModel = getChatModel();
+    const chatModels = await getAvailableChatModelProviders();
+    const provider = chat_model_provider || Object.keys(chatModels)[0];
+    const chatModel = chat_model || Object.keys(chatModels[provider])[0];
 
     let llm: BaseChatModel | undefined;
 
-    if (models[provider] && models[provider][chatModel]) {
-      llm = models[provider][chatModel] as BaseChatModel | undefined;
+    if (chatModels[provider] && chatModels[provider][chatModel]) {
+      llm = chatModels[provider][chatModel] as BaseChatModel | undefined;
     }
 
     if (!llm) {

src/websocket/connectionManager.ts

@@ -1,39 +1,100 @@
 import { WebSocket } from 'ws';
 import { handleMessage } from './messageHandler';
-import { getChatModel, getChatModelProvider } from '../config';
-import { getAvailableProviders } from '../lib/providers';
+import {
+  getAvailableEmbeddingModelProviders,
+  getAvailableChatModelProviders,
+} from '../lib/providers';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import type { Embeddings } from '@langchain/core/embeddings';
+import type { IncomingMessage } from 'http';
 import logger from '../utils/logger';
+import { ChatOpenAI } from '@langchain/openai';
 
-export const handleConnection = async (ws: WebSocket) => {
-  const models = await getAvailableProviders();
-  const provider = getChatModelProvider();
-  const chatModel = getChatModel();
+export const handleConnection = async (
+  ws: WebSocket,
+  request: IncomingMessage,
+) => {
+  try {
+    const searchParams = new URL(request.url, `http://${request.headers.host}`)
+      .searchParams;
 
-  let llm: BaseChatModel | undefined;
-  let embeddings: Embeddings | undefined;
+    const [chatModelProviders, embeddingModelProviders] = await Promise.all([
+      getAvailableChatModelProviders(),
+      getAvailableEmbeddingModelProviders(),
+    ]);
 
-  if (models[provider] && models[provider][chatModel]) {
-    llm = models[provider][chatModel] as BaseChatModel | undefined;
-    embeddings = models[provider].embeddings as Embeddings | undefined;
-  }
+    const chatModelProvider =
+      searchParams.get('chatModelProvider') ||
+      Object.keys(chatModelProviders)[0];
+    const chatModel =
+      searchParams.get('chatModel') ||
+      Object.keys(chatModelProviders[chatModelProvider])[0];
 
-  if (!llm || !embeddings) {
+    const embeddingModelProvider =
+      searchParams.get('embeddingModelProvider') ||
+      Object.keys(embeddingModelProviders)[0];
+    const embeddingModel =
+      searchParams.get('embeddingModel') ||
+      Object.keys(embeddingModelProviders[embeddingModelProvider])[0];
+
+    let llm: BaseChatModel | undefined;
+    let embeddings: Embeddings | undefined;
+
+    if (
+      chatModelProviders[chatModelProvider] &&
+      chatModelProviders[chatModelProvider][chatModel] &&
+      chatModelProvider != 'custom_openai'
+    ) {
+      llm = chatModelProviders[chatModelProvider][chatModel] as
+        | BaseChatModel
+        | undefined;
+    } else if (chatModelProvider == 'custom_openai') {
+      llm = new ChatOpenAI({
+        modelName: chatModel,
+        openAIApiKey: searchParams.get('openAIApiKey'),
+        temperature: 0.7,
+        configuration: {
+          baseURL: searchParams.get('openAIBaseURL'),
+        },
+      });
+    }
+
+    if (
+      embeddingModelProviders[embeddingModelProvider] &&
+      embeddingModelProviders[embeddingModelProvider][embeddingModel]
+    ) {
+      embeddings = embeddingModelProviders[embeddingModelProvider][
+        embeddingModel
+      ] as Embeddings | undefined;
+    }
+
+    if (!llm || !embeddings) {
+      ws.send(
+        JSON.stringify({
+          type: 'error',
+          data: 'Invalid LLM or embeddings model selected, please refresh the page and try again.',
+          key: 'INVALID_MODEL_SELECTED',
+        }),
+      );
+      ws.close();
+    }
+
+    ws.on(
+      'message',
+      async (message) =>
+        await handleMessage(message.toString(), ws, llm, embeddings),
+    );
+
+    ws.on('close', () => logger.debug('Connection closed'));
+  } catch (err) {
     ws.send(
       JSON.stringify({
         type: 'error',
-        data: 'Invalid LLM or embeddings model selected',
+        data: 'Internal server error.',
+        key: 'INTERNAL_SERVER_ERROR',
       }),
     );
     ws.close();
+    logger.error(err);
   }
-
-  ws.on(
-    'message',
-    async (message) =>
-      await handleMessage(message.toString(), ws, llm, embeddings),
-  );
-
-  ws.on('close', () => logger.debug('Connection closed'));
 };

src/websocket/messageHandler.ts

@@ -57,7 +57,13 @@ const handleEmitterEvents = (
   });
   emitter.on('error', (data) => {
     const parsedData = JSON.parse(data);
-    ws.send(JSON.stringify({ type: 'error', data: parsedData.data }));
+    ws.send(
+      JSON.stringify({
+        type: 'error',
+        data: parsedData.data,
+        key: 'CHAIN_ERROR',
+      }),
+    );
   });
 };
 
@@ -73,7 +79,11 @@ export const handleMessage = async (
 
     if (!parsedMessage.content)
       return ws.send(
-        JSON.stringify({ type: 'error', data: 'Invalid message format' }),
+        JSON.stringify({
+          type: 'error',
+          data: 'Invalid message format',
+          key: 'INVALID_FORMAT',
+        }),
       );
 
     const history: BaseMessage[] = parsedMessage.history.map((msg) => {
@@ -99,11 +109,23 @@ export const handleMessage = async (
         );
         handleEmitterEvents(emitter, ws, id);
       } else {
-        ws.send(JSON.stringify({ type: 'error', data: 'Invalid focus mode' }));
+        ws.send(
+          JSON.stringify({
+            type: 'error',
+            data: 'Invalid focus mode',
+            key: 'INVALID_FOCUS_MODE',
+          }),
+        );
       }
     }
   } catch (err) {
-    ws.send(JSON.stringify({ type: 'error', data: 'Invalid message format' }));
+    ws.send(
+      JSON.stringify({
+        type: 'error',
+        data: 'Invalid message format',
+        key: 'INVALID_FORMAT',
+      }),
+    );
     logger.error(`Failed to handle message: ${err}`);
   }
 };

src/websocket/websocketServer.ts

@@ -10,9 +10,7 @@ export const initServer = (
   const port = getPort();
   const wss = new WebSocketServer({ server });
 
-  wss.on('connection', (ws) => {
-    handleConnection(ws);
-  });
+  wss.on('connection', handleConnection);
 
   logger.info(`WebSocket server started on port ${port}`);
 };

tsconfig.json

@@ -1,7 +1,8 @@
 {
   "compilerOptions": {
     "lib": ["ESNext"],
-    "module": "commonjs",
+    "module": "Node16",
+    "moduleResolution": "Node16",
     "target": "ESNext",
     "outDir": "dist",
     "sourceMap": false,

ui/app/layout.tsx

@@ -3,6 +3,7 @@ import { Montserrat } from 'next/font/google';
 import './globals.css';
 import { cn } from '@/lib/utils';
 import Sidebar from '@/components/Sidebar';
+import { Toaster } from 'sonner';
 
 const montserrat = Montserrat({
   weight: ['300', '400', '500', '700'],
@@ -26,6 +27,15 @@ export default function RootLayout({
     <html className="h-full" lang="en">
       <body className={cn('h-full', montserrat.className)}>
         <Sidebar>{children}</Sidebar>
+        <Toaster
+          toastOptions={{
+            unstyled: true,
+            classNames: {
+              toast:
+                'bg-[#111111] text-white rounded-lg p-4 flex flex-row items-center space-x-2',
+            },
+          }}
+        />
       </body>
     </html>
   );

ui/components/ChatWindow.tsx

@@ -5,6 +5,7 @@ import { Document } from '@langchain/core/documents';
 import Navbar from './Navbar';
 import Chat from './Chat';
 import EmptyChat from './EmptyChat';
+import { toast } from 'sonner';
 
 export type Message = {
   id: string;
@@ -19,11 +20,102 @@ const useSocket = (url: string) => {
 
   useEffect(() => {
     if (!ws) {
-      const ws = new WebSocket(url);
-      ws.onopen = () => {
-        console.log('[DEBUG] open');
-        setWs(ws);
+      const connectWs = async () => {
+        let chatModel = localStorage.getItem('chatModel');
+        let chatModelProvider = localStorage.getItem('chatModelProvider');
+        let embeddingModel = localStorage.getItem('embeddingModel');
+        let embeddingModelProvider = localStorage.getItem(
+          'embeddingModelProvider',
+        );
+
+        if (
+          !chatModel ||
+          !chatModelProvider ||
+          !embeddingModel ||
+          !embeddingModelProvider
+        ) {
+          const providers = await fetch(
+            `${process.env.NEXT_PUBLIC_API_URL}/models`,
+            {
+              headers: {
+                'Content-Type': 'application/json',
+              },
+            },
+          ).then(async (res) => await res.json());
+
+          const chatModelProviders = providers.chatModelProviders;
+          const embeddingModelProviders = providers.embeddingModelProviders;
+
+          if (
+            !chatModelProviders ||
+            Object.keys(chatModelProviders).length === 0
+          )
+            return toast.error('No chat models available');
+
+          if (
+            !embeddingModelProviders ||
+            Object.keys(embeddingModelProviders).length === 0
+          )
+            return toast.error('No embedding models available');
+
+          chatModelProvider = Object.keys(chatModelProviders)[0];
+          chatModel = Object.keys(chatModelProviders[chatModelProvider])[0];
+
+          embeddingModelProvider = Object.keys(embeddingModelProviders)[0];
+          embeddingModel = Object.keys(
+            embeddingModelProviders[embeddingModelProvider],
+          )[0];
+
+          localStorage.setItem('chatModel', chatModel!);
+          localStorage.setItem('chatModelProvider', chatModelProvider);
+          localStorage.setItem('embeddingModel', embeddingModel!);
+          localStorage.setItem(
+            'embeddingModelProvider',
+            embeddingModelProvider,
+          );
+        }
+
+        const wsURL = new URL(url);
+        const searchParams = new URLSearchParams({});
+
+        searchParams.append('chatModel', chatModel!);
+        searchParams.append('chatModelProvider', chatModelProvider);
+
+        if (chatModelProvider === 'custom_openai') {
+          searchParams.append(
+            'openAIApiKey',
+            localStorage.getItem('openAIApiKey')!,
+          );
+          searchParams.append(
+            'openAIBaseURL',
+            localStorage.getItem('openAIBaseURL')!,
+          );
+        }
+
+        searchParams.append('embeddingModel', embeddingModel!);
+        searchParams.append('embeddingModelProvider', embeddingModelProvider);
+
+        wsURL.search = searchParams.toString();
+
+        const ws = new WebSocket(wsURL.toString());
+
+        ws.onopen = () => {
+          console.log('[DEBUG] open');
+          setWs(ws);
+        };
+
+        ws.onmessage = (e) => {
+          const parsedData = JSON.parse(e.data);
+          if (parsedData.type === 'error') {
+            toast.error(parsedData.data);
+            if (parsedData.key === 'INVALID_MODEL_SELECTED') {
+              localStorage.clear();
+            }
+          }
+        };
       };
+
+      connectWs();
     }
 
     return () => {
@@ -74,6 +166,12 @@ const ChatWindow = () => {
     const messageHandler = (e: MessageEvent) => {
       const data = JSON.parse(e.data);
 
+      if (data.type === 'error') {
+        toast.error(data.data);
+        setLoading(false);
+        return;
+      }
+
       if (data.type === 'sources') {
         sources = data.data;
         if (!added) {

ui/components/MessageBox.tsx

@@ -1,22 +1,17 @@
+'use client';
+
 /* eslint-disable @next/next/no-img-element */
 import React, { MutableRefObject, useEffect, useState } from 'react';
 import { Message } from './ChatWindow';
 import { cn } from '@/lib/utils';
-import {
-  BookCopy,
-  Disc3,
-  FilePen,
-  PlusIcon,
-  Share,
-  ThumbsDown,
-  VideoIcon,
-} from 'lucide-react';
+import { BookCopy, Disc3, Share, Volume2, StopCircle } from 'lucide-react';
 import Markdown from 'markdown-to-jsx';
 import Copy from './MessageActions/Copy';
 import Rewrite from './MessageActions/Rewrite';
 import MessageSources from './MessageSources';
 import SearchImages from './SearchImages';
 import SearchVideos from './SearchVideos';
+import { useSpeech } from 'react-text-to-speech';
 
 const MessageBox = ({
   message,
@@ -36,15 +31,16 @@ const MessageBox = ({
   rewrite: (messageId: string) => void;
 }) => {
   const [parsedMessage, setParsedMessage] = useState(message.content);
+  const [speechMessage, setSpeechMessage] = useState(message.content);
 
   useEffect(() => {
+    const regex = /\[(\d+)\]/g;
+
     if (
       message.role === 'assistant' &&
       message?.sources &&
       message.sources.length > 0
     ) {
-      const regex = /\[(\d+)\]/g;
-
       return setParsedMessage(
         message.content.replace(
           regex,
@@ -53,9 +49,13 @@ const MessageBox = ({
         ),
       );
     }
+
+    setSpeechMessage(message.content.replace(regex, ''));
     setParsedMessage(message.content);
   }, [message.content, message.sources, message.role]);
 
+  const { speechStatus, start, stop } = useSpeech({ text: speechMessage });
+
   return (
     <div>
       {message.role === 'user' && (
@@ -95,7 +95,7 @@ const MessageBox = ({
               <Markdown className="prose max-w-none break-words prose-invert prose-p:leading-relaxed prose-pre:p-0 text-white text-sm md:text-base font-medium">
                 {parsedMessage}
               </Markdown>
-              {!loading && (
+              {loading && isLast ? null : (
                 <div className="flex flex-row items-center justify-between w-full text-white py-4 -mx-2">
                   <div className="flex flex-row items-center space-x-1">
                     <button className="p-2 text-white/70 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white">
@@ -105,11 +105,21 @@ const MessageBox = ({
                   </div>
                   <div className="flex flex-row items-center space-x-1">
                     <Copy initialMessage={message.content} message={message} />
-                    <button className="p-2 text-white/70 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white">
-                      <FilePen size={18} />
-                    </button>
-                    <button className="p-2 text-white/70 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white">
-                      <ThumbsDown size={18} />
+                    <button
+                      onClick={() => {
+                        if (speechStatus === 'started') {
+                          stop();
+                        } else {
+                          start();
+                        }
+                      }}
+                      className="p-2 text-white/70 rounded-xl hover:bg-[#1c1c1c] transition duration-200 hover:text-white"
+                    >
+                      {speechStatus === 'started' ? (
+                        <StopCircle size={18} />
+                      ) : (
+                        <Volume2 size={18} />
+                      )}
                     </button>
                   </div>
                 </div>

ui/components/SearchImages.tsx

@@ -29,6 +29,10 @@ const SearchImages = ({
         <button
           onClick={async () => {
             setLoading(true);
+
+            const chatModelProvider = localStorage.getItem('chatModelProvider');
+            const chatModel = localStorage.getItem('chatModel');
+
             const res = await fetch(
               `${process.env.NEXT_PUBLIC_API_URL}/images`,
               {
@@ -39,6 +43,8 @@ const SearchImages = ({
                 body: JSON.stringify({
                   query: query,
                   chat_history: chat_history,
+                  chat_model_provider: chatModelProvider,
+                  chat_model: chatModel,
                 }),
               },
             );

ui/components/SearchVideos.tsx

@@ -42,6 +42,10 @@ const Searchvideos = ({
         <button
           onClick={async () => {
             setLoading(true);
+
+            const chatModelProvider = localStorage.getItem('chatModelProvider');
+            const chatModel = localStorage.getItem('chatModel');
+
             const res = await fetch(
               `${process.env.NEXT_PUBLIC_API_URL}/videos`,
               {
@@ -52,6 +56,8 @@ const Searchvideos = ({
                 body: JSON.stringify({
                   query: query,
                   chat_history: chat_history,
+                  chat_model_provider: chatModelProvider,
+                  chat_model: chatModel,
                 }),
               },
             );

ui/components/SettingsDialog.tsx

@@ -3,12 +3,14 @@ import { CloudUpload, RefreshCcw, RefreshCw } from 'lucide-react';
 import React, { Fragment, useEffect, useState } from 'react';
 
 interface SettingsType {
-  providers: {
+  chatModelProviders: {
     [key: string]: string[];
   };
-  selectedProvider: string;
-  selectedChatModel: string;
-  openeaiApiKey: string;
+  embeddingModelProviders: {
+    [key: string]: string[];
+  };
+  openaiApiKey: string;
+  groqApiKey: string;
   ollamaApiUrl: string;
 }
 
@@ -20,6 +22,19 @@ const SettingsDialog = ({
   setIsOpen: (isOpen: boolean) => void;
 }) => {
   const [config, setConfig] = useState<SettingsType | null>(null);
+  const [selectedChatModelProvider, setSelectedChatModelProvider] = useState<
+    string | null
+  >(null);
+  const [selectedChatModel, setSelectedChatModel] = useState<string | null>(
+    null,
+  );
+  const [selectedEmbeddingModelProvider, setSelectedEmbeddingModelProvider] =
+    useState<string | null>(null);
+  const [selectedEmbeddingModel, setSelectedEmbeddingModel] = useState<
+    string | null
+  >(null);
+  const [customOpenAIApiKey, setCustomOpenAIApiKey] = useState<string>('');
+  const [customOpenAIBaseURL, setCustomOpenAIBaseURL] = useState<string>('');
   const [isLoading, setIsLoading] = useState(false);
   const [isUpdating, setIsUpdating] = useState(false);
 
@@ -27,9 +42,54 @@ const SettingsDialog = ({
     if (isOpen) {
       const fetchConfig = async () => {
         setIsLoading(true);
-        const res = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/config`);
-        const data = await res.json();
+        const res = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/config`, {
+          headers: {
+            'Content-Type': 'application/json',
+          },
+        });
+
+        const data = (await res.json()) as SettingsType;
         setConfig(data);
+
+        const chatModelProvidersKeys = Object.keys(
+          data.chatModelProviders || {},
+        );
+        const embeddingModelProvidersKeys = Object.keys(
+          data.embeddingModelProviders || {},
+        );
+
+        const defaultChatModelProvider =
+          chatModelProvidersKeys.length > 0 ? chatModelProvidersKeys[0] : '';
+        const defaultEmbeddingModelProvider =
+          embeddingModelProvidersKeys.length > 0
+            ? embeddingModelProvidersKeys[0]
+            : '';
+
+        const chatModelProvider =
+          localStorage.getItem('chatModelProvider') ||
+          defaultChatModelProvider ||
+          '';
+        const chatModel =
+          localStorage.getItem('chatModel') ||
+          (data.chatModelProviders &&
+            data.chatModelProviders[chatModelProvider]?.[0]) ||
+          '';
+        const embeddingModelProvider =
+          localStorage.getItem('embeddingModelProvider') ||
+          defaultEmbeddingModelProvider ||
+          '';
+        const embeddingModel =
+          localStorage.getItem('embeddingModel') ||
+          (data.embeddingModelProviders &&
+            data.embeddingModelProviders[embeddingModelProvider]?.[0]) ||
+          '';
+
+        setSelectedChatModelProvider(chatModelProvider);
+        setSelectedChatModel(chatModel);
+        setSelectedEmbeddingModelProvider(embeddingModelProvider);
+        setSelectedEmbeddingModel(embeddingModel);
+        setCustomOpenAIApiKey(localStorage.getItem('openAIApiKey') || '');
+        setCustomOpenAIBaseURL(localStorage.getItem('openAIBaseUrl') || '');
         setIsLoading(false);
       };
 
@@ -49,6 +109,16 @@ const SettingsDialog = ({
         },
         body: JSON.stringify(config),
       });
+
+      localStorage.setItem('chatModelProvider', selectedChatModelProvider!);
+      localStorage.setItem('chatModel', selectedChatModel!);
+      localStorage.setItem(
+        'embeddingModelProvider',
+        selectedEmbeddingModelProvider!,
+      );
+      localStorage.setItem('embeddingModel', selectedEmbeddingModel!);
+      localStorage.setItem('openAIApiKey', customOpenAIApiKey!);
+      localStorage.setItem('openAIBaseURL', customOpenAIBaseURL!);
     } catch (err) {
       console.log(err);
     } finally {
@@ -94,66 +164,167 @@ const SettingsDialog = ({
                 </Dialog.Title>
                 {config && !isLoading && (
                   <div className="flex flex-col space-y-4 mt-6">
-                    {config.providers && (
+                    {config.chatModelProviders && (
                       <div className="flex flex-col space-y-1">
                         <p className="text-white/70 text-sm">
                           Chat model Provider
                         </p>
                         <select
-                          onChange={(e) =>
-                            setConfig({
-                              ...config,
-                              selectedProvider: e.target.value,
-                              selectedChatModel:
-                                config.providers[e.target.value][0],
-                            })
-                          }
+                          value={selectedChatModelProvider ?? undefined}
+                          onChange={(e) => {
+                            setSelectedChatModelProvider(e.target.value);
+                            setSelectedChatModel(
+                              config.chatModelProviders[e.target.value][0],
+                            );
+                          }}
                           className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
                         >
-                          {Object.keys(config.providers).map((provider) => (
-                            <option
-                              key={provider}
-                              value={provider}
-                              selected={provider === config.selectedProvider}
-                            >
-                              {provider.charAt(0).toUpperCase() +
-                                provider.slice(1)}
-                            </option>
-                          ))}
+                          {Object.keys(config.chatModelProviders).map(
+                            (provider) => (
+                              <option key={provider} value={provider}>
+                                {provider.charAt(0).toUpperCase() +
+                                  provider.slice(1)}
+                              </option>
+                            ),
+                          )}
                         </select>
                       </div>
                     )}
-                    {config.selectedProvider && (
+                    {selectedChatModelProvider &&
+                      selectedChatModelProvider != 'custom_openai' && (
+                        <div className="flex flex-col space-y-1">
+                          <p className="text-white/70 text-sm">Chat Model</p>
+                          <select
+                            value={selectedChatModel ?? undefined}
+                            onChange={(e) =>
+                              setSelectedChatModel(e.target.value)
+                            }
+                            className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                          >
+                            {config.chatModelProviders[
+                              selectedChatModelProvider
+                            ] ? (
+                              config.chatModelProviders[
+                                selectedChatModelProvider
+                              ].length > 0 ? (
+                                config.chatModelProviders[
+                                  selectedChatModelProvider
+                                ].map((model) => (
+                                  <option key={model} value={model}>
+                                    {model}
+                                  </option>
+                                ))
+                              ) : (
+                                <option value="" disabled>
+                                  No models available
+                                </option>
+                              )
+                            ) : (
+                              <option value="" disabled>
+                                Invalid provider, please check backend logs
+                              </option>
+                            )}
+                          </select>
+                        </div>
+                      )}
+                    {selectedChatModelProvider &&
+                      selectedChatModelProvider === 'custom_openai' && (
+                        <>
+                          <div className="flex flex-col space-y-1">
+                            <p className="text-white/70 text-sm">Model name</p>
+                            <input
+                              type="text"
+                              placeholder="Model name"
+                              defaultValue={selectedChatModel!}
+                              onChange={(e) =>
+                                setSelectedChatModel(e.target.value)
+                              }
+                              className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                            />
+                          </div>
+                          <div className="flex flex-col space-y-1">
+                            <p className="text-white/70 text-sm">
+                              Custom OpenAI API Key
+                            </p>
+                            <input
+                              type="text"
+                              placeholder="Custom OpenAI API Key"
+                              defaultValue={customOpenAIApiKey!}
+                              onChange={(e) =>
+                                setCustomOpenAIApiKey(e.target.value)
+                              }
+                              className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                            />
+                          </div>
+                          <div className="flex flex-col space-y-1">
+                            <p className="text-white/70 text-sm">
+                              Custom OpenAI Base URL
+                            </p>
+                            <input
+                              type="text"
+                              placeholder="Custom OpenAI Base URL"
+                              defaultValue={customOpenAIBaseURL!}
+                              onChange={(e) =>
+                                setCustomOpenAIBaseURL(e.target.value)
+                              }
+                              className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                            />
+                          </div>
+                        </>
+                      )}
+                    {/* Embedding models */}
+                    {config.embeddingModelProviders && (
                       <div className="flex flex-col space-y-1">
-                        <p className="text-white/70 text-sm">Chat Model</p>
+                        <p className="text-white/70 text-sm">
+                          Embedding model Provider
+                        </p>
                         <select
+                          value={selectedEmbeddingModelProvider ?? undefined}
+                          onChange={(e) => {
+                            setSelectedEmbeddingModelProvider(e.target.value);
+                            setSelectedEmbeddingModel(
+                              config.embeddingModelProviders[e.target.value][0],
+                            );
+                          }}
+                          className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                        >
+                          {Object.keys(config.embeddingModelProviders).map(
+                            (provider) => (
+                              <option key={provider} value={provider}>
+                                {provider.charAt(0).toUpperCase() +
+                                  provider.slice(1)}
+                              </option>
+                            ),
+                          )}
+                        </select>
+                      </div>
+                    )}
+                    {selectedEmbeddingModelProvider && (
+                      <div className="flex flex-col space-y-1">
+                        <p className="text-white/70 text-sm">Embedding Model</p>
+                        <select
+                          value={selectedEmbeddingModel ?? undefined}
                           onChange={(e) =>
-                            setConfig({
-                              ...config,
-                              selectedChatModel: e.target.value,
-                            })
+                            setSelectedEmbeddingModel(e.target.value)
                           }
                           className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
                         >
-                          {config.providers[config.selectedProvider] ? (
-                            config.providers[config.selectedProvider].length >
-                            0 ? (
-                              config.providers[config.selectedProvider].map(
-                                (model) => (
-                                  <option
-                                    key={model}
-                                    value={model}
-                                    selected={
-                                      model === config.selectedChatModel
-                                    }
-                                  >
-                                    {model}
-                                  </option>
-                                ),
-                              )
+                          {config.embeddingModelProviders[
+                            selectedEmbeddingModelProvider
+                          ] ? (
+                            config.embeddingModelProviders[
+                              selectedEmbeddingModelProvider
+                            ].length > 0 ? (
+                              config.embeddingModelProviders[
+                                selectedEmbeddingModelProvider
+                              ].map((model) => (
+                                <option key={model} value={model}>
+                                  {model}
+                                </option>
+                              ))
                             ) : (
                               <option value="" disabled selected>
-                                No models available
+                                No embedding models available
                               </option>
                             )
                           ) : (
@@ -169,11 +340,11 @@ const SettingsDialog = ({
                       <input
                         type="text"
                         placeholder="OpenAI API Key"
-                        defaultValue={config.openeaiApiKey}
+                        defaultValue={config.openaiApiKey}
                         onChange={(e) =>
                           setConfig({
                             ...config,
-                            openeaiApiKey: e.target.value,
+                            openaiApiKey: e.target.value,
                           })
                         }
                         className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
@@ -194,6 +365,21 @@ const SettingsDialog = ({
                         className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
                       />
                     </div>
+                    <div className="flex flex-col space-y-1">
+                      <p className="text-white/70 text-sm">GROQ API Key</p>
+                      <input
+                        type="text"
+                        placeholder="GROQ API Key"
+                        defaultValue={config.groqApiKey}
+                        onChange={(e) =>
+                          setConfig({
+                            ...config,
+                            groqApiKey: e.target.value,
+                          })
+                        }
+                        className="bg-[#111111] px-3 py-2 flex items-center overflow-hidden border border-[#1C1C1C] text-white rounded-lg text-sm"
+                      />
+                    </div>
                   </div>
                 )}
                 {isLoading && (

ui/package.json

@@ -1,6 +1,6 @@
 {
   "name": "perplexica-frontend",
-  "version": "1.0.0",
+  "version": "1.4.0",
   "license": "MIT",
   "author": "ItzCrazyKns",
   "scripts": {
@@ -22,7 +22,9 @@
     "next": "14.1.4",
     "react": "^18",
     "react-dom": "^18",
+    "react-text-to-speech": "^0.14.5",
     "react-textarea-autosize": "^8.5.3",
+    "sonner": "^1.4.41",
     "tailwind-merge": "^2.2.2",
     "yet-another-react-lightbox": "^3.17.2",
     "zod": "^3.22.4"

ui/yarn.lock

@@ -2632,6 +2632,11 @@ react-is@^16.13.1:
   resolved "https://registry.yarnpkg.com/react-is/-/react-is-16.13.1.tgz#789729a4dc36de2999dc156dd6c1d9c18cea56a4"
   integrity sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ==
 
+react-text-to-speech@^0.14.5:
+  version "0.14.5"
+  resolved "https://registry.yarnpkg.com/react-text-to-speech/-/react-text-to-speech-0.14.5.tgz#f918786ab283311535682011045bd49777193300"
+  integrity sha512-3brr/IrK/5YTtOZSTo+Y8b+dnWelzfZiDZvkXnOct1e7O7fgA/h9bYAVrtwSRo/VxKfdw+wh6glkj6M0mlQuQQ==
+
 react-textarea-autosize@^8.5.3:
   version "8.5.3"
   resolved "https://registry.yarnpkg.com/react-textarea-autosize/-/react-textarea-autosize-8.5.3.tgz#d1e9fe760178413891484847d3378706052dd409"
@@ -2834,6 +2839,11 @@ slash@^3.0.0:
   resolved "https://registry.yarnpkg.com/slash/-/slash-3.0.0.tgz#6539be870c165adbd5240220dbe361f1bc4d4634"
   integrity sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==
 
+sonner@^1.4.41:
+  version "1.4.41"
+  resolved "https://registry.yarnpkg.com/sonner/-/sonner-1.4.41.tgz#ff085ae4f4244713daf294959beaa3e90f842d2c"
+  integrity sha512-uG511ggnnsw6gcn/X+YKkWPo5ep9il9wYi3QJxHsYe7yTZ4+cOd1wuodOUmOpFuXL+/RE3R04LczdNCDygTDgQ==
+
 source-map-js@^1.0.2, source-map-js@^1.2.0:
   version "1.2.0"
   resolved "https://registry.yarnpkg.com/source-map-js/-/source-map-js-1.2.0.tgz#16b809c162517b5b8c3e7dcd315a2a5c2612b2af"
@@ -2845,6 +2855,7 @@ streamsearch@^1.1.0:
   integrity sha512-Mcc5wHehp9aXz1ax6bZUyY5afg9u2rv5cqQI3mRrYkGC8rW2hM02jWuwjtL++LS5qinSyhj2QfLyNsuc+VsExg==
 
 "string-width-cjs@npm:string-width@^4.2.0", string-width@^4.1.0:
+  name string-width-cjs
   version "4.2.3"
   resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
   integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==