Skip to main content

GenAI Chatbot

Introduction​

GenAI Chatbot is a next-generation conversational AI assistant designed to provide natural, context-aware interactions.
It utilizes efficient Small Language Models (SLMs) and supports direct import of models fine-tuned within GenAI Studio, enabling easy deployment and immediate use of custom models.

The chatbot features advanced capabilities, including:

  • Audio processing (Speech-to-Text [STT] using faster-whisper-base and Text-to-Speech [TTS])
  • Retrieval-Augmented Generation (RAG)
  • Embedded vector database (VectorDB) using ChromaDB with the embedding model all-MiniLM-L6-v2
  • Model Context Protocol (MCP)

GenAI Chatbot is optimized for embedded platforms, including Qualcomm IQ9075, Ryzen AI 8000, NVIDIA Jetson Orin Nano, NVIDIA Jetson AGX Orin, and NVIDIA Jetson Thor.

VendorDownload ModelsDelete ModelsGenAI StudioRAGTTSEvaluateMCP Tools
NvidiaGuideGuide✅✅✅✅✅
AMDGuideGuide❌✅✅✅✅
Qualcomm❌❌❌✅✅✅✅
  • ✅：Supported
  • ❌：Not supported

Small Language Model​

Download Models from Ollama (Support Device: NVIDIA)​

Ollama-1

  1. Visit https://www.ollama.com/search and click Models.
  2. Use the search bar to find the desired model.
  3. Note the model name.

Ollama-2

  1. You’ll see the model name and version.
  2. Model size depends on hardware resources.
  3. Ensure compatibility with your device.

Ollama-3

  1. In GenAI-Chatbot, create a new chat window.
  2. Enter the model name and click Download.

Ollama-4

  1. A notification will confirm the download.

Ollama-5

  1. Once completed, you’ll see a confirmation message.

Ollama-6

  1. The model now appears in the selection menu.

Delete Models (Support Device: NVIDIA)​

⚠️ Delete Models For AMD

Delete-1

  1. Click User → Admin Panel.
  2. Open Settings → Models.
  3. Click the download icon.

Delete-2

  1. Select the model to delete → Click Delete.

Delete-3

  1. Click Confirm.

Delete-4

  1. A success notification will appear.

Download Models from GenAI Studio​

Genai-studio-1

  1. Open GenAI Studio Hub from left menu.
  2. Enter the URL of your GenAI Studio.
  3. Click Save.
  4. A confirmation message appears.
  5. All available models will display.
  6. Click Download for the model you want.

Genai-studio-3

  1. After download, a success message appears.

Genai-studio-4

  1. You can now select this model in a new chat.

Create a new Knowledge (RAG)​

Rag-1

  1. Go to Workspace → Knowledge.
  2. Click the + icon to add new knowledge.

Sample files:

Rag-2

  1. Enter title and description.
  2. Click Create Knowledge.

Rag-4

  1. Click + → Upload Files to upload.

Rag-5

  1. Select files to upload.

Rag-6

  1. Upload success notification appears.

Create a Chatbot Assistant with RAG​

Rag-7

  1. Go to Models tab.
  2. Click + to add a new model.

Rag-8

  1. Fill in Title, Subtitle, Base Model, and System Prompt.

Rag-9

  1. Select the Knowledge you created.
  2. Click Save & Create.

Rag-10

  1. A success notification appears.
  2. Click model to start chat.

Rag-11

  1. The chat window shows the Knowledge model.

Rag-12

  1. Responses will include retrieved knowledge.

Text-to-Speech with Azure AI Speech API​

1. Create an Azure AI Speech API Account​

Create an Azure Account

Azure-1

Get started with Azure’s free account: new users receive $200 credit for 30 days and free access to popular services.

Azure-2

AI Speech – Text-to-Speech: 500,000 neural characters/month free

2. Set Up the Azure Speech​

Azure-speech-1

  1. Go to Azure Portal → Create a resource.
  2. Click AI + Machine Learning → Speech.
  3. Click Start → Create and fill in details:
    • Subscription
    • Resource Group
    • Region
    • Name
    • Pricing Tier (Free F0)

Azure-speech-5

  1. Click Create and note Key1/Key2 and Endpoint URL for authentication.

3. Setup Azure Text-to-Speech in GenAI Chatbot​

Stt-tts-1

  1. Go to Admin Panel.

Stt-tts-2

  1. Click Audio → Select Text-to-Speech Engine → Azure AI Speech.

Stt-tts-3

  1. Enter API Key, click Save.
  2. A success notification will appear.

Evaluate the SLM Response​

Chatbot-benchmark

Each response includes a performance info button showing token count, speed, and computation time for real-time monitoring.

MCP Tools​

1. A MCP Tool (Sample)​

Mcp-tool-1

  1. Click User → Admin Panel → Tools.
  2. Ensure file-utils and mcp-system-info are listed.
  3. Click Save.

2. New a Chat with Tools​

Mcp-tool-2

  1. Click New Chat → + → Select Tool (e.g., system-info).
  2. Tool icons appear next to + button.

3. To get System Info by MCP Tool​

Example: Ask “What is the current memory usage?”

Mcp-tool-3

4. MCP Server APIs​

MCP System-Info-Server Docs: http://localhost:23952/mcp-system-info/docs

Image-1757556413952

Add a MCP Server​

Link to Guide

Example​

Creating an Audio + RAG Chatbot for Medication Assistant​

EAS-GenAI DevKit

1. Configuring TTS with Azure AI Speech API​

  1. Create an Account on Azure AI Speech API
  2. Setting Up on GenAI Chatbot

2. Create a Knowledge​

3. Create a Chatbot Assistant with RAG (Knowledge)​

4. Start a Voice Chatbot Assistant with RAG​

Rag-stt-tts-4

  1. Select RAG model.
  2. Click mic icon to start voice mode.
  3. The system shows listening status.
  4. Your speech converts to text.
  5. The assistant replies with voice output.