• Current Version: 0.6

This is a work in progress as I continue to research, experiment and utilize AI impelementations. Everything runs locally, and can be used as individual services, via OpenWeb-UI or N8N.

• Open-WebUI – Web interface for running and managing LLMs like Ollama.
• Ollama – Local LLM runner/manager for models like LLaMA, Mistral, etc.
• Whisper – OpenAI’s speech/video file/audio file-to-text (audio transcription).
• N8N – Workflow automation tool, like open-source Zapier.
• Redis – In-memory key-value store, often used for caching.
• Postgres – Powerful open-source relational database.
• SearXNG – Privacy-respecting metasearch engine.
• Stable Diffusion – Text-to-image AI generator.
• Crawl4AI – Web crawler + requester

Haven't put much up in this public repo yet, but below is what I have currently available.

/GainSec-Local-AI-Stack/n8n-workflows/OpenWebUI-UploadTo-WhisperServer-N8N-Workflow.json - Allows you to integrate your own Whisper Server into OpenWebUI via N8N. Upload the mp3 to Open-WebUI, it causes a local file trigger in N8N which reads the MP3 file, sends to the whisper server, extracts the transcript from the response and writes it to a file.

/GainSec-Local-AI-Stack/n8n-workflows/crawl4ai_outputfiles-final.json - Used along with the Open-WebUI Function, this takes the recieved input, transforms it into a useful format, sends it to crawl4AI API, and outputs a bunch of different files from Crawl4AI's response. It creates a raw output file, a URL list (uses a code node to extract ALL Urls), a Link list (grabs all Links from crawl4ai structured output), markdown file, HTML file and Screenshot of Website.

[/GainSec-Local-AI-Stack/n8n-workflows/V1_Local_RAG_AI_Agent.json] (https://github.com/GainSec/GainSec-Local-AI-Stack/blob/main/n8n-workflows/V1_Local_RAG_AI_Agent.json) - Local RAG AI Agent implementation from Self-hosted AI Package

[/GainSec-Local-AI-Stack/n8n-workflows/V3_Local_Agentic_RAG_AI_Agent.json] (https://github.com/GainSec/GainSec-Local-AI-Stack/blob/main/n8n-workflows/V3_Local_Agentic_RAG_AI_Agent.json) - Local RAG AI Agent implementation from Self-hosted AI Package

/GainSec-Local-AI-Stack/openweb-ui-functions/crawl4ai-openwebui-function.py - Open-WebUI Function that takes user input and sends it to N8N to be used by Crawl4AI

/GainSec-Local-AI-Stack/openweb-ui-functions/n8npipe-local-agentic-rag.py - Open-WebUI Function that pipes into the RAG AI Agentic Implementations from Self-hosted AI Package

/GainSec-Local-AI-Stack/setup_dirs.sh - Script to create all the required directories and give them proper permissions

/GainSec-Local-AI-Stack/example.env - Example .env file that is used alongside the docker docker compose

/GainSec-Local-AI-Stack/docker-compose.yml - The docker compose that is used to pull-up all the services

/GainSec-Local-AI-Stack/start_services.py - Python script to properly spin up all the services + run some other checks

/GainSec-Local-AI-Stack/shared - Where most data lives that is used across services or exported/output

/GainSec-Local-AI-Stack/shared/crawl/ - Where the N8N crawl4ai workflow outputs files to along with some subdirectories: html, image, links, markdown, pdf, urls (the raw output is saved into /shared/crawl/)

/GainSec-Local-AI-Stack/shared/transcripts - Where the N8N Whisper workflow outputs transcriptions

/GainSec-Local-AI-Stack/shared/uploads - Where Open-WebUI stores uploads which are then grabbed by the N8N Whisper workflow

1. Follow Ollama Instructions to use your GPU with docker: Instructions

2. cp example.env .env && nano .env - Modify values as seen fit

3. chmod +x setup_dirs.sh && ./setup_dirs.sh - Create directories required by the services/volumes if they don't exist

4. sudo python3 start_services.py --profile=gpu-nvidia - For other profile options see ColeAm00's original script I modified Other Profiles

5. Navigate to N8N, create a local account.

6. Now hit the + button on the top left, add at the least, these three credentials: ollama account - Base Url: http://ollama:11434/, OpenAPI Account - Base URL: http://ollama:11434/v1 (API Key doesn't matter unless you set it to something), Postgres account - Host: postgres, Database: postgres, User: postgres, Password: whatever you set in your .env

7. Feel free to import the included workflows as well at this point.

8. Navigate to Open-WebUI, create a local account.

9. Click your username on the bottom left then select Admin Panel.

10. Select Settings then select Connections. Deactivate OpenAI API and activate Ollama API. Hit the + button and enter http://ollama:11434 in URL. Toggle the switch on and hit the refresh button next to the toggle to confirm connection. Under Connections is Models, which should show your list of available models. If not, enter the ollama docker and download some. Hit Save.

11. Select Tools and hit the + button. Enter the URL as http://crawl4ai:11235, add a name of description of crawl4ai, turn on the toggle and hit the refresh button. (Make sure you leave the openapi.json default value there). Change visibility to Public. Hit Save.

12. Repeat the same process for Whisper. URL: http://whisper-server:9000 and stable diffusion. URL: http://sd-a1111:8080 Hit Save.

13. Now Select Web Search. Change the Web Search Engine value to searxng and the Searxng Query URL to http://searxng:8080/search?q=<query>&format=json. Turn the toggle on and hit save.

14. Select Audio and change the STT Model to large - Note this is separate then the Whisper-Server we're also running, as Open-WebUI has a built-in whisper server. If you end up not doing anything custom to your whisper service then feel free to just use this one rather then a whole separate service. Hit Save

15. Select Images Turn both toggles to on, and change the Image Generation Engine to Automatic1111. Note that the stable diffusion service atm takes the longest to spin up because it builds it each time. I will fix that in the future and will hopefully find better inference files so newer and better models can be used. Within AUTOMATIC1111 Base URL enter http://sd-a1111:8080/ and hit the refresh to the right. At this time, you should see the default model as v1-5-pruned-*. Hit Save.

16. To the left of Settings select Functions. Hit the + button and paste the n8npipe-local-agentic-rag.py script into it. Name it N8N-Pipe-Local-Agentic-Rag-Pipe Give it any description and hit save. Now hit the gear next to it and enter the N8N Webhook URL from the N8N Workflow, toggle it to enabled and hit save. Do the same for the crawl4ai-openwebui-function.py fuction.

17. Now hit New Chat select your model or Fuction, select your tool (Web Search, Image, or hit the + and select whisper-server or crawl4ai) and enjoy!

18. If you want to hit any of the services directory they can be hit

• GainSec.com
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6
• Part 7
• More Coming Soon

• Jon Gaines - GainSec

• Cole Medin & the N8N-IO Team. Using Cole's docker-compose and start_services.py as a starting point. Local-AI-Packaged