I Built My Own Intelligent AI Gateway

Over the last four months, I've been turning a collection of older GPUs and spare hardware into one local AI platform. Instead of buying more powerful GPUs to run bigger models, I wanted to see how much I could do with what I already had. I'm running multiple AI models across an RTX 3090, RTX 3060, RTX 2000 Ada, and even Intel's integrated GPU and NPU. Each has its own job, from coding and knowledge retrieval to vision, voice, and intelligent model routing. In this video, I'll walk through the hardware, software, and models I'm using, along with demos showing how everything works together. Video Notes: https://technotim.com/posts/my-self-h... Just a quick note, this video isn't sponsored. The paid promotion label is there because Sliger sent me the case a few years ago, and Noctua sent me the fans and PSU last year. Neither company even knew I was making this video or had any input into it. I am grateful they sent these components for this build - thank you! I wasn't paid to make it. Some of these are Amazon affiliate links, which means I may earn a small commission at no extra cost to you. HARDWARE USED IN THIS VIDEO SERVER & COMPONENTS • Sliger CX4170a 4U Case (Non-affiliate link) https://sliger.com/products/rackmount... • Intel Core Ultra 7 270K Plus (Amazon affiliate link) https://link.amazon/B0j4AUMvj • ASUS PRIME Z890-P WIFI (Amazon affiliate link) https://link.amazon/B095g5sS6 • Samsung 990 PRO 2TB NVMe (Amazon affiliate link) https://link.amazon/B0eSBbv00 • 128GB DDR5 RAM (Amazon affiliate link) https://link.amazon/B0g2IDB8b GPUs These are older GPUs, so I've linked to NVIDIA's official product pages instead of Amazon listings. • NVIDIA RTX 3090 FE 24GB (Non-affiliate link) https://www.nvidia.com/en-us/geforce/... • NVIDIA RTX 3060 12GB (Non-affiliate link) https://www.nvidia.com/en-us/geforce/... • NVIDIA RTX 2000 Ada 16GB (Non-affiliate link) https://www.nvidia.com/en-us/products... COOLING & POWER • Noctua NH-D12L chromax.black (Non-affiliate link) https://www.amazon.com/dp/B0CVKZ9T3Q • Noctua NF-A12x25 PWM chromax.black.swap (Amazon affiliate link) https://link.amazon/B01ZZpzOY • Seasonic PRIME TX-1600 Noctua Edition (Amazon affiliate link) https://link.amazon/B0h71ZYVZ 00:00 Why I Built a Local AI Platform 01:27 The Hardware 02:48 Building the Server 04:24 Ubuntu and vLLM 05:00 Choosing Models for Each GPU 06:03 Smaller Tasks on RTX 3060 07:31 Complex Coding on RTX 3090 08:48 Intelligent Routing and RAG 10:05 Intel iGPU and NVIDIA Working Together 12:47 Voice AI on the Intel NPU 13:28 Home Assistant Voice Demo 15:11 Prompt Injection Detection 15:34 Bringing Everything Together 16:54 OpenWebUI Demo 18:18 Final Thoughts Merch Shop 🛍️: https://l.technotim.com/shop Support me on Patreon:   / technotim   Sponsor me on GitHub: https://github.com/sponsors/timothyst... Subscribe on Twitch:   / technotim   Become a YouTube member:    / @technotim   Gear Recommendations: https://l.technotim.com/gear Get Help in Our Discord Community: https://l.technotim.com/discord 2nd channel:    / @technotimtinkers   (Affiliate links may be included in this description. I may receive a small commission at no cost to you.) Thank you for watching!