What is RAG The Problem RAG Solves Large Language Models (LLM) learn from a massive but frozen snapshot of the world. Once training ends, the model’s knowledge is sealed. It cannot read your internal documentation, does not know what changed last quarter, and has never seen your proprietary data. The result: when you ask an LLM about anything outside its training data, it fabricates a plausible-sounding answer. This is called hallucination — and it is not a bug that will be fixed. It is a fundamental property of how language models work. Three strategies exist to close this gap: Strategy How it works … Continue reading RAG on Azure: Self-Hosted vs Managed Stack

Inference servers introduce a threat model that differs from standard web APIs. The differences matter: This guide covers the controls needed at each layer: edge, API management, in-cluster networking, identity, observability, and supply chain. This blog post uses  https://rtrentinsworld.com/2026/03/27/running-llm-inference-on-aks/ as reference. Edge Protection: WAF and DDoS The first line of defense for any publicly reachable inference endpoint is a Web Application Firewall running in Prevention mode, not Detection mode. Detection mode logs attacks but passes them through. Every prompt injection payload, malformed JSON body, and RCE attempt in HTTP headers reaches your APIM and potentially your GPU pods. Switching to Prevention blocks them … Continue reading Securing Applications That Rely on Inference Servers

When you call Azure OpenAI or the OpenAI API, most of the operational surface disappears: Microsoft or OpenAI manages the GPU, the model weights, the inference runtime, and the content filters. Your ops surface is prompts, evals, and cost control. Self-hosted LLMOps is what remains when you take all of that back. You own the GPU lifecycle, the model serving process, the scaling logic, the guardrails, the observability pipeline, and the feedback loop that improves quality over time. The tradeoffs that make self-hosting worth it — data sovereignty, cost at volume, no vendor lock-in, full control over serving parameters — … Continue reading Self-Hosted LLMOps

Most teams running LLMs start with a cloud API. At some point — whether driven by cost, compliance, or latency — the question becomes: should we self-host? And if we do, should we run on a VM or on Kubernetes? This post answers those questions with specifics. It covers when AKS + GPU inference makes sense, how to choose the right model for your use case, and how to size every layer of the stack: GPU node, pod configuration, and replica count. When Does GPU Inference on AKS Make Sense? Option A: Cloud API (Azure OpenAI) No infrastructure. Pay per token. … Continue reading Running LLM Inference on AKS

If you’ve ever had to write a Design Document from scratch — you know the pain. You’re staring at dozens of Terraform files, cross-referencing module parameters, tracing spoke-to-transit attachments, figuring out which firewall image string maps to which vendor and … Continue reading I Got Tired of Writing Design Documents, So I Built a Tool That Does It for Me

Pyr-Edge Pyr-Edge ingests VPC flow logs from AWS, Azure, and GCP, providing real-time analysis, anomaly detection, and natural language querying capabilities through an intuitive web interface. https://github.com/rtrentin73/pyr-edge Continue reading Pyr Edge: Anomaly Detection and AI-assisted operations

What it is kubectl-ai acts as an intelligent interface, translating user intent into precise Kubernetes operations, making Kubernetes management more accessible and efficient. How to install Gemini API Key Go to https://aistudio.google.com/ then Get API Keys: Depending on the tier you will need to import a Google Cloud Project for billing purposes. Testing A simple test to validate the configuration. I asked kubectl-ai to list k8s clusters i have access: Costs https://ai.google.dev/gemini-api/docs/pricing References https://github.com/GoogleCloudPlatform/kubectl-ai?tab=readme-ov-file Continue reading kubectl-ai