Most teams running LLMs start with a cloud API. At some point — whether driven by cost, compliance, or latency — the question becomes: should we self-host? And if we do, should we run on a VM or on Kubernetes? This post answers those questions with specifics. It covers when AKS + GPU inference makes sense, how to choose the right model for your use case, and how to size every layer of the stack: GPU node, pod configuration, and replica count. When Does GPU Inference on AKS Make Sense? Option A: Cloud API (Azure OpenAI) No infrastructure. Pay per token. … Continue reading Running LLM Inference on AKS

If you’ve ever had to write a Design Document from scratch — you know the pain. You’re staring at dozens of Terraform files, cross-referencing module parameters, tracing spoke-to-transit attachments, figuring out which firewall image string maps to which vendor and … Continue reading I Got Tired of Writing Design Documents, So I Built a Tool That Does It for Me

Pyr-Edge Pyr-Edge ingests VPC flow logs from AWS, Azure, and GCP, providing real-time analysis, anomaly detection, and natural language querying capabilities through an intuitive web interface. https://github.com/rtrentin73/pyr-edge Continue reading Pyr Edge: Anomaly Detection and AI-assisted operations

What it is kubectl-ai acts as an intelligent interface, translating user intent into precise Kubernetes operations, making Kubernetes management more accessible and efficient. How to install Gemini API Key Go to https://aistudio.google.com/ then Get API Keys: Depending on the tier you will need to import a Google Cloud Project for billing purposes. Testing A simple test to validate the configuration. I asked kubectl-ai to list k8s clusters i have access: Costs https://ai.google.dev/gemini-api/docs/pricing References https://github.com/GoogleCloudPlatform/kubectl-ai?tab=readme-ov-file Continue reading kubectl-ai

Summary k8sfluxops is a GitOps repository that manages a complete Kubernetes infrastructure on GKE using Flux v2. https://github.com/rtrentinavx/k8sfluxops It demonstrates a production-grade setup with: 🎯 Core Purpose Declarative, Git-driven management of Kubernetes infrastructure where all changes are version-controlled and automatically reconciled by Flux. 📦 What It Deploys Category Components Ingress Traefik (routes / → nginx, /boutique/ → Online Boutique) Observability Grafana, Jaeger, OpenTelemetry Collector, Hubble UI, Kube-ops-view Policy/Security OPA Gatekeeper with 4 constraint templates, Policy Manager UI Cost Management Kubecost Backup Velero with GCS backend + UI Cluster Mgmt Rancher, Weave GitOps dashboard Demo Apps Online Boutique (10 microservices with OTel tracing), Nginx Autoscaling HPA for … Continue reading Deploying and Operating a (GKE) K8S using GitOps (Flux)