Artificial Intelligence

What Is Your Local AI Server Actually Doing? Observability Basics (Part 7 of 11)

If nobody can see inside a private AI server, it freezes without warning. In our lab we built tracing,...

Sep 28 · 13 min read >

How Do You Keep an AI Server Stable? GitOps for Local AI (Part 6 of 11)

A silent image upgrade, a pruned container image and a backup that had quietly stopped pushed us to run...

Sep 21 · 12 min read >

How Do You Run Several AI Models on One GPU Without Slowdowns? (Part 5 of 11)

Our first smart router made every request slower and pushed other clients' models out of memory. This is how...

Sep 14 · 13 min read >

What Happens When a Prompt Is Too Big for a Local LLM? (Part 4 of 11)

On a local AI server, a prompt too big for the model gets silently truncated, and raising the window...

Sep 7 · 12 min read >

Why Does a Local AI Server Freeze? Sizing Memory for LLMs (Part 3 of 11)

In its first month of testing, our lab AI server froze five times with no kernel error and no...

Aug 31 · 12 min read >

How Do You Control Access to Local AI Models? Use a Gateway (Part 2 of 11)

Every model call on our lab's private AI server goes through one gateway: one key per client, a token...

Aug 24 · 13 min read >

Local AI or Cloud AI? Why We Built a Private AI Server (Part 1 of 11)

We built a private AI server in our lab and ran realistic test workloads on it for seven weeks....

Aug 17 · 12 min read >

From 96 to 340 Gbit/s: Benchmarking a Two-Node DGX Spark Cluster

How we wired two NVIDIA DGX Spark workstations through a MikroTik CRS804-4DDQ switch, discovered four hardware ceilings, and ended...

May 14 · 8 min read >