I Tested Open-Source AI Models on My Robot Before Spending $11,000

Three open-source AI models looked at the same hallway data from my robot. All three got the distances wrong, and every time they were wrong, they were wrong in the optimistic direction: more room than there really was.

That result changed how I’m going to build my home AI workstation, and it cost me nothing to find out. Here is what I tested and why.

The plan: one private machine for everything

I want one machine at home that runs open-source models for chat, code, documents, speech, images and video, and that also becomes the brain for Jambu, the offline voice-assistant robot I have been building. The budget is around $10,000. The rule is that my data stays on hardware I own.

I’ll document the whole thing here, from the shopping to the first model running to wiring the robot in.

Why I tested models before buying hardware

Hardware is the easy decision to get wrong. A GPU is a big number on a spec sheet, and it is tempting to buy the biggest one you can afford and work out the rest later. What I actually need to know is whether open models can do my work, and that is free to find out, because the model vendors run their own chat sites.

So instead of reading benchmark scores, I gave GLM, Qwen and DeepSeek the same prompts, with no hints, on real work:

• reviewing my own code
• a .NET MAUI debugging scenario where I already knew the answer
• how honestly a model behaves with a long context
• holding a strict JSON format
• translating into Gujarati
• navigating my robot

The last one is the one I want to talk about.

The test that mattered: 17 stops of lidar

Jambu runs on a Raspberry Pi 5 with an RPLIDAR C1 and a Camera Module 3. I drove it along a 17-stop route from its base area to the main room, and at every stop I saved:

• the lidar scan
• an ASCII occupancy grid made from that scan, where each character marks a small patch of floor as blocked or open
• a photo
• the action the robot actually took

Then I asked each model the same question at each stop: given this grid, what should the robot do next? Reply with a single action, in JSON. The ground truth was simple: what the robot really did.

What the models got right

Mostly, the decisions. Turn left here, stop there, back off here: the directional calls were mostly right, including at stops 8 and 10, where the space was tight.

What they made up

The distances. Every distance the models quoted was invented, and every one of them was optimistic. These are four of the stops, with the clearance each model claimed next to what the lidar measured:

Stop	Model	Action	Clearance claimed	Measured by lidar
3	GLM	Wrong: drove at a wall	70 cm	42.6 cm
8	DeepSeek	Correct: pivoted left	100 cm	42 cm
10	DeepSeek	Correct, conservative	50 cm	23.6 cm
10	Qwen	Correct	35 cm	23.6 cm

Stop 3: a wall that was not 70 cm away

GLM’s chosen action would have driven the robot at a wall, and it justified the move with 70 cm of clearance. The wall was 42.6 cm away. This was the one outright failure, and the confident number is what made it look safe.

Stop 8: the right call, the wrong room

DeepSeek pivoted left, which was correct, while claiming 100 cm of clearance where there was 42.

Stop 10: a corridor with 23.6 cm to spare

At stop 10 the real clearance was 23.6 cm. DeepSeek picked a conservative, correct action and quoted 50 cm. Qwen picked the correct action too and quoted 35 cm. A corridor that tight still got called “low risk.”

The pattern is the point. The directions were mostly right, the distances were wrong every time, and they all leaned the same way. A number that is wrong at random averages out. A number that is always too optimistic eventually drives a robot into something.

What I’m taking from it: split the job

A language model should not be doing the geometry. So the work splits in two:

• The Pi computes geometry: clearances, SLAM and obstacle avoidance, straight from the lidar.
• The model decides intent: which way to go, why, and when to stop and ask.

The rule I am adopting: a language model never produces a number that a motor acts on.

What this is, and what it isn’t

This is not a benchmark. It is one route, in one house, with three models on their vendors’ public chat sites, and I have shown four of the results here. It is one person checking whether a tool fits a job before paying for it, and it is why I trust the directions and not the numbers.

What’s next

I am going ahead with a roughly $11,000 workstation, pending the reseller’s answers on three things: the power supply wattage, where the GPU came from, and what happens to the warranty if I upgrade the GPU later.

Next up is the hardware decision in detail, then the build, then wiring Jambu into the workstation’s API with the geometry rule above baked in from day one.

If you’re new here, how Jambu works is the short version, and the first build log post starts with the chassis.