Nobody Can Deprecate the Model in My Rack
Arlo Gilbert · November 11, 2026
On my own hardware, I decide which model runs and when it changes.
A few days ago, while researching this post, I asked a frontier model to help me dig into uncensored models and security research. Partway through, a notice appeared. The safeguards had flagged my message as cyber-related and switched the conversation to a different model.
The research continued. The answers were fine. I just wasn't talking to the model I wanted to use and frankly, that kind of made me mad.
Anthropic explains that its safeguards are “intentionally broad” and “can sometimes flag legitimate coding and cybersecurity tasks.” I can understand that decision. Serving millions of people means dealing with some who intend to do real harm. But I'd selected a model for a reason, and the service decided I had a different goal.
I've recently spent about $35,000 making sure I have another option.
There are now four NVIDIA DGX Sparks in my rack, connected through a MikroTik switch. They run DeepSeek V4.1 Flash on TensorFold, with a one-million-token context window, and serve up to 16 agents at once.
I did the purchasing and the install and the cabling. Hermes, the open source agent from Nous Research, configured the switch and now manages the cluster and runs my evaluations. I approve the major changes. My contribution involved considerably more plugging things in than I'd once have expected from a project like this.
Naturally, I wanted to know how it compared with what I was already paying for. I ran the same prompts through my rack and OpenAI's GPT-6.1 Sol in Codex. These were the generation speeds I measured:
| Model and setup | Prose | Code |
|---|---|---|
| My rack, one agent | 78 tokens/sec | 95 tokens/sec |
| OpenAI Sol 6.1 (Default Tier) | 23 tokens/sec | 50 tokens/sec |
| OpenAI Sol 6.1 (Ultrafast) | 90 tokens/sec | 245 tokens/sec |
For prose, my rack came close to Ultrafast. For code, Ultrafast was comfortably ahead. Both results seem useful to know before getting carried away about either local models or frontier services. As an aside, OpenAI promotes 8x faster for it's Ultrafast tier. I've never seen that performance ever, across hundreds of sessions. It is faster, but it's nowhere near 8x.
These are generation speeds on my prompts. They don't tell me which model produces better code, catches more mistakes, or finishes a job with less supervision. They do tell me that running locally doesn't require me to sit around waiting for text to crawl across a screen.
And I can keep this particular model running for as long as the hardware and software support it.
With a hosted service, that decision belongs to the provider. A retirement date can force me to revisit prompts, evaluations and workflows that were working perfectly well. A changing “latest” alias can alter the answers without requiring any change to my code. A policy update can restrict work the service accepted yesterday.
Even an open model leaves me dependent on somebody else if my only access to it is through their endpoint. Keeping the weights locally means the model can disappear from a website without disappearing from my rack. I still have to maintain the software that runs it, but I don't need a vendor to keep offering it.
I can also modify it.
That matters because models arrive with judgments already built into their behavior. They answer some questions directly, refuse others, and treat certain claims as more credible than others. Training shapes those choices, alongside the prompts and safeguards used when the model runs.
Nous Research recently published a useful example. Its Arete project evaluated models on 803 prompts covering topics including freedom of expression, democracy and privacy. It measured factual recall separately from whether an answer adopted the Chinese government's framing. In its reported results, DeepSeek V4.1 Flash adopted that framing 16% of the time, GLM 5.3 did so 15% of the time, and Qwen3.8-27B did so 22% of the time. Nous Research's evaluation and results
Nous then fine-tuned Qwen. The resulting model, Arete-27B, reduced that figure to 0.9%, with no meaningful regression on the capability benchmarks the team ran. Those are the lab's own results on its own evaluation, but the model and evaluation are available for others to test. Arete model card
Nous summarized the argument this way: “The values in any open model are only ever one fine-tune away.” Arete announcement
That's a broader promise than any single experiment can establish. It's also a good reason to experiment.
I'm running Nous's evaluation on DeepSeek V4.1 Flash, GLM 5.3 Flash and Qwen 3.8 Flash Next, and building versions tuned toward Western values. I've only just begun that project so I'm not sure what is possible. I want to know which behaviors I can change and what else changes with them.
“Western values” deserves scrutiny, too. Choosing different training examples and sources doesn't make my choices neutral. Someone else could reasonably disagree with the evaluation or want a model that answers differently. Publishing the method makes that disagreement inspectable. Sharing the weights lets people customize their own.
For me, that's a reason to use open models from Chinese labs and evaluate them carefully. I don't have to assume the behavior they shipped with is permanent. I also don't get to assume that a successful fine-tune resolves every concern about the model.
It certainly doesn't resolve restrictions based on where the model came from. The DHS notice published on October 20 covers DeepSeek products and third-party products and services developed using DeepSeek implementations in federal civilian agency IT. I wouldn't treat a change in political framing as evidence that a model falls outside that scope.
I use the same ability to modify behavior for security work. I keep a version modified to remove refusals and use it to test the open source software my projects depend on. I fix what it finds. My attempts to get hosted models to help with this work have repeatedly run into refusals or flags.
That experience doesn't tell me how often safeguards prevent abuse. It tells me they interfere with work I need to do. A hosted service's restrictions also don't govern the models someone can download and run elsewhere.
I'm comfortable taking responsibility for this use on my own equipment. The model stays behind my access controls. Removing refusals increases the responsibility I have for how it's used; it doesn't make that responsibility disappear.
This is where my hardware project starts to overlap with my work in privacy.
An AI assessment relies partly on observations about how a system behaves. If the underlying model changes, some of those observations may no longer hold. The name on the invoice can stay the same while the system being assessed changes underneath it.
Privacy teams already deal with changes in vendors and subprocessors. Model changes deserve similar attention. We need to know what changed, whether it affects the intended use, and which tests need to run again.
Keeping the weights gives me something specific to preserve and test. It doesn't automatically give me good governance. I still need to record which version ran, how it was configured, what information it received, and what I changed.
Nor does keeping a model guarantee that I can reproduce every answer exactly. Prompts, retrieved information, tools and runtime settings matter too. But preserving the model removes one major dependency from that investigation. I don't have to hope a provider still offers the version I'm trying to examine.
The expense and maintenance are mine as well. The hardware cost about $35,000, before putting a value on my time. I have to handle updates, failures and security patches. If I choose to keep an older model, I also keep its limitations.
For most people, and plenty of companies, buying this equipment would be a poor use of money. A hosted service can be the right choice. I'd still want clear answers to a few questions before building something important on it:
- Can I pin the model version I've tested?
- What can change while that version stays pinned, including safeguards and routing?
- How much notice will I get before retirement, and what are my options then?
- What evidence will I have of which model handled a particular request?
The switch during my research caused no real damage. I finished the work. But it made a dependency visible that I'd prefer to choose deliberately.
I'll keep using hosted models where they're useful. When I need the model I tested to be there tomorrow, I have one in the rack with all the guardrails removed, tuned to behave as I want it.