Optimizing Local LLMs on Apple Silicon / EP07
Building a Personal AI on an 8GB iMac — From QwenPaw to Hermes
Connecting a small Qwen model to OpenCode Go and Codex on an 8GB iMac. Controlled A/B/C tests, tool-call and remote-session failures, and the move toward Hermes as the main personal agent.
In this post
- Why start this experiment on an 8GB iMac?
- Designing Local → Cloud escalation
- A/B/C: how far did the tasks go?
- A QwenPaw command was different from a model calling a tool
- Running it from an iPhone
- What I wanted was more than a router
- Why turn toward Hermes?
- The personal AI system I want on the Mac Studio
- What remains to verify?
Why start this experiment on an 8GB iMac?
Before ordering the Mac Studio, I wanted to try something on the 8GB iMac: let a small local model start a task, then hand it to the cloud if it cannot finish. Did that require a large model from the outset?
The machine was an M1 iMac with 8GB memory and a 256GB SSD. Working on several apps already stretched both. This was the environment from EP00; keeping a huge model running was not practical. I decided to connect a smaller model to the coding tools I already used.
On September 23, 2026, I started with Qwen2.5 1.5B Instruct Q4_K_S through Ollama. I did not expect 1.5B to handle everything. I wanted to see whether starting small and handing off inadequate results would work as an actual execution path.
| Component at the time | Configuration |
|---|---|
| Local inference | Ollama 0.20.0 · Qwen2.5 1.5B Instruct Q4_K_S |
| Cloud work | OpenCode 1.4.3 using Go models · Codex CLI 0.156.0 |
| Conversation/command entry point | QwenPaw 2.2.1 |
| Remote iPhone access | Paseo CLI 0.9.1 |
The pipeline used Python’s standard library. No Docker or database: sequential provider calls and one JSONL record per task. I asked Ollama to unload the model after each request. Completing connection tests did not establish stable, long-running operation on 8GB.
Designing Local → Cloud escalation
The flow was straightforward. Ask the local model, check the result, and return it if it passes. Otherwise, move to the next permitted tier. Check the OpenCode Go result too, and reach Codex CLI when necessary.
The router decides where work goes; the judge checks whether the result meets its conditions. Initially the judge was not another language model. Code checked bullet counts and required words, exact JSON sequence, file existence and file contents, then mapped that evidence to PASS or ESCALATE. Its configuration name, mock, did not mean the provider responses were fabricated.
Cloud calls required allow_cloud permission. A failed local attempt was not permission to send private material outside the machine. Tests used non-sensitive text and generated test directories.
A/B/C: how far did the tasks go?
I set up three tests to finish at different tiers. B and C deliberately induced failure. This tested whether a failed check led to a real next call, rather than whether the system could estimate arbitrary task difficulty.
A · B · C: where did each task finish?
Case A · A short text summary
A short summary was sent to the local model.
- 1Ollama · Qwen 1.5BDeterministic JudgePASSCheck passed → return result
The local result passed deterministic checks and was returned.
Case B · Escalation after a JSON check failed
The local output-token limit was reduced to induce a failed JSON check.
- 1Ollama · Qwen 1.5BDeterministic JudgeESCALATECheck failed → next permitted tier
- 2OpenCode GoDeterministic JudgePASSCheck passed → return result
A real OpenCode Go call produced a result that passed the check.
Case C · File creation reached Codex
The task required an exact file; OpenCode was configured as a read-only plan agent.
- 1Ollama · Qwen 1.5BDeterministic JudgeESCALATECheck failed → next permitted tier
- 2OpenCode GoDeterministic JudgeESCALATECheck failed → next permitted tier
- 3Codex CLIDeterministic JudgePASSCheck passed → return result
A direct Codex CLI call created the file and passed the check. This was not ACP delegation.
Total elapsed time for three different tasks
Case A · A short text summary3.82 s
Case B · Escalation after a JSON check failed6.15 s
Case C · File creation reached Codex18.72 s
A summarized a short text. The local answer met the bullet-count and required-word checks, so the task ended locally. Those checks established the requested format, not complete semantic accuracy.
B restricted local output to eight tokens, making a complete JSON response difficult. The local check failed; a real OpenCode Go call then passed the exact-sequence check.
C required a file with exact contents. The local model did not create it. OpenCode, configured as a read-only plan agent, did not meet that requirement either. A direct Codex CLI call finally created the file and passed the content check. This was not evidence that the OpenCode model could not code: write permissions differed in this test.
The paths connected. The next problem was less tidy. Code could catch a malformed output. What would catch an answer whose format was correct but whose content was wrong?
A QwenPaw command was different from a model calling a tool
The /poc_route command plugin worked in QwenPaw Console. It invoked the same CLI pipeline, so I did not need a second implementation of provider selection.
Having the chat model read a Skill and choose a tool was another matter. In two attempts, the 1.5B model produced text instead of the actual call. It still did not connect at an 8K context setting, so I disabled that Skill. Saying it would execute something did not make the execution happen.
The native ACP server initialized and created a new session but did not load the installed /poc_route command. The request went to the weak chat model without invoking the pipeline. A working Console command did not establish the same command path through ACP.
I would not call this a structural flaw in QwenPaw. The small model’s tool-calling ability, server/plugin wiring and my PoC’s scope all mattered. I needed to identify which layer failed.
Running it from an iPhone
I added Paseo as the remote entry point. Instead of using QwenPaw’s native ACP server directly, a small ACP adapter forwarded the tested command to its localhost Console API. Paseo carried requests and results; routing stayed in the pipeline.
Reopening an existing session from the iPhone initially failed. After adding ACP session/load and testing resumption, an iPhone-originated Case A reached local Ollama, passed the judge and returned. At that stage, cloud paths B/C had not been tested from the phone.
The subsequent v0.2 records extended that scope. Task IDs reported from iPhone natural-language JSON and coding requests matched Paseo ingress traces. Local retries and OpenCode were followed by Codex, which passed exact checks or tests. A mobile task finishing at OpenCode alone still needed separate confirmation.
ACP also progressed independently. QwenPaw’s ACP client service delegated a synthetic code change and follow-up tests in the same session through a Codex adapter. This was different from the earlier native ACP server’s command-loading failure. The main pipeline still ended with a direct CLI call, and none of this established Hermes integration.
What I wanted was more than a router
The paths worked, but using them still felt like operating a test harness. Package the command, supply checks, then look up the task ID. That was not how I wanted to ask for a Cookly bug fix.
I wanted to say, “Analyze and fix the error in Cookly,” and have the system locate the project, remember earlier decisions, split investigation from changes and tests, hand off only failed work, and report the evidence at the end. That is the single-conversation experience I have in mind when thinking about personal AI such as ChatGPT Dots.
Local-to-cloud routing was one part of it. Something had to hold on to the objective and progress: why an attempt failed, which files had already changed, and whose project permissions applied. Switching model names did not supply that continuity.
The follow-up semantic-judge tests exposed another gap. The 1.5B judge matched only 13 of 30 synthetic cases, with 13 false PASS decisions. I configured semantic-only passes to return UNVERIFIED_DRAFT rather than VERIFIED. This small evaluation does not generalize to every model, but my configuration was not suitable for automatic approval.
I also ran JevMLX separately in isolation. Its 1.5B configuration matched only 10 of the initial 24 synthetic cases; the main pipeline retained Ollama for semantic checks. Changing the tool did not establish a trustworthy judge. On the Mac Studio I will need to choose models and checks again and evaluate real tasks.
Why turn toward Hermes?
For the Mac Studio, I now want Hermes as the main agent. Already using it through Telegram matters to that choice. Rather than pick one more model first, I want to choose the conversation entry point and connect the tested paths behind it.
Hermes documents foundations such as curated memory, past-session search, tools, Skills and child-agent delegation. That does not complete my project management, quality-based escalation or memory-aware model switching. Those integrations still need testing.
QwenPaw remains part of this experiment’s history, but is excluded from the new target architecture. I do not expect removing it to solve everything. The decision is to choose the agent that maintains the conversation and brings work back together.
The personal AI system I want on the Mac Studio
EP06 introduced the target design. The 8GB experiments here were the starting point for that design. I am reusing its architecture diagram.
Behind one conversation
Telegram · Paseo · Mac
Intent / planning · Tasks / sessions · Long-term memory
Tool / agent delegation · Progress · Result aggregation
Privacy · Cloud permission · Action permissions
Task type / difficulty · Choose execution target
General / coding LLM Small decision model
Permitted cloud work Check model and plan
Complex coding work CLI tested · ACP pending
Vision · ComfyUI · STT / TTS · Documents · Files · Code
Deterministic checks → Semantic assessment → Quality policy
Retry / escalation returns to Policy / Router
Result · Verification status · Execution path · Open issues
Requests would reach Hermes from the Mac, Telegram and, if integrated, Paseo/iPhone. Hermes would locate the project and session and divide the work. Policy/router would constrain execution targets using privacy, difficulty, cost, memory and latency. Verified local results return; inadequate ones move to a permitted cloud worker and are checked again.
Weave Router is a model-selection candidate. JevMLX is a candidate for lightweight decisions and semantic checks over constrained choices with MLX models, not the name of an independent judge model. Neither has been integrated with Hermes here. A semantic PASS must not override failed compilation or tests. Codex ACP is a planned delegation path; the separate successful experiment is not successful Hermes integration.
Local First is the default, but difficult coding could start with a specialist agent. LOCAL_ONLY blocks escalation even after local failure. That policy must cover Hermes’ main model, the judge and external tools, not only workers.
The ordered machine is an M5 Max Mac Studio with 128GB unified memory and a 1TB SSD. I would rather keep a frequently used medium LLM and small decision model available and load heavier models on demand.
Resident models and on-demand models
- Medium general LLM
- Small judge / router
- STT or embeddings when needed
- Large Qwen / DeepSeek candidates
- Vision · Image · Video · Music
macOS · Xcode · Simulator · KV Cache · Runtime buffers
- Finish and unload less-used models if necessary
- Load needed model → execute → verify
- Return to baseline after the task
STT or embeddings may be resident when needed. Large Qwen/DeepSeek, vision, image, video and music models are on-demand candidates. As EP02 discussed, weights share memory with KV caches, execution buffers and development tools. Loading everything simultaneously is not the objective.
I would also separate user preferences, project decisions, current-session progress and a knowledge base such as Obsidian and technical documents. Those are my logical categories, not four built-in Hermes stores. Moving between Cookly, KnitLens, Delsa, Sobok and Sound Travel should retrieve relevant context. Blog drafting is one use case.
What remains to verify?
Next I want to connect remote requests, file changes, checks and returned results through Hermes. Each task needs action permissions and cloud-transmission approval, with bounded retries or a stop on failure. Logs should retain routes, decisions, latency and usage without indiscriminately storing credentials or source text.
The initial B/C records contained a default privacy-policy label inconsistent with actual cloud use. The follow-up report documented that conflict and the code fix separately. Rather than rewrite historical traces, I want to check that recorded approval and actual provider calls agree. A policy name in a log does not establish safe execution.
OpenCode Go and Codex subscription usage also need separation from metered API charges. Initial B/C OpenCode calls reported roughly 38,000 input tokens despite short requests. A small handoff can still lead to a large coding-agent context. Per-task dollar cost was null; I did not turn it into a savings claim.
Peak memory, swap, model load/unload time, concurrent sessions, routing delay and end-to-end task latency will be measured after the Mac Studio arrives. Token speed from EP01, acceleration from EP03 and streaming from EP04 ultimately need to work inside this task flow. The hardware decision connects to EP05.
The tiers could connect on 8GB. What became harder was deciding when to trust success and who would maintain context until the task finished. Now I want several tools completing work behind one conversation, more than one larger model.
Records and sources
Checked October 14, 2026: archived September 23 v0.1/v0.2 reports, the ACP report, pipeline/provider/remote-adapter source and original JSONL. A/B/C elapsed times derive from their recorded start/end timestamps. Phone origin relies on the contemporary user report matched to Paseo Task IDs. No PoC was rerun while writing, and no Hermes integration or Mac Studio performance was measured. Raw logs, credentials and pairing material remain private.
- Hermes: memory and session search, child-agent delegation.
- Candidates and connections: JevMLX, Weave Router, Paseo providers, Codex ACP adapter, OpenCode Go.
Comments & likes
No account needed. Your nickname and comment will be public. Comments appear after review.