🧑 You

Do some research on the best possible local model that can be used to run Pengy (you) on a 48 gig macbook pro M4. Check the community sentiment around it.

▶ Tool: read_file [path='~/skills/skill_index.md']

▶ Tool: read_file [path='~/skills/reddit/reddit_skill.md']

▶ Tool: read_file [path='~/skills/news/news_skill.md']

▶ Tool: read_file [path='~/skills/rss/rss_skill.md']

▶ Tool: web_search [max_results={}, query='best local LLM for agentic workflows]…

▶ Tool: web_search [max_results={}, query='best local model for AI agents tool ]…

▶ Tool: web_search [max_results={}, query='Qwen3 Llama 3.3 DeepSeek local agent]…

▶ Tool: fetch_url [url='https://willitrunai.com/macs/m4-pro-48gb']

▶ Tool: fetch_url [url='https://openclawdc.com/blog/best-local-llm-m4-max/']

▶ Tool: run_bash [command='cd ~/skills/rss && python fetch_rss.py "https://ww]…

▶ Tool: web_search [max_results={}, query='reddit r/LocalLLaMA best model for a]…

▶ Tool: run_bash [command='cd ~/skills/rss && "${UV_BIN:-$(command -v uv || e]…

▶ Tool: web_search [max_results={}, query='reddit localllama "Qwen 3.6 27B" age]…

▶ Tool: web_search [max_results={}, query='reddit localllama "gpt-oss 20B" vs "]…

▶ Tool: fetch_url [url='https://old.reddit.com/r/LocalLLaMA/comments/1rh12xz/q]…

▶ Tool: fetch_url [url='https://deepinsightai.io/qwen-3-6/']

▶ Tool: fetch_url [url='https://codersera.com/blog/qwen-3-6-as-local-claude-co]…

▶ Reasoning

I now have a wealth of information. Let me synthesize this into a comprehensive report. Let me also fetch one more Reddi…

🤖 Assistant

I've gathered a thorough picture from hardware-fit calculators, hands-on guides, and the r/LocalLLaMA community. Here's the research.

---

🐧 Best Local Model to Run Pengy on a 48 GB MacBook Pro M4

TL;DR Recommendation

Qwen3.6-27B (dense) at Q8_0 is the best single model to run an agentic AI like Pengy on your 48 GB M4. It's the strongest open-weight agentic coding / tool-calling model that fits comfortably in your memory, with a 197K context window and Apache-2.0 licensing. For maximum speed you can sub in Qwen3-Coder 30B-A3B (MoE) which runs at ~32 tok/s on your chip. Run either on llama.cpp (not Ollama) with Q8 weights + an F16 KV cache.

---

The Shortlist (all fit 48 GB unified memory)

Model	Type	Mem @ chosen quant	Speed (M4 Pro 48GB)	Context	Best for
Qwen3.6-27B	Dense 27B	~30 GB (Q8_0)	~10–17 tok/s	197K	Agentic coding, tool calling, daily driver
Qwen3-Coder 30B-A3B	MoE, 3B active	~26 GB	32 tok/s	108K	Fast interactive agent loops
Qwen3.6-35B-A3B	MoE, 3B active	~20 GB (Q4)	fast	262K	Planning / "idea expander"
gpt-oss-20B	Dense 20B	~22 GB (Q8)	fast	—	Cleanest tool-call JSON
Qwen 3.5-27B	Dense 27B	~26 GB	~23 tok/s	61K	General chat / RAG

The 70B-class models (Llama 3.3 70B, Qwen 2.5 72B) won't fit at a usable quant on 48 GB — they need ~54 GB just for weights, and macOS itself eats 6–10 GB. So your practical ceiling is the ~30B dense / MoE tier. (willitrunai.com, openclawdc.com)

---

Why Qwen3.6-27B is the pick for an agent

Pengy is an agent — it does multi-step tool calling, skill routing, and long reasoning loops. That changes the model selection criteria vs. "best chatbot":

1. Agentic coding is its specialty. Qwen3.6-27B scores 77.2 on SWE-bench Verified (vs Claude 4.5 Opus's 80.9), ties Opus on Terminal-Bench 2.0 (59.3), and wins on SkillsBench (48.2 vs 45.3) and Claw-Eval. It's described as having "beat the 397B Qwen 3.5 MoE on agentic coding." For a sub-30B model that's a different tier entirely (Gemma4-31B scores 52.0 on the same bench). (codersera.com)

1. Hybrid architecture = long context without memory blowup. It interleaves Gated DeltaNet + FFN blocks with Gated Attention + FFN blocks, so a 27B dense model carries 262K (practically ~197K) context cheaply. Long agent sessions with a big tool/skill library in context is exactly its design target.

1. Thinking Preservation. It's trained to retain reasoning from earlier turns across iterative steps — the behaviour you want when an agent edits state over many tool calls.

1. Tool-call reliability is strong with the right runtime. Community testing across 5 agent frameworks (Hermes, PydanticAI, LangChain, smolagents, OpenClaude) reported 100% tool-call success at 4-bit — but required runtime guards and 80–150 token tool-usage injection. (deepinsightai.io)

1. License. Apache 2.0 — fully commercial, no per-token cost.

---

⚠️ The 48 GB Reality Check

This is the single most important practical caveat from the community:

"A 48 GB Mac 'just barely fits Q8 and context' — and the same commenter warns it's 'not enough to run a dev environment alongside.'" — codersera.com, summarizing r/LocalLLaMA

Concretely on your machine:

• Q8 weights ≈ 30 GB, macOS + apps ≈ 6–10 GB, leaving ~8–12 GB for KV cache + context.
• The 17 GB default 4-bit pull drifts on long agentic runs — it is not the version people praise. Q8 is the floor for serious agent work.
• Cap context at ~64K (not 128K+) or the KV cache eats 20+ GB and you OOM.
• MLX coding checkpoints are tuned for Apple Silicon: 27b-coding-nvfp4 (20 GB) or 27b-coding-mxfp8 (31 GB).

(openclawdc.com, codersera.com)

---

🗣️ Community Sentiment (r/LocalLLaMA & r/LocalLLM)

The headline vibe

Qwen 3.6 is being talked about as a threshold-crossing moment for local AI:

"This is the first time the benefit outweighs the effort." — r/LocalLLaMA user, via deepinsightai.io

"Qwen 3.6 doesn't win because it's the smartest model. It wins because for the first time, a local model reduces more work than it creates." — deepinsightai.io summary of Reddit threads

27B vs 35B-A3B — pick by job role, not size

A detailed Reddit comparison (writing a master architecture plan) rated:

• Qwen3.6-27B → 9.3 ("best practical default" / "daily driver")
• Qwen3.6-35B-A3B → 9.2 ("more expansive" / "idea expander")
• Gemma 4 → 8.9, Qwen 3.5-27B → 8.8

"35B is like a resource mine. 27B is what I'd actually use daily."

The multi-agent inflection point (important caveat for Pengy)

A 96%-upvoted thread ("Qwen3.5-35B nailed my simple multiagent workflow") found that sub-100B models generally fall apart on orchestrator-style multi-agent loops — gpt-oss-20b, qwen3-coder-next, GLM-4.7-flash, and Devstral-Small-2 all failed to reliably complete a simple 10-transcript summarization workflow, while 100B+ models (gpt-oss-120b, qwen3.5, deepseek-v3.2) succeeded. Qwen3.5-35B was the first sub-100B model to nail it — but it "thinks with a lot of tokens, so it takes a while."

"Anything under 100b is ok for one shot but falls apart in longer agent workflows." — top comment

"100B+ seems to be the inflection point where models can actually maintain state across a multi-hop tool sequence without losing the thread."

This matters for Pengy: a single model doing everything (orchestration + tool calls) is the hard case. The community's fix is tiered routing — a capable model as orchestrator, lighter models as workers — or the hybrid cloud-architect / local-coder pattern.

The "Ollama should be banned" insight

The most-upvoted diagnosis (139 upvotes) in the agentic-coding threads was blunt:

"The real problem is Ollama. Ollama should be banned."

Hyperbole, but the pattern is real and repeated: people who hit context/tool/perf problems on Ollama saw them vanish after switching to llama.cpp with up-to-date unsloth quants + Q8_0 + F16/BF16 KV cache, with one 68-upvote reply confirming "the switch cleared up most of my problems and noticeably improved performance."

The "babysitting" tax

"It's a BEAST for its size and we save so much money... just do a bit more handholding."

"If you're an actual software developer who understands how your application works, the 27B is an extremely handy little beast. If you're a vibecoder, it's going to be much less useful."

One field report measured ~12% JSON tool-call format-error rate local vs ~0.5% for Claude, and long-context drift past ~14K tokens (though that setup may not have used Q8). The consensus: gate every tool call, keep a human (or a frontier cloud model) in the loop for the hard parts.

Sentiment summary

Aspect	Community verdict
Coding ability	✅ Excellent for its size, near-frontier on well-specified tasks
Tool calling	⚠️ Strong with runtime guards + Q8 + good harness; not fire-and-forget
Multi-agent orchestration	⚠️ Sub-100B is the hard case; 35B-A3B is the first to break through
Speed on Apple Silicon	✅ 27B ~10–17 tok/s; 30B-A3B MoE ~32 tok/s — comfortable interactive
48 GB fit	⚠️ Q8 "just barely fits" — cap context, can't run heavy IDE alongside
Runtime	🔴 Avoid Ollama for serious agents → use llama.cpp/vLLM + unsloth Q8
Overall	✅ "First time local AI feels actually usable" — supervised, not autonomous

---

🎯 My Concrete Recommendation for Running Pengy

Primary: Qwen3.6-27B at Q8_0 on llama.cpp

bash

# Don't use the 17GB default — pull Q8

ollama pull qwen3.6:27b-q8_0 # ~30 GB, the version people actually praise

# OR better, llama.cpp with unsloth quants + F16 KV cache:

./llama-server -m Qwen3.6-27B-Q8_0.gguf -c 65536 --cache-type-k f16 --cache-type-v f16

• Cap context at 64K (KV cache stays manageable on 48 GB).
• This is your reasoning + tool-calling brain for Pengy.

Speed alternative: Qwen3-Coder 30B-A3B (MoE)

bash

ollama run qwen3-coder # 32 tok/s, only 3B active params

• Use when you want snappy interactive loops; slightly weaker on deep multi-step reasoning than the dense 27B.

Production hardening (from community consensus):

1. Runtime: llama.cpp or vLLM, not Ollama, with Q8_0 + F16 KV cache.
2. Add tool-usage injection (~80–150 tokens per step) and runtime guards that catch malformed tool-call JSON.
3. For complex multi-agent orchestration, consider tiered routing — keep the 27B as orchestrator and use a lighter model for narrow worker tasks, or keep a cheap cloud tier ($20 Claude) as the "architect" and let local Qwen do the high-volume execution.

The honest framing from the community: Qwen3.6-27B makes a local Pengy viable and cost-saving, but it's a supervised agent, not a drop-in autonomous replacement for a frontier cloud model. On your 48 GB M4 specifically, it's the best card you can play — just mind the memory headroom and the runtime.