📋 stuff

by anon · 2026-06-12 12:44:48
← all clips

You 🧑

Check out the pengy app in ~/Personal/Pengy. Once you get an idea of how it works we'll be adding a feature

▶ Tool: directory_tree [path='~/Personal/Pengy', max_depth=4, show_hidden=True]

▶ Tool: read_file [path='~/Personal/skills/skill_index.md']

Assistant 🤖

I'll explore the Pengy app to understand its structure and functionality. Let me start with the directory layout and key files.

▶ Tool: read_multiple_files [paths=['~/Personal/Pengy/README.md', '~/Personal/Pengy/spec]…

▶ Tool: read_multiple_files [paths=['~/Personal/Pengy/pengy/core/chat_manager.py', '~/Pe]…

▶ Tool: read_multiple_files [paths=['~/Personal/Pengy/pengy/ui/main_window.py', '~/Perso]…

▶ Tool: read_multiple_files [paths=['~/Personal/Pengy/pengy/ui/chat_worker.py', '~/Perso]…

Assistant 🤖

I've thoroughly explored the Pengy app. Here's my understanding of how it works:

Architecture Overview

Pengy is a local-first AI agent with two frontends sharing the same core:

Layer

Files

Responsibility

Core

config.py, chat_manager.py, llm_client.py, tools.py

Pure Python, shared by GUI + CLI. Config, chat CRUD, OpenAI-compatible API client, 11 tool definitions

GUI

main_window.py, chat_view.py, chat_input.py, chat_worker.py, chat_history.py, settings_dialog.py

PySide6/Qt6 three-pane desktop app (sidebar, markdown chat, input with attachments)

CLI

cli/main.py

rich-based interactive REPL + single-shot mode

Key Design Patterns

  1. Generator protocol for tool flow — LLMClient.chat() is a Python generator that yields tool_request / tool_result / final_response dicts. The caller .send()s confirmation dicts back in to resume after tool calls. This lets both the GUI (QThread + threading.Event) and CLI (synchronous) drive the same core.
  2. Non-streaming API — Uses chat.completions.create without stream=True. Full responses render at once, which simplifies things since tool round-trips dominate latency anyway.
  3. Shared storage — ~/.config/pengy/settings.json and ~/.config/pengy/chats.json are used by both GUI and CLI.
  4. System message templating — {date}, {username}, {hostname}, {osinfo} are filled at send time, not save time.
  5. Tool confirmation modes — all (YOLO, skip all), safe (auto-approve read-only tools like read_file, web_search, directory_tree, etc.), none (confirm everything).
  6. Context compaction — elide_old_tool_results() replaces tool output in older turns with [tool output from earlier turn elided] to save context window space.

The 11 Built-in Tools

UI Details

Data Flow

User message → append to chat history → prepend rendered system message → API call

→ No tool calls → render final response → save chat

→ Tool calls → confirm/execute loop → feed results back → API call again → final response → save chat

I also reviewed the tests (test_core.py covers chat manager edge cases, test_smoke.py does import validation).

What feature would you like to add? I can see the spec, the code structure, and I know where to hook in (core tools, CLI slash commands, GUI settings/widgets, etc.).

You 🧑

The app right now is pretty feature complete and works great, however I noticed one edge case I'd like to fix: Right now you can paste in text to the input window and it works or you can attach text or images using the attach feature, but sometime I've got an image in my clipboard and I'd like to paste it in for image model analysis. So without changing anything give me an estimate of how complex it is to add image paste into the input box. The image doesn't need to show up just show something (I think it shows a broken image right now) and have that something passed to the LLM like it's an attachment image

Assistant 🤖

This is a low-complexity, well-contained change — I'd estimate it at roughly 20–30 lines of code in a single file (pengy/ui/chat_input.py), with no changes needed to the core, CLI, or message-sending pipeline.

Why it's straightforward

The heavy lifting is already done:

  1. main_window.py::send_message() already accepts an images list of Path objects, base64-encodes them, and sends them as image_url content parts to the LLM.
  2. ChatInputWidget._on_submit() already iterates self._attachments, checks _is_image_file(path), and routes images into the images list.
  3. The chip UI (_add_chip) already handles image attachments with a 🖼 icon.

All you need is to bridge the clipboard → _attachments list.

The exact hook

The right place is _InputEdit (the inner QTextEdit). Override insertFromMimeData():

def insertFromMimeData(self, source: QMimeData):

if source.hasImage():

image = source.imageData() # QImage (or QPixmap in some cases)

# save to a temp PNG

tmp = tempfile.mkstemp(suffix=".png", prefix="pengy_clip_")

if image.save(tmp_path):

self.image_pasted.emit(Path(tmp_path))

return # don't insert into the text document

super().insertFromMimeData(source)

Then add a image_pasted = Signal(Path) on _InputEdit, connect it in ChatInputWidget to a handler that appends the path to self._attachments and calls self._add_chip(path).

What doesn't need to change

Minor considerations

Topic

Assessment

Temp file cleanup

Low concern — saving to /tmp with tempfile.mkstemp is fine; OS reclaims them. Could optionally delete on send if you wanted to be tidy.

QImage API quirks

source.imageData() sometimes returns a QPixmap variant depending on the source app; a one-liner img = img.toImage() if hasattr(img, 'toImage') else QImage(img) handles it.

Paste vs. attach UX

The pasted image will show up as a chip (🖼 pengy_clip_abc123.png) just like a picked file, which is consistent and clear.

Fallback

If the image save fails, falling back to super().insertFromMimeData() preserves the existing (broken) behavior, so there's no regression risk.

Verdict

Small task — probably 15 minutes to implement and 5 minutes to test. It's a single-file UI sugar feature that reuses the existing attachment infrastructure perfectly.

You 🧑

Awesome, implement it

▶ Tool: replace_in_file [path='~/Personal/Pengy/pengy/ui/chat_input.py', old_str='""]…

▶ Tool: replace_in_file [path='~/Personal/Pengy/pengy/ui/chat_input.py', old_str='cl]…

▶ Tool: replace_in_file [path='~/Personal/Pengy/pengy/ui/chat_input.py', old_str=' ]…

▶ Tool: replace_in_file [path='~/Personal/Pengy/pengy/ui/chat_input.py', old_str=' ]…

▶ Tool: run_python [code='import py_compile, sys\ntry:\n py_compile.compile(]…

Assistant 🤖

Let me verify the file is syntactically clean and review the final state:

Assistant 🤖

Done! The change is live in one file (pengy/ui/chat_input.py) and compiles cleanly.

What changed

pengy/ui/chat_input.py

  1. New imports — os, tempfile, QMimeData, QImage, QPixmap
  2. _InputEdit now overrides insertFromMimeData()
  3. Detects source.hasImage() from the clipboard
  4. Normalizes the image data (handles QPixmap, QImage, or QVariant wrappers)
  5. Saves it to a temp PNG (/tmp/pengy_clip_xxxx.png)
  6. Emits image_pasted(Path) instead of inserting anything into the text widget
  7. Falls back to normal paste behavior if the image save fails
  8. ChatInputWidget connects the new signal to _on_image_pasted()
  9. Appends the temp path to the existing _attachments list
  10. Adds a chip (🖼 pengy_clip_xxxx.png) above the input box, just like a manually attached file

How it works end-to-end

  1. You copy an image to your clipboard (screenshot, browser image, etc.)
  2. Ctrl+V in the Pengy input box
  3. The image is saved to a temporary PNG and appears as a chip
  4. When you hit Enter, the existing send_message() logic in main_window.py sees the image path in _attachments, base64-encodes it, and sends it to the LLM as an image_url content part — exactly the same path as a picked file attachment

No changes needed to main_window.py, core/, cli/, or the message-sending pipeline. The temp files live in /tmp and are cleaned up by the OS on reboot.