Bookmarks

Hi HN,

Last month at a SundAI hackathon, my team built a prototype for an app called iClaw. The goal was to develop an AI agent using Apple Intelligence. I've since continued hacking away at this idea when I had time, and now I'm releasing it on https://geticlaw.com with code on GitHub.

Apple Intelligence is a really poor model choice for an AI agent. It's not great at extracting information (it often injects preambles like "Here's the page..."), it struggles with following directions, and it gets overwhelmed when you ask it to choose from more than 3 available tools. But it's also a great choice because it comes pre-installed on many Macs without any configuration. It's also incredibly fast compared to just about every other 2-4B model I've run on my MacBook Air.

iClaw is very experimental and will make mistakes, but it's designed around safety. iClaw lives in the App Sandbox where it only has access to the permissions you grant it. All tools calls that create or delete (files, emails, calendar events) require explicit consent. You can also fully disable any tool, so they can't be called.

It was fun pushing the limits of what the 3B Apple Foundation Model (AFM) could do. I trained a LoRA adapter for better instruction following, and to "learn" a DSL for rendering custom widgets. I built, re-built, and re-re-built a 40+ tool library routing system using text classifiers and a multi-step decision framework. I built a Safari Extension that allows iClaw to access what's on your browser (I envision use cases like data extraction, form filling, or agentic navigation).

In all honesty, iClaw isn't ready for prime time. My ultimate goal was to distribute on the Mac App Store, which I perceived as a "safety & trust" signal. iClaw tries to do too much, so it easily gets lost. But I'm also surprised at just how much is possible, even without a purpose-built model. Check it out at https://geticlaw.com and GitHub, and let me know what you think.

Hi HN! I built a CLI tool called ShellTalk for macOS, Linux, and web (WebAssembly) that maps English text to the corresponding Bash commands.

ShellTalk is written in Swift and available under the Apache 2.0 license on GitHub. I was inspired a few weeks ago after reading the Meta-Harness paper and seeing a tool called Hunch that did something similar using the Apple Foundation model. I often forget flag names and orders, but I wanted something that worked consistently. The 3B AFM worked surprisingly well with Hunch, but it felt slow and sometimes slight changes in what I wrote would result in very different outputs.

ShellTalk attempts to match the input with an intent category (Git, File I/O, etc), then a template, and finally to slot-fill and adapt to the specific command version and BSD vs GNU syntax. It has a few other tricks including using NSSpellChecker on macOS to auto-correct certain typos, and scores the output on safety (i.e. is the action destructive or non-reversible).

It's clearly far from perfect, but has very tight testing and validation cycles compared to using an LLM, is very portable, and might eventually work in other languages or environments like Windows. I'm curious to hear what others think.

Apple has quietly shipped a pretty complete on-device AI stack into macOS, with these features first getting API access in MacOS 26. There are multiple components in the foundation model, but the skills it shipped with actually make this ~3b parameter model useful. The API to hit the model is super easy, and no one is really wiring them together yet.

- Foundation Models (macOS 26) - a ~3B-parameter LLM with an API. Streaming, structured output, tool use. No API key, no cloud call, no per-token cost. - NLContextualEmbedding (Natural Language framework, macOS 14+) -- a BERT-style 512-dim text embedder. Exactly what OpenAI and Cohere sell, sitting in Apple's SDKs since iOS 17. - SFSpeechRecognizer / SpeechAnalyzer - on-device speech-to-text including live dictation. Solid accuracy on Apple Silicon.

I built cyberWriter, a Markdown editor, on top of all three, mostly as a test and showcase to see what it can do. I actually integrated local and cloud AI first, and then Apple shipped the foundation model, it stacked on super easy, and now users with no local or API AI knowledge can use it with just a click or two. Well the real reason is because most markdown editors need plugins that run with full system access, and I work on health data and can't have that.

Vault chat / semantic search. The app indexes your Markdown folder via NLContextualEmbedding (around 50 seconds for 1000 chunks on an M1). The search bar gets a "Related Ideas" section that matches by meaning - typing "orbital mechanics" surfaces notes about rockets and launch windows even when those exact words never appear. Ask the AI a question and it retrieves the top 5 chunks as context. Plain RAG, but the embedder, retrieval, chat model, and search all run locally.

AI Workspace. Command+Shift+A opens a chat panel, Command+J triggers inline quick actions (rewrite, summarize, change tone, fix grammar, continue). Apple Intelligence is the default; Claude, OpenAI, Ollama, and LM Studio all work if you prefer. The same context layer - document selection, attached files, retrieved vault chunks - feeds every provider through the same system-message path. Because the vault context is file and filename aware, it can create backlinks to the referenced file if it writes or edits a doc for you.

Voice notes and dictation. Record a voice note directly into your doc, transcribe it with SpeechAnalyzer, or just dictate into the editor while you think. Audio never leaves the Mac.

The privacy story is straightforward because the primitives are already private. Vectors live in a `.vault.embeddings.json` file next to your vault, never sent anywhere. If you use Apple Intelligence, even the retrieved text stays on-device. For cloud models there is a clear toggle and an inline warning before any filenames or snippets leave the machine.

Honest limitations:

- 512-dim embeddings are solid mid-tier. A GPT-4-class embedder catches subtler relationships this will miss. - 256-token chunks can split long paragraphs mid-argument. - Foundation Models caps its context window around 6K characters, so vault context is budgeted to 3K with truncation markers on the rest. - Multilingual support is English-only right now. NLContextualEmbedding has Latin, Cyrillic, and CJK model variants; wiring the language detector across chunks is Phase 2.

The developer experience for these APIs is genuinely good. Foundation Models streams cleanly, NLContextualEmbedding downloads assets on demand and gives you mean-poolable token vectors in a handful of lines. Curious what others here are building on this stack - feels like low-hanging fruit that has been sitting there for a while.

https://imgur.com/a/HyhHLv2

The Apple AI embedding feature is going live today. I'm honestly surprised it even works out of the box.

Hi HN! I built an on-device coding agent called Junco, designed to explore what's possible using the AI (Apple Intelligence) you already have on your Mac.

Junco is a ~9MB Mach-O binary written entirely in Swift using the LanguageModelSession API. It's primarily an exploration and learning exercise for me, but it's also exciting to see what's possible. A clear pattern emerged: deterministic scaffolding is critical to guiding a small model along. Whereas Claude Code can defer task decomposition to Opus 4.6, small models need a lot more hand-holding.

Of all the techniques I explored, the Compile-Verify-Fix (CVF) loop was clearly a winner. Even when I was just prompting the AFM directly, I always knew I would need some self-healing mechanism because the Apple Foundation Model (AFM) was not designed to code. When it wrote code, it was often indented strangely and riddled with minor syntax issues.

I initially set out to build a general-purpose coding agent, but given the AFM was not tuned for coding, I quickly realized that scope would never work so I focused specifically on Swift. It also helps that XCode ships with API headers that allow for on-device API signature discovery, since the model lacks recent world knowledge (the knowledge cutoff is probably around mid-2024).

So I wouldn't recommend Junco anytime soon for production work. Make sure you've committed any changes before playing around with it. But conceptually the idea of a batteries-included, on-device coding agent shows real promise.