The Icon Saga: From LLM Generation to Lucide Lookup

Introduction

AI Hero has agents for everything: groceries, todos, weather, medications, habits, shopping lists. Each of these agents manages items that users interact with daily. A grocery list might have "organic bananas" and "oat milk." A todo list might have "call dentist" and "review quarterly report." A weather widget shows conditions for "San Francisco" or "Tokyo."

We wanted each item to have its own icon. Not a generic placeholder, but something that actually represented the item. The banana should look like a banana. The phone call should look like a phone. The rain in Tokyo should look different from the sunshine in San Francisco.

The obvious solution? Let an LLM generate SVG icons on the fly. Feed it "almond milk" and get back a beautiful carton icon. Feed it "rainy Tokyo" and get a moody skyline with raindrops. The technology was there. The vision was compelling. What could go wrong?

What you'll learn from this journey:

• How we built an LLM-powered icon generation system
• The 40+ commits we made trying to make it work well
• Why we eventually threw it all away for a simpler solution
• The pragmatic engineering lessons we learned along the way

The LLM Approach

On January 1st, 2026, at 12:31 PM, we committed what seemed like a brilliant feature: dynamic SVG icon generation powered by GPT-4o-mini. The system was elegant in its ambition.

When a user added "almond milk" to their grocery list, the frontend would call our new /v1/agents/grocery/{agent_id}/visualize endpoint with the item name. The backend would canonicalize the name (so "2% milk" and "whole milk" would both become "milk"), then ask the LLM to generate an SVG icon. The result would be cached in Redis for 30 days and returned to the frontend, which would render it with proper sanitization to prevent XSS attacks.

The initial results were genuinely exciting. We had icons! Real, contextual icons that matched our items. A banana looked like a banana. A phone looked like a phone. The caching meant we only hit the API once per unique item, and the sanitization kept us safe from malicious SVG content.

But there were problems. The icons took 500-2000ms to generate on cache miss. The quality varied wildly—sometimes we got beautiful minimalist icons, sometimes we got cluttered messes. And every API call cost money. Still, we were optimistic. Surely we could fix this with better prompts?

The Optimization Rabbit Hole

What followed was an intense six-hour sprint of optimization. Over 40 commits, we tried everything we could think of to make LLM icon generation work well.

Model Upgrades

We started with GPT-4o-mini, then upgraded to GPT-5.2 hoping for better quality. We enabled reasoning mode, then disabled it. We tried different temperature settings. Each change helped a little, but never enough.

Prompt Engineering Adventures

We developed what we called "joyful minimalism"—a set of five design principles for our icons:

• One gesture per icon
• Round over sharp
• Let it breathe (70% empty canvas)
• Subtract until it breaks
• No text ever

We reverse-engineered the Lucide icon style and fed those instructions to the LLM. We provided reference SVGs for common icon types. We tried fixed icon vocabularies where the LLM could only choose from predefined shapes. We experimented with different viewBox sizes—24x24, 256x256, 128x128—each with proportionally scaled stroke widths.

The Image Prompt Era

We added an image_prompt field that generated detailed visual descriptions before the SVG. For weather icons, we tried location-specific skylines—the Golden Gate Bridge for San Francisco, the Empire State Building for New York. For groceries, we described containers and identifying marks. The prompts got increasingly elaborate.

## Weather Icon: San Francisco, Rainy
- Silhouette of Golden Gate Bridge in background
- Gentle rain drops falling at 15-degree angle
- Low clouds obscuring bridge towers
- Minimalist, monochrome style
- No text or labels

Caching and Logging

We implemented LLM prefix caching to speed up responses. We added client-side caching to prevent duplicate API calls. We logged every generated icon to MongoDB so we could analyze patterns and improve over time. We added cache versioning so we could invalidate old icons when we improved the prompts.

Looking back at the commit log, we can see the growing desperation in the commit messages: "improve icon quality," "simplify prompts for cleaner icons," "constrain to fixed icon vocabulary," "enhance icon prompts with richer descriptions." Each one a small step forward, but the fundamental problems remained.

The icons were still slow. They were still inconsistent. And we were still paying for every generation. Worse, debugging was nearly impossible—when an icon looked wrong, we couldn't easily explain why the LLM had made that choice.

The Pivot

On January 2nd at 11:59 AM, we committed what might be our most important change: we deleted almost everything we had built and replaced it with a keyword-based Lucide icon lookup.

Lucide is an open-source icon library with 1,666 beautifully designed, consistent SVG icons. Each icon has semantic keywords—"apple" maps to fruit, food, grocery; "check-square" maps to task, complete, todo. Instead of generating icons, we would simply find the best matching icon from this library.

The transformation was dramatic:

• Speed: From 500-2000ms to ~5ms
• Cost: From API calls per item to zero
• Consistency: From variable to deterministic
• Debugging: From "why did the LLM do that?" to "these keywords matched this icon"

We added two JSON files to our backend: icons.json (46,356 lines containing all 1,666 Lucide icons with their SVG element definitions) and icons-keywords.json (keyword mappings for semantic search). The endpoint now extracts keywords from the item name, scores each icon by how many keywords match, and returns the best match.

Yes, we sometimes get a generic icon when a perfect match doesn't exist. "Organic free-range eggs from local farm" might just get the egg icon, not a special organic-farm-egg icon. But it's instant, it's consistent, and it's good enough.

Making It Better

The pivot wasn't the end of the story. Over the following days, we refined the Lucide-based system to make it even smarter.

Multi-Keyword Scoring

Instead of single-keyword matching, we implemented an inverted index where each icon maps to multiple keywords. When searching for "almond milk," we score icons by how many relevant keywords they match. An icon with keywords ["almond," "milk," "carton," "dairy"] scores higher than one with just ["milk"].

Agent-Kind Disambiguation

The same word can mean different things in different contexts. "Apple" in a grocery list should show fruit. "Apple" in a todo about tech should maybe show a laptop. We added an agent_kind parameter that boosts icons matching the agent's domain. Grocery agents boost food-related icons. Tech-related agents might boost device icons.

Safe Rendering

We changed the API response from raw SVG strings to structured element arrays. Instead of returning <svg><path d="..."/></svg>, we return {elements: [{tag: "path", attrs: {d: "..."}}]}. The frontend renders these using React.createElement, eliminating any need for dangerouslySetInnerHTML. It's cleaner, safer, and more type-safe.

The system today is simple, fast, and reliable. It does exactly what we need: show appropriate icons for items across all our agents.

What We Learned

Those 40+ commits weren't wasted. They taught us exactly what we needed: a deep understanding of what we were actually trying to achieve, and why the LLM approach wasn't the right tool for this job.

Know When to Pivot

Sunk cost fallacy is real. We had invested significant time in the LLM approach, and it was tempting to keep optimizing. But sometimes the right answer isn't "make this better"—it's "try something completely different."

Instant and Consistent Beats Slow and Perfect

Users don't need the perfect icon for "organic almond milk from Trader Joe's." They need an icon that's good enough, instantly. A 5ms lookup that returns a milk carton every time is better than a 2-second generation that might return a beautiful custom icon or might return something weird.

Pre-Built Libraries Exist for a Reason

Lucide's 1,666 icons cover almost every concept we need. The designers who created them spent far more time on icon design than we ever could. Using their work isn't cutting corners—it's standing on the shoulders of giants.

The Journey Matters

We wouldn't have appreciated the Lucide solution if we hadn't first struggled with LLM generation. The optimization rabbit hole taught us exactly why instant, deterministic lookup is valuable. Sometimes you need to try the complicated thing to appreciate the simple thing.

In the end, the best icon system isn't the one with the most sophisticated AI—it's the one that shows users the right icon, instantly, every time.