Before I started on the current project I built semantic search over a quarter-million court rulings. Vector search, embeddings, the usual.
Then I called a working user and watched him use it for ten minutes. He didn't. He opened a different site, typed in a statute number, opened the first case, jumped to the citations at the bottom, opened one, jumped again. Twenty minutes, no search box involved. He found exactly what he needed.
I'd built ranked text retrieval. He needed graph navigation. Start at a known node, walk citations, see the court hierarchy at every step, never see a result from a different domain unless he asked for it. The data was fine, the model was fine, the whole framing was wrong.
When you have a hammer (vector search, an LLM, a particular library) the first prototype solves the problem with that tool. That's not the same as solving the user's problem. The diagnostic question I should have asked on the first call wasn't "what would you like." That's hypothetical. It was "what do you click first today." The answer is usually a navigation, not a query. Navigation is a graph problem. Query is a search problem. If you build the wrong category, no amount of polish recovers it.
Most v1 products are answers to the developer's available tool, not the user's flow. The interesting work is in the gap between those.
What I do differently now. First call before the first commit. Watch them work, don't ask them to imagine. Write down what they click, in order, before opening an editor. Pick the data structure to match the click sequence, not the algorithm to match the data structure.
The product I'm shipping now started with eight calls before line one of code. Different category entirely. Same lesson.
— Simon