Optimising ditto's Shopify UX
When controlled tests met the real world: redesigning ditto's Shopify integration after it collapsed under actual user data, and what that taught me about testing in authentic environments.
Impact
- Identified a critical gap between controlled testing and real-world Shopify environments
- Rebuilt the product import flow to handle large, messy, inconsistently named inventories
- Every store's structure was different, and the same UX had to hold for all of them
At ditto, the AI marketing assistant I co-founded and designed, the Shopify integration was how the product learned what an online store actually sold. Connect your store, and ditto could write about your real products. I led the design of that integration, and it looked straightforward. Then real user stores met our tidy assumptions and it fell apart in ways our controlled tests never hinted at. This is the story of rebuilding it, and of why I no longer trust clean test data.
The Challenge
Our initial tests used carefully organised dummy Shopify stores, and everything worked. Real stores were another matter:
- Large product inventories caused system overloads and crashes.
- Poorly categorised and inconsistently named products confused our AI, obstructing effective automation.
- Displaying extensive product data during onboarding overwhelmed users, who abandoned the flow.
None of this had shown up internally, and the underlying reason was scope: no two real stores were structured alike. Inventories varied wildly in size, naming, and organisation, and the same integration UX had to work for all of them. Our test data had been none of those things.
User Insights & Research
Usability testing with real stores
Sessions with small business owners showed what the sanitised tests had hidden. People didn't want to learn the integration or wade through their own product data; they wanted proof the AI could handle it. One tangible outcome reassured them more than any amount of explanation, the same pattern that ran through all of ditto's trust work.
Looking sideways
I also looked at how content marketing and editing tools handle heavy lifting: complexity kept behind the scenes, interfaces that reveal themselves progressively. That pattern shaped the redesign.
UX Process & Iterative Design
Starting over
Patching wasn't going to work, so I redesigned the integration end to end. As a two-person company, that decision wasn't free; it meant arguing for a rebuild with the developer who'd have to do it, my co-founder, and making the case that the patch route would cost more in the long run.
Processing in the background
Inventory processing moved entirely behind the scenes. The user never saw the complexity, only the result, which turned our biggest technical constraint into the calmest part of the experience.
Onboarding while the AI works
Onboarding was restructured so users could start their first campaign while the AI was still crunching their store data. Tutorials appeared only where they helped.
Information on demand
Product data surfaced only when the task at hand needed it, basics first, depth on request. The interface stayed manageable regardless of how large or chaotic the store behind it was.
Testing against the mess
Every iteration after that was tested against genuine store data, never dummy inventories again. I watched what people did rather than what they said.
Results & Impact
A note on scale first: ditto's user base at this stage was small enough that I could watch every store connection individually. That cuts both ways. I can't quote percentage improvements from a dashboard, but I saw every failure and every fix first-hand.
Store owners who had been abandoning the integration went on to start campaigns within minutes of connecting. The feedback changed too: people stopped asking whether the AI understood their products and started talking about the content it made, which is the shift that mattered. And because the new architecture no longer cared how large or messy a store was, the fix held as inventories grew.
Key Learnings
The most important lesson from this project was the danger of controlled testing. Our dummy stores were too clean, too well-organised, nothing like the real businesses we were building for. Moving to live user data earlier would have saved weeks of rework.
The broader principle applies everywhere in product design: the gap between how you think users work and how they actually work is almost always larger than you expect. Building for the messy real world, not the idealised version, is the job.
