After the Filter Menu: Four things that actually changed, and one slide I break on purpose

Over the last few months, I have given more or less the same presentation several times. Students, meetups, groups passing through our Lisbon office. Somebody has to explain how AI and data actually show up inside a digital product company, and I still have not found a cure for my inability to say no.

João Almeida

Site Reliability Engineer|MB.ioneer since October 2021

Fun fact:He can name all elements of Thorin's Company from the Hobbit.

The presentation is itself a small generative AI product, because that is what nervous engineers do when handed a stage. Visible run seed, generated scenes that have to validate against a schema before rendering, a cached fallback behind every live model call, and an inspector showing the prompt, the context, the raw JSON, and the validation result. Which is to say I spent two weekends building something that can publicly humiliate me in real time. At one point I press a key that deliberately breaks it.

The talk has one thesis, repeated until annoying: the model is not the product; the product is the system around the model. The useful side effect of giving it more than once is that you learn which parts land and which question keeps coming back. The one that keeps coming back is the narrow one. What has genuinely changed in how customers interact with us, and what does that cost us as engineers?

1. The customer stopped learning our vocabulary

For two decades, the job of a product interface was to teach customers our internal language: facets, trims, packages, and engine codes. A filter menu is, if we are honest, a vocabulary exam the customer did not study for. We called it self-service.

The Mercedes Virtual Assistant inverts that. It launched in September 2025, in Austria and for stock vehicles, and it lets someone describe what they need in ordinary words and get real vehicles back. One market, one slice of the catalogue, which is the least glamorous and most honest way to ship an AI feature.

A second one has since shipped in Germany. On mercedes-benz.de the new CLA opens into an exploration experience the marketing copy promises as “KI-gestützte Beratung”, and there is an AI panel docked beside the configurator. Read what it offers, because the scope is the interesting part: explanations of equipment, packages and options, an overview of your current configuration, and finding options by name or by function. Not “describe your life and we will pick your car”. It explains, and it finds, while the configurator beside it keeps counting how many real stock vehicles still match what you have built. The model handles the language. The inventory query handles the truth. There is also, right there in the panel, a button labelled AI disclaimer. Hold that thought.

The interesting part is not the chat box. The chat box is the easy part. The interesting part is that “something electric, big enough for two kids and a dog, mostly city driving” has to become a structured query against real-time inventory. If the model is inventing availability, you do not have a product, you have a liability with a nice font. So the work did not disappear, it moved. It used to be information architecture. Now it is data, retrieval, and contracts.

2. The output is data, not prose

This is the part I built the whole app to prove. The model never generates interface code. It generates JSON that has to validate against a fixed schema before the renderer sees it, and the renderer only maps validated data onto known components. The model varies the content, never the structure.

So, about that key. It injects one field into the generated payload:

{ "availability": 999999 }

The schema says availability is an integer between 0 and 50. The payload is valid JSON, it looks structured, it looks confident, and it is nonsense. Validation rejects it, the system renders a safe fallback, and the inspector shows why. Less impressive than a flawless live generation, and a much better demo. AI output is input, not truth. Treat it like a request body from a client you do not control.

That matters because the one thing models are reliably excellent at is sounding certain. The generated views show the model’s own confidence next to each inferred need, labelled as not calibrated, and I stand on that deliberately. A model saying 88% does not mean it is right 88% of the time. It gets stranger: a paper in this year’s ACL Findings describes an “ownership bias”, where instruction-tuned models give up to 26% higher confidence to an answer when they generated it themselves than to the identical answer handed to them by a user. The number is not a measurement. It is a mood.

3. The conversation acquired legal weight

In 2024 a British Columbia tribunal decided Moffatt v. Air Canada. A passenger asked the airline’s chatbot about bereavement fares, the bot contradicted the airline’s actual policy, and he relied on it. Air Canada argued, memorably, that the chatbot was a separate legal entity responsible for its own actions. The tribunal called that a remarkable submission, noted that the chatbot is simply part of the airline’s website, and found negligent misrepresentation.

The money was small. The implication is not. Your assistant’s answer is your company speaking, carrying roughly the weight of your terms page, except it is generated at runtime by something probabilistic. Which is why “is this answer grounded in a trusted source” became a product requirement rather than a nice-to-have, and why that little disclaimer button sits in the configurator panel.

4. Some of your customers are not people

This shift is still underpriced inside most companies. Across 2025 and 2026 the shopping interface started being standardised for agents instead of humans. Stripe and OpenAI published the Agentic Commerce Protocol in September 2025. Google and Shopify announced the Universal Commerce Protocol at NRF in January 2026, and it is not a checkout bolt-on its capabilities cover catalog lookup, cart building, identity linking over OAuth 2.0, checkout sessions, and order management through delivery and returns, over REST and JSON-RPC.

That is not a UX redesign. That is an API surface with auth scopes, idempotency, error semantics, and a real question about who stays merchant of record. For years part of our job was making sure humans could not accidentally break the system. Now it includes making sure a very polite robot holding a scoped payment token cannot do it on their behalf.

Which leads to the part that reliably makes a room go quiet. Simon Willison’s “lethal trifecta” is the clearest framing of agent risk I know: danger appears when one agent loop combines private data, untrusted content, and the ability to act externally. Any two are survived. All three, hidden instruction inside something your agent happened to read can walk your data out the front door. A shopping assistant has your order history, reads product pages and vendor feeds, and can send, submit and purchase. It is close to a perfect specimen.

The answer is not “no tools”. It is boring and operational: classify what comes back from a tool, restrict what the agent may do next once the context is tainted, require approval for external actions, and treat prompt injection as a release gate.

The part that is not a win

Trust does not automatically reward transparency: Schilke and Reimann ran thirteen experiments with more than five thousand participants and found that disclosing AI use reduced trust, consistently, across task types and audiences. Softening it to “a human reviewed this” did not help, and being caught was worse. So “AI-powered” is not a feature badge; it is a claim earned by behaviour.

What actually changed

If I compress the whole talk into one idea: the ambiguity moved. It used to live in the customer’s head while they stared at a dropdown, guessing which of our words described their life. Now it lives inside our system, and we own all of it: the translation, the grounding, the validation, the fallback, the permissions, and the explanation when it was wrong.

Which is why the thesis is not a slogan. Data, context, integration, user experience, security, evaluation, operations: that ring is the product, and the model is the piece of it we control least.

Every room asks better questions than I am ready for. The one I still cannot answer well is “how do you know it helped?”. I say “that is the hard part” with a confidence level that is itself not calibrated.

The demo has survived every run so far. The slide I break on purpose keeps breaking correctly. My heart rate recovers somewhere around the way home.

Sources

• Mercedes-Benz.io, Behind the Engine: How the Mercedes Virtual Assistant Is Transforming Online Vehicle Shopping (https://www.mercedes-benz.io/blog/2026-02-24-behind-the-engine-how-the-mercedes-virtual-assistant-is-transforming-online-vehicle-shopping) (February 2026)
• Mercedes-Benz Germany, new car store (https://www.mercedes-benz.de/passengercars/buy/new-car.html) and the CLA exploration experience (https://www.mercedes-benz.de/passengercars/exploration.html) with its AI configuration panel
• Sanz-Guerrero, Mager & von der Wense, Large Language Models Are Overconfident in Their Own Responses (https://arxiv.org/abs/2606.03437), ACL 2026 Findings
• Moffatt v. Air Canada, 2024 BCCRT 149 — summary via the American Bar Association (https://www.americanbar.org/groups/business_law/resources/business-law-today/2024-february/bc-tribunal-confirms-companies-remain-liable-information-provided-ai-chatbot/)
• OpenAI & Stripe, Buy it in ChatGPT: Instant Checkout and the Agentic Commerce Protocol (https://openai.com/index/buy-it-in-chatgpt/) (September 2025)
• Google & Shopify, Universal Commerce Protocol (https://ucp.dev) (announced January 2026)
• Simon Willison, The lethal trifecta for AI agents (https://simonwillison.net/tags/lethal-trifecta/) (June 2025)
• Schilke & Reimann, The transparency dilemma: How AI disclosure erodes trust, Organizational Behavior and Human Decision Processes (2025), summary (https://news.arizona.edu/news/disclosing-ai-use-can-backfire-research-shows)
• Gartner, Half of companies that cut customer service staff due to AI will rehire by 2027 (https://www.gartner.com/en/newsroom/press-releases/2026-02-03-gartner-predicts-half-of-companies-that-cut-customer-service-staff-due-to-ai-will-rehire-by-2027) (February 2026)