
AI Talks, Zadar Walks and Crystal Waters
Last month, 10 Quality Assurance Engineers (QA) and a Frontend Engineer embarked on a journey to Zadar, Croatia, for the annual Infobip Conference.
The talks were mainly about Artificial Intelligence (AI), and since it still doesn’t allow us to be omnipresent, each of us attended the ones that brought us more curiosity, gathering up bits and bops (or should I say bits and bytes) of various insights, and that is exactly what we want to tell you more about.
That, and the fact that we were lucky to end up in a place that is famous for having one of the most beautiful sunsets in the world (but personally I cannot decide on that, as they are all magical).

Joana Paiva

Nivaldo Caires
Navigating back to the conference, it was intriguing to see how AI is being adopted by other companies. As a QA, it left me wondering what is the best approach to evaluate these non-deterministic agents. After hearing different perspectives on the topic, I’m starting to think we’re just under the surface and there’s a whole sea to explore when it comes to testing AI.
One thing that stood out for me was the idea that, testing an agent in production “is the only way to test” it. Currently we use evaluation frameworks to test different aspects of an agent (is it calling the right tools? Is the answer relevant to the prompt the user wrote? Is it replying with a toxicity level? …and so on ). The idea behind the talk was that evals are not enough. It’s no surprise that it is impossible to test every scenario, and now with AI products, there are infinite possible scenarios and no exact expected outputs. This non-determinism has shifted the way we think about testing. The speaker highlighted that 4 pillars should be in place to help us identify and analyze issues with agents:
- User feedback: Collect feedback directly from users, for example, through thumbs-up/down ratings
- Codebase evaluation: Use evaluation frameworks such as DeepEval to assess the agent’s performance and identify potential issues
- Production monitoring: Analyze session traces using tools such as Langfuse to detect issues arising in real-world usage
- Agent self-reporting: Enable the agent to identify and report bugs encountered during execution
Together, these would provide measurable reliability metrics and give input to triage the existing issues with the agent. As more AI products emerge in Mercedes-Benz.io, these practices will become increasingly important to ensure quality and reliability. For my work, it means evolving our QA approach to better identify issues and keep up with the pace of AI development.
Now let’s take a dive into what Nivaldo thought of this experience:
About the conference, it was my first one, and it went beyond what I expected. We saw some really cool AI products, met people who are invested in AI, and took part in some really interesting talks.
But for me, the most valuable part was the seeds that were planted in our minds. I left the conference with a lot of ideas that I’m excited to bring back and implement with my team, particularly around:
- evals and the new gaps that exist with the introduction of AI in costumer facing features
- the future of the QA role as a whole
- how team responsibilities will change to match AI throughput
- how going back to basic and improve regular processes can boost our day-to-day tasks (release flows with n8n, using AI as a information processor for human decision).
One of the things that surprised me the most was everything we got beyond the conferences and workshops. Having a group of people to discuss these topics with while watching the sunset in Zadar was incredible.
In the end, we all left with new ideas, great memories, and, above all, stronger connections with our colleagues.
I couldn’t agree more, we’ve spent a great time, learnt new things, laughed and enriched our team spirit!
Related articles

Jens Wallner
How Information Architecture Creates Shared Understanding
In this edition of Tecknowledgy, Jens Wallner, Information Architect at Mercedes-Benz.io, shares why alignment across Product, Design, Engineering and Business starts with something deceptively simple: being clear about
Nov 13, 2026

João Almeida
After the Filter Menu: Four things that actually changed, and one slide I break on purpose
Over the last few months, I have given more or less the same presentation several times. Students, meetups, groups passing through our Lisbon office. Somebody has to explain how AI and data actually show up inside a digi
Nov 6, 2026

Daniel Costa
From hackathon idea to everyday tool: what Toolbox taught me about building for real needs
At Mercedes-Benz.io, I work as an Application Architect, also known as a Solutions Architect, in the DSDBook area. DSDBook is the booking engine behind online workshop appointments at Mercedes-Benz, helping customers sch
Nov 3, 2026