All Articles
AI Talks, Zadar Walks and Crystal Waters

AI Talks, Zadar Walks and Crystal Waters

Joana Paiva, Nivaldo Caires · November 17, 2026

Last month, 10 Quality Assurance Engineers (QA) and a Frontend Engineer embarked on a journey to Zadar, Croatia, for the annual Infobip Conference.

The talks were mainly about Artificial Intelligence (AI), and since it still doesn’t allow us to be omnipresent, each of us attended the ones that brought us more curiosity, gathering up bits and bops (or should I say bits and bytes) of various insights, and that is exactly what we want to tell you more about.

That, and the fact that we were lucky to end up in a place that is famous for having one of the most beautiful sunsets in the world (but personally I cannot decide on that, as they are all magical).

Joana Paiva

Joana Paiva

Quality Assurance EngineerMB.ioneer since September 2023
Fun fact:She is completely fascinated by stop motion animation, she could talk about it for hours. Maybe one day she'll make her own!
Nivaldo Caires

Nivaldo Caires

Quality Assurance EngineerMB.ioneer since December 2022
Fun fact:His ideal weekend involves a motorcycle and a tent.

Navigating back to the conference, it was intriguing to see how AI is being adopted by other companies. As a QA, it left me wondering what is the best approach to evaluate these non-deterministic agents. After hearing different perspectives on the topic, I’m starting to think we’re just under the surface and there’s a whole sea to explore when it comes to testing AI.

One thing that stood out for me was the idea that, testing an agent in production “is the only way to test” it. Currently we use evaluation frameworks to test different aspects of an agent (is it calling the right tools? Is the answer relevant to the prompt the user wrote? Is it replying with a toxicity level? …and so on ). The idea behind the talk was that evals are not enough. It’s no surprise that it is impossible to test every scenario, and now with AI products, there are infinite possible scenarios and no exact expected outputs. This non-determinism has shifted the way we think about testing. The speaker highlighted that 4 pillars should be in place to help us identify and analyze issues with agents:

  • User feedback: Collect feedback directly from users, for example, through thumbs-up/down ratings
  • Codebase evaluation: Use evaluation frameworks such as DeepEval to assess the agent’s performance and identify potential issues
  • Production monitoring: Analyze session traces using tools such as Langfuse to detect issues arising in real-world usage
  • Agent self-reporting: Enable the agent to identify and report bugs encountered during execution

Together, these would provide measurable reliability metrics and give input to triage the existing issues with the agent. As more AI products emerge in Mercedes-Benz.io, these practices will become increasingly important to ensure quality and reliability. For my work, it means evolving our QA approach to better identify issues and keep up with the pace of AI development.

Now let’s take a dive into what Nivaldo thought of this experience:

About the conference, it was my first one, and it went beyond what I expected. We saw some really cool AI products, met people who are invested in AI, and took part in some really interesting talks.

But for me, the most valuable part was the seeds that were planted in our minds. I left the conference with a lot of ideas that I’m excited to bring back and implement with my team, particularly around:

  • evals and the new gaps that exist with the introduction of AI in costumer facing features
  • the future of the QA role as a whole
  • how team responsibilities will change to match AI throughput
  • how going back to basic and improve regular processes can boost our day-to-day tasks (release flows with n8n, using AI as a information processor for human decision).

One of the things that surprised me the most was everything we got beyond the conferences and workshops. Having a group of people to discuss these topics with while watching the sunset in Zadar was incredible.

In the end, we all left with new ideas, great memories, and, above all, stronger connections with our colleagues.

I couldn’t agree more, we’ve spent a great time, learnt new things, laughed and enriched our team spirit!

Share this Article
Techsphere

Share your
excellence

Mercedes-Benz.io uses cookies

On this website, only cookies that are technically necessary for its operation are used.

For more information, please refer to ourPrivacy PolicyorCookie Statement.