I wanted this conversation because most of what I hear about agents is still at the demo stage. One agent, one task, one impressive screenshot. Jannik, a Microsoft MVP from Germany who started out in Intune and device management, is somewhere else entirely. In his company, every department (sales, marketing, IT) has its own organisation of agents, with sub-agents underneath. And each one is hired like a person.
That idea stuck with me long after we stopped recording. You can listen to the full episode on the AI for HumanIT site. What follows is what I took from it.
Hire it like a human
Every agent in Jannik's setup has a CV, a description and a profile picture. It is onboarded like any other employee. It gets access to specific tools and has specific skills. Then people across the business hand it work: producing campaigns, running parts of the sales process, supporting customer projects.
The framing matters more than the novelty. If an agent is a colleague, you naturally ask the right questions. What should it have access to? Who is responsible for it? What does it do on day one versus month six? As Jannik put it, when you onboard someone new you don't give them access to everything straight away. Agents should be treated the same way: step by step, use case by use case.
He was just as clear that this only works because they rebuilt their processes around AI rather than bolting it onto the old ones. In his words, companies take the processes they've had "since hundreds of years" and try to bring AI into them, and that will fail.
Rebuild the process, bring your people
When I pushed on what actually has to change, his answer was surprisingly people-centred:
- Identity. Agents need their own identities, with conditional access policies, just like users and applications.
- People. A CEO announcing that everyone must shift their work to AI overnight will fail. You educate people, upskill them, and build the new way of working with them.
- Question everything. For each daily process, ask whether it's needed at all, or whether an AI should do it. Sometimes that means retiring a legacy tool with no interface and finding something ready for AI.
- Treat it as a journey. Start with targeted proofs of concept, but set the end goal of rethinking how work gets done.
His line that AI is not buying a Copilot licence is one I'll be repeating. Having spent years as a project manager, I recognised all of this as change management. The technology is new; the discipline isn't.
Choosing models and knowing when you're done
Jannik picks models on three things: price, latency and quality. Price and quality tend to move together, though open source models sometimes break that rule. His advice is to go as cheap as you can while still doing the job. Don't reach for the biggest model for everything; he mentioned Sonnet as fast and very good, and said the Chinese models are very good and very cheap.
He also made the point that large language models aren't the answer to every automation problem. A classic step-by-step automation behaves the same way every time. Traditional anomaly detection, once tested over months, will keep performing. An LLM still carries some randomness, so he would keep classical processes classical and use generative AI to trigger them where it helps.
On testing, the useful phrase was golden questions: a set of questions and expected answers supplied by the customer, especially in domains where he isn't the expert. He's an IT person, not a finance specialist. Beyond that, a proof of concept goes to the customer, who gives thumbs up or thumbs down. That signal can be fed to an AI that proposes changes and opens a pull request, which a human reviews before a new version ships. If quality doesn't improve, you step back and question the architecture, the model or the connections.
Throughout, he kept returning to ownership. The developer steers. The AI does the lazy work.
Copilot, Foundry and owning the architecture
Jannik was candid about Copilot. He sees Copilot Studio as something for end users to click together use cases such as chatting with SharePoint, and fine for collaboration and quick wins inside the Microsoft estate. But he doesn't think you can build serious AI solutions with it. His weekend example: a simple request to sort emails into subfolders beneath his archive folder. It didn't work. His own application did it a minute later.
His own route is Microsoft Foundry for model deployment, for the data agreements, security stack and content safety, and then his own code against the APIs rather than the SDKs. He doesn't recommend that for everyone; you have to know what you're doing. Most of his applications run on Azure App Service, with Kubernetes usually overkill, plus Azure AI Search, Application Insights, Key Vault and a strong landing zone with proper pipelines. His frustrations with Foundry were around regional model availability and capacity, particularly when building in Europe.
He also had a practical gripe with native MCP: doing the same action a hundred times means a hundred calls and responses piling into the context, which burns tokens.
Above all, he said: don't trust the model. It may be a better developer than any of us, but it misunderstands, makes wrong decisions and works from outdated information. You need enough Azure knowledge to judge its suggestions.
Governing agents in practice
The governance stack he described is familiar to anyone in IT: sandboxed code execution with controlled URLs, content safety for prompt injection, agent identities, conditional access, Defender, and a Purview concept so agents only reach what they should. His key point was that you need solid governance for SharePoint, Teams and Azure first, and then layer AI governance on top.
For the registry, he uses Agent 365. Each agent gets an agent card with an owner, a sponsor, its access level, its tools and activity logs (with OpenTelemetry forwarding), plus an agent ID you can manage through Entra. Copilot and Foundry agents are discovered automatically. Anything built outside that needs a blueprint, an identity and federation done by hand. He thinks it isn't yet the single place for all AI activity, but it's heading the right way.
What I took away
For anyone in IT being asked to let agents loose, Jannik's approach gives you a checklist you already half know. Give agents identities. Onboard them gradually. Keep a registry of agents and MCP servers. Test against golden questions. And own the architecture yourself.
The part I keep returning to, though, was his answer about what we shouldn't lose: the human touch, the occasional spelling mistake, the boring post someone wrote themselves. I said on the show that my old crude product mock-ups took weeks and now take a morning. The speed is real. So is what disappears with it. That's partly why I still make this podcast the slow way.
Listen to the episode on Spotify