Building the digital workforce: Multi-agent apps with Azure AI Foundry | BRK148

0:00SHAWN HENRY: All right.

0:00Hi, everyone.

0:01Hope your Build is going well.

0:02We're kind of in the middle innings now.

0:04We're halfway through.

0:05Hope everyone's learning all sorts of stuff

0:08and enjoying themselves.

0:09My name is Shawn Henry.

0:10I run product for Semantic Kernel and AutoGen, and I'm here

0:14with my colleague, Salman, who is Engineering Manager

0:18for the Agent Service part of AI Foundry.

0:22So today we're going to talk

0:23about building your digital workforce, multi-agent apps

0:27with Azure AI Founders.

0:28This is going to be a very technical course.

0:29We've got a lot of demos,

0:32code-first stuff we're looking at.

0:33We'll spend a lot of time in Visual Studio,

0:36so I hope you guys are ready for that.

0:38But we'll start off with the slide.

0:42Who has not seen a slide like this like 10 times already?

0:45Right? Like, what is an agent?

0:47Yeah.

0:47So I won't spend too much time on this.

0:48You guys all know what agents are, I hope.

0:50Anyone not know what an agent is yet?

0:52I mean, I think there's all sorts of different definitions.

0:54But really, you know, an agent is, you know, is an AI app

1:00that has connected to an LLM to make inferences,

1:03has instructions and context, and most importantly,

1:06has tools so it can perform actions.

1:09Hopefully, you've all seen this already.

1:10What I want to talk about, though, is how we kind

1:12of deconstruct this and get from very simple agents all the way

1:15up to complex multi-agent systems.

1:18So first of all, we can think of agents

1:20that actually don't have tools.

1:21Now, really, like, a tool is what makes an agent.

1:24But there are times when you might actually use an

1:26agentic framework.

1:27You might actually use the Agent Service

1:29and not actually have tools that you're connecting to your agent.

1:33So if you have a simple task, like you're summarizing text,

1:35we've all done this by making inference calls to,

1:39you know, GPT-4 or whatever.

1:41But there's cases where you may want to use Postgres

1:43in the Agent Service or use tools

1:44like Semantic Kernel and AutoGen.

1:46And the reason for that is that you get state.

1:50You get that state stored if you want to make multiple calls.

1:52If you want to have that prompt stay persistent.

1:55If you want to observe the agent,

1:58you want to see telemetry.

1:59Or if you just want to have the same API across all

2:01of your systems and be able to grow, you know,

2:02start with agents and then build more complex agents

2:06and then multi-agent systems.

2:07So you know, it's a great idea to just start

2:09with these agent frameworks and use them, Agent Services,

2:14and use them, you know, from the beginning.

2:15Because very quickly you end up growing

2:17into more complex single agent systems.

2:21And kind of the canonical one that's up here is agentic RAG.

2:25So we've all seen rag where you do retrieval-augmented

2:27generation over search.

2:29And, of course, adding an agent

2:30to the mix can give you much better results.

2:33When you have an agent, it can take the human input

2:38from the user, can generate a bunch of queries,

2:40look at the text in natural language, generate a bunch

2:42of queries, do a search over that over your vector database,

2:46return a bunch of results,

2:47and then synthesize that result together.

2:49And there's lots of places where you would use a single agent

2:51like this, and they can get very complex.

2:53You can give them lots of instructions,

2:54and you can give them lots of tools to use.

2:56But at some point you'll often end

2:57up building a multi-agent system.

3:00And a multi-agent system is

3:01where you have many agents -- there go the cameras.

3:05All right.

3:06A multi-agent system is where you have, you know,

3:10one or more agents working together

3:12to solve a particular task.

3:13And, you know, things you can do

3:15with a multi-agent system is you can give it a very complex task

3:18to perform.

3:19So in this case we're asking it to build,

3:21kind of do some research

3:22and build a large marketing campaign.

3:24And so we have, you know, an agent that specializes

3:26in talking to the user.

3:27We have an agent that builds the plan.

3:29It's the coordinator agent.

3:30And then it talks to all these other agents

3:32and solves the problem and, you know,

3:35comes back with a solution.

3:36Each of these agents that it's using may be specialized

3:38into talking with a particular web service, doing searches,

3:42talking to a database, or executing code.

3:46So putting all these agents together enables you

3:48to build much more complex

3:50and reliable applications on top of agents.

3:53And we see data to back this up.

3:54So this is data where, you know, we've looked

3:57at single-agent systems with models, in this case GPT-3.5,

4:03and then built multi-agent systems with them.

4:06And they get very similar performance now to the kind

4:08of the next generation model.

4:09So 3.5-based agent systems with multi-agents now become like 4.

4:14As we do the same thing with 4, this also scales,

4:16starts behaving more like 4.1 and 4o.

4:21So this is kind of a great way

4:22when you build multi-agent systems is you can kind

4:25of get more performance.

4:28So you can use kind of older models,

4:30get the same performance.

4:31Older models, less expensive models,

4:32get the same performance, newer models.

4:34You can use newer models and get performance

4:36that you couldn't get with those models alone.

4:38And more importantly, you can kind of dial in your quality

4:42by building your agent system

4:44and having different agents collaborate together

4:46for a certain amount of time.

4:47You can kind of build a better-quality answer

4:49out of your multi-agent system.

4:51So that's why we build multi-agent systems is really

4:53to keep the agents on context,

4:55be able to solve complex problems with them.

4:59So at Microsoft, we have a lot of different ways

5:01for building multi-agent systems, and they all kind

5:03of fit together on a spectrum.

5:05You know, at the top end, at kind of the SaaS layer,

5:07we have Copilot Studio, no-code approach

5:09for building enterprise-connected agents.

5:12We have the PaaS service with Azure AI Foundry Agent Service,

5:18which allows you to have managed hosted agents available

5:20in the Foundry.

5:21And then we have things like Semantic Kernel and AutoGen,

5:23which allow you to build agents kind of much closer

5:25to the metal using your models and your vector stores

5:28and your tools directly in a pro-code environment.

5:36So what are we going to talk about today?

5:38So we're going to talk

5:39about building agents on Azure AI Foundry.

5:42And there's a lot of things you can do with that.

5:43You can see kind of the menu here, and we'll touch

5:45on a bunch of these things.

5:45We'll talk about connecting your agents to things

5:49like Azure AI Search and Bing for grounding.

5:52We'll talk about all the different models

5:54that you can use that we support through Foundry.

5:56We'll talk about the open ecosystem and how we have agents

5:58that connect together and how we connect agents

6:00to other services.

6:01But we'll really focus on two areas.

6:03One at the bottom, we'll talk about the Foundry Agent Service,

6:06which is where you can build and host your agents

6:08and your multi-agent systems.

6:10And we'll talk about the agent frameworks that you can build

6:12on top of agents hosted in Foundry or agents

6:15that are hosted elsewhere.

6:17So to kick things off, I will hand it over to Salman to talk

6:20about the Agent Service.

6:23SALMAN QUAZI: Perfect.

6:24Thank you, Shawn.

6:26So I'm super excited to be here today.

6:28Just a few months ago, we announced public preview

6:31of the Agent Service at Ignite,

6:33and today we're making it generally available

6:35for everybody.

6:37One of the most important things we've been working

6:39on over the last year or so is to make the service reliable,

6:44secure, and interoperable.

6:47We already have tens of thousands

6:48of customers using the Agent Service

6:50to run production workloads, and we've been processing trillions

6:55of inference tokens and terabytes of vector indexes

6:58that customers are already depending on.

7:01So I'm very excited about the general availability

7:03so that you can actually build production workloads

7:07on top of this.

7:10So while we were working on all of the security

7:14and reliability aspects, there were three key areas

7:16that we were focusing on in terms of differentiation.

7:20One is around enterprise-grade security.

7:24So the Agent Service, by definition,

7:26actually is a stateful service,

7:29which means that we persist a lot of state

7:31about conversation history, about run execution,

7:34and we do these things so that we can simplify recurring

7:38complexity that you all have to deal with.

7:41So what we've done is we've made these state stores

7:44single-tenant, which means that they live or optionally can live

7:48in your subscription, and you have full configurability

7:52of these resources through Azure policies and other controls

7:57that Azure already provides.

7:59One of the most important things that you can do

8:02on these resources is that you can put a virtual network

8:06so you can actually contain what services can talk

8:10to those resources, and that provides you defense in depth

8:15on top of everything that you're doing.

8:19The second differentiation

8:21that we've done is around model choice.

8:23We've heard consistent feedback from customers around being able

8:27to leverage just not OpenAI models,

8:30but other models as well.

8:31And as you've seen from many talks, that we have

8:33over 1,000 models in our model catalog.

8:37And independent of the projection

8:39of enterprise-grade security and the hosted tools that we have,

8:43you can select any model from the model catalog

8:46and have the Agent Service interoperate

8:48across the model ecosystem.

8:52And then lastly, we've invested a lot

8:54around building hosted tools.

8:58So hosted tools is its idea that they are services

9:01that are managed by Microsoft,

9:02and you can just use them to provide value.

9:06And we've added tools like SharePoint grounding,

9:10which allows you to ground your agent responses

9:14through SharePoint sites.

9:15We've added Azure Functions integration, which allows you

9:19to have hosted functions running in the cloud

9:22and have the agent interact with the Azure function directly

9:26over the cloud as opposed to coming back down to the client.

9:30We've also invested in OpenAPI-based tools,

9:34which allows the agents to talk to any API

9:37that has a Swagger spec, which most do,

9:41and you can connect these agents to be able to read

9:45or use these APIs to make mutate state outside of them.

9:50We've also have Logic Apps integration

9:52that we're announcing.

9:53And some of these tools are in preview,

9:55but the service itself is in GA.

9:58And the last one that I'm very excited to talk

10:00about is a browser automation tool,

10:03which will allow browser automation tasks

10:06through a hosted browser scenario.

10:10So that is our single agent story, right?

10:14An enterprise-grade, interoperable,

10:16secure set of service with tons of hosted tools

10:20and many more to come.

10:22And this gives you a cohesive story that you can go in front

10:25of your security organization with to say

10:28that this is our security model

10:31and that satisfies the most strictest of organizations,

10:34including banks and healthcare.

10:38But today we are here not to talk about single agents.

10:43We're here to talk about multi-agents.

10:45And to talk about that, I want to make sure all

10:49of you have this sort of picture in your mind as we go

10:53through the demos today.

10:56When we design traditional software,

11:00we have complete deterministic process

11:02with deterministic outcomes.

11:03You control every aspect of the control flow,

11:06and you know exactly what's going to happen.

11:08In fact, there's a name

11:09when traditional software don't have determinism.

11:12It's called segmentation fault.

11:14If you haven't planned for it, things happen

11:16and your process crashes.

11:18But on the other end of this is a probabilistic process

11:22and a probabilistic outcome, and these are multi-agent systems,

11:26where you're completely sort of relinquishing control

11:28over how the control flow works, and you're leveraging these LLMs

11:35to direct flow control.

11:38Now, obviously, we cannot live in this other spectrum, right?

11:40You can't build software systems

11:43that are completely probabilistic.

11:44You want predictable outcomes for your end customers.

11:49And I'm going to talk about how we will achieve

11:51that through a new service that we're going

11:53to be announcing today, which is in preview,

11:57called the AI Foundry Workflow Service.

12:00But to get to that, I'm going to sort

12:03of slowly make the progression.

12:05We want to start with the single agents, and we're going to go

12:09to this concept called connected agents.

12:13And to do that, I'm going to switch over to my demo,

12:16because I think we've had enough time on PowerPoints.

12:22So we're talking about connected agents.

12:25Now, all of you have built single agents.

12:28Single agents, you've given it power through APIs and tools.

12:32Connected agents is this idea

12:34that you use other agents as tools.

12:37So I'm going to go ahead and quickly create one.

12:39So I'm creating a new agent.

12:41And down below, you'll see this new sort of toggle

12:45that has this concept of connected agents.

12:48And all you're seeing here is essentially a list

12:50of all your existing agents within this project scope.

12:53So you can simply connect

12:55to your Bing search agent, give it a name.

12:59And you can provide instructions to this main agent

13:02that I'm creating now, Agent 856, and say,

13:06trigger this agent any time you need real-time information.

13:15And this works in conjunction

13:16with however the sub-agent itself is configured.

13:20So as you can see, when you're building traditional software,

13:23you don't write everything in a single main method.

13:26You break it down into different functions.

13:29Functions have single responsibilities.

13:31So sort of in that same mental model,

13:34sub-agents are those functions.

13:36They do a very small set of tasks.

13:38They are prompted.

13:40They're given tools that perform that specific task.

13:43And this top-level agent is then able to use that sub-agent

13:48to accomplish higher-level tasks.

13:50So I'm going to go ahead and add that.

13:52And I'm going to add one more.

13:54And I'm going to add this coding agent.

13:57Give it a name.

13:58Coder.

13:59You can use this agent for coding tasks.

14:08And notice that I am not spending time mentally thinking

14:12about how the code execution agent is configured,

14:16what tools it has access to,

14:18what permissions it's given, how it's prompted.

14:22All of that becomes sort of the responsibility

14:24of that single function or the single agent

14:27that I'm now composing up.

14:29So I'm going to add that.

14:32And then, now that I have this,

14:33I can just try it in the playground.

14:35Now, the beauty of this is that the API interface that you use

14:39to interact with a single agent is the same API interface

14:43that you have to talk to this connected multi-agent.

14:46Because, at the end of the day, they're just tools.

14:48So it works really nicely.

14:50So I can ask, what is the weather

14:52in Palo Alto, California?

14:57And notice, I don't have Bing grounding set up.

14:59I don't have any of those tools set up.

15:01And yet, this top-level agent talks to this connected agent

15:07to hopefully give me an answer.

15:09And these are all live demos.

15:11So I am not -- I actually don't have anything recorded.

15:15So as you can see, this works.

15:18So you can actually break this complexity down.

15:20And you get better precision and recall

15:23because these sub-agents can only view a narrow scope

15:26of the world rather than the entirety of things

15:30that it would otherwise have to see.

15:33So you get much better control and precision.

15:37So that is connected agents.

15:39So this takes you from the world of building single agents.

15:43As you start to hit limits on them, as you start

15:46to add more functions and tools, you may slowly graduate

15:50into this concept of connected agents

15:52to provide you additional value.

15:54But this, again, is an implicit orchestration.

16:00What I mean by that is you are specifically not controlling the

16:06order in which these agents are invoked,

16:09how many times they're invoked, or what parameters or values

16:12that are used to invoke them.

16:15All of those things are sort of left up to the LLM to decide.

16:19And that generally works, but that may

16:22or may not be what you desire.

16:24So for that, I was going to show you a demo today

16:28around the workflow service,

16:30which is something I'm very, very excited about.

16:34So again, to mentally associate yourself, we're not recommending

16:43that you jump into building workflows right away.

16:46You build your single agents.

16:47You see what these limits are.

16:49You start to go into connected agents

16:51because all your tool chain, all your SDKs, they all work.

16:55So it's an easy progression to get to connected agents.

16:59And then if you start to hit limits within connected agents

17:02because of particular scenarios that you're trying to build,

17:05then you can sort of move on to workflows.

17:07And even workflows use the same set of tool chain.

17:11So it is an easy transition.

17:13So we'll talk about that today.

17:18So the class of problems workflow solves are

17:22where you want much more granular control

17:25over how the control of these agents happen

17:30and the data bindings that need to happen

17:34as you move through the system.

17:37So to give you an example,

17:38if you're building a customer agent bot,

17:41you don't want the bot to act in a way where I can log in

17:45and say, my name is Shawn and all of a sudden,

17:48the bot treats me as Shawn and starts to act like I'm Shawn

17:51and give me all of Shawn's information.

17:53So there's particular data bindings that you want

17:56to guarantee that the model will not hallucinate

18:00and they work consistently throughout the stack.

18:02So those are things that you get out of the workflow service.

18:06And the second thing you get is

18:07that unlike other multi-agent systems like LangChain or Swarm,

18:13where they are essentially SDK offerings,

18:17you basically build multi-agents through code,

18:21we have a declarative system for multi-agents

18:24where you declare what your process looks

18:26like through Semantic Kernel's process framework.

18:30And once you declare it, you can publish it.

18:32And when you publish it, you get an endpoint that you can use

18:35to interact with that workflow.

18:37So with that context in mind,

18:39let's spend a few minutes looking at what this setup is.

18:44So to do this, I'm going

18:45to build a deep research multi-agent.

18:50Now, deep research, some of you may have used in ChatGPT.

18:54It's an option where the agent goes into doing research

18:59on the web, does follow-up research, and eventually

19:01after tens of minutes of work,

19:03it produces a pretty comprehensive summary

19:07of that work.

19:09You can use it to research your next vacation plans.

19:12You can also research it to figure

19:14out many other complex tasks

19:17that almost produce publication-quality work.

19:22Now, to do that, we have built this deep research agent using,

19:28or at least inspired by the AutoGen team's work

19:31through Magentic One, where they have this notion of teams

19:36where each block here is an agent

19:42that does something specific.

19:44For example, it can do research, it can gather facts,

19:48and it maintains this ledger of work where the planner,

19:54essentially given the task, plans a ledger of work.

19:57And as the work gets done, the ledger is marked as completed.

20:02And eventually, when all of the work in the ledger is done,

20:05the research is considered to be complete.

20:10Now, we're going to open-source all of these workflows

20:16that you can then build on top of.

20:18These are opinionated versions of things

20:20that we think how deep research should be done,

20:22but you can change them, you can mutate them,

20:25you can make them even better.

20:27And so to run this, the way we have this configured today is --

20:34let me go ahead and close that.

20:38So we have this notion within the workflow service of sort

20:47of dependency injection,

20:48where the workflow can be given an instance of another workflow,

20:53and that instance can be injected with a set

20:56of properties that you can use to run it.

21:00So in this case, as you can see, this workflow has a single box.

21:03The single box is that deep research workflow

21:07that you saw earlier, and it accepts a certain set of inputs.

21:11For example, it accepts a set of team members and an instruction.

21:14The team members are the specialized agents

21:18that can do work.

21:19For example, it could be a research agent

21:21and a coding agent.

21:22So you can inject those team members to it,

21:25and you can start off interacting with it.

21:29So what I'm going to do now is, as you can see, I am connected

21:34to my Build 2025 project, AI Foundry project,

21:39and I'm going to deploy this.

21:42And what it does is it deploys this workflow into the service,

21:47and you get a managed endpoint that you can use

21:49to interact with this workflow.

21:52So this is the point that you're not actually building this

21:57in code.

21:57You're just declaring what your workflow looks like.

22:01Once that's done, so as you can see, there's a workflow ID

22:03that comes up, and I can go ahead and run it.

22:08When I run it, what I get is a chat experience directly

22:13in VS Code, where I am actually interactively chatting

22:16with this workflow.

22:18Now, our workflows have two modalities.

22:20You can interactively chat with them for interactive scenarios,

22:25but you can also do asynchronous workflows

22:29where you can run them, and they can run in the cloud for hours

22:32and days, and we will take care of hydrating state,

22:37persisting them, and rehydrating them when necessary,

22:40and doing it in a way that's durable, so that you don't have

22:43to deal with that complexity.

22:45So to get this going, I'm going to just give it a task

22:49of studying the latest S&P report

22:52and generate a report for me.

22:55And this will take a while, so I'm going

22:57to just slowly kick this off and then move on to something else,

22:59but I want to show you that there's these events that happen

23:03that the workflow has started, and it's going

23:06to produce a ledger of work,

23:08so it knows exactly what the plan is,

23:11and then other agents sort of pick up work from that ledger,

23:14and they do them, and ultimately, this completes.

23:16So we'll see it when it's sort of done in the side.

23:21So I'm going to let this run, and in the meantime,

23:24because this is obviously a developer talk,

23:27I want to spend time showing you what it looks like building one

23:31of these workflows in code,

23:33because that's precisely what most of you will be doing.

23:37So to do that, obviously, we can use Semantic Kernel,

23:44but I wanted to take this sort of a step further,

23:47where I'm using a classic sort of SDK

23:51to build these single agents, and then just

23:53for the multi-agent piece, I'm using the process framework.

23:58So if you look at my CS project, for the process framework piece,

24:04I'm using this Semantic Kernel package,

24:07and my code is pretty simple.

24:11So this is a very simple setup.

24:13I've got a teacher and a student, so two agents.

24:17The teacher agent is configured

24:19to give preschool-level math homework to a student,

24:23and then the student is asked to do that homework.

24:27And if they get it right, you progress into the next state,

24:30and if they get it wrong, they're asked to continue

24:32to retry until they finish.

24:36And then once I create those agents,

24:39you can see that there's this build operation that I can do

24:43on it which builds the process,

24:47your business process automation.

24:49So here is where I'm using process framework.

24:52So if you've been to Scott Hanselman's talk yesterday,

24:56he used the exact same SDK to build a linear process

25:00where he automated sort of his podcasting flow.

25:03So I'm using a different kind of process

25:05to have these two agents talk to each other.

25:08So this brings up another interesting point

25:10about process framework is that unlike other multi-agent systems

25:16where there is sort of official support for conversation models

25:20where you have two-way chat or group chat or handoff,

25:26here we don't have those notions.

25:28We work off of events, and with events you can model any type

25:33of conversation as long as you raise the right events

25:37with the right payload.

25:38So event-based programming is something that we've been doing

25:41for decades now, so it's sort of bringing

25:44that same concept into multi-agents.

25:48And so here I've got two agents that are configured external

25:54to this method, and they happen

25:58to be creating two specific threads that they are using.

26:02So you can see this term scoped.

26:04What this means is that these two agents have a narrow view

26:08of the world.

26:09They get to see only the things that they are operating on.

26:13Now this is just a choice because of the problem

26:15that I'm trying to solve, but I could be trying

26:18to solve a problem where I have shared knowledge,

26:20which would be effectively what group chat is,

26:24because group chat is this notion

26:25that every agent has universal knowledge

26:28of everything that's happening,

26:29whereas this is sort of a two-way chat.

26:33So you can model a lot of these conversation states using very

26:36simple states and transitions.

26:39So once you do that, you build this process,

26:45and you can publish it.

26:47And when you publish it, you get a workflow ID.

26:51So these are sort of virtual identifiers that you can use

26:55to execute tasks on top of.

27:00So the key point to take away here is

27:03that the process isn't actually building.

27:08You're not writing code to define your multi-agents,

27:12but you're writing the process through code.

27:16And once you do that, we fall back to all the existing APIs

27:20that we have seen before, create, thread, and run.

27:23And instead of running a single agent,

27:25which would be the parameter here,

27:28here we're giving it the workflow ID,

27:30which is the multi-agent.

27:32So that is something we also spend a lot

27:34of time optimizing on, where we want

27:37to make your transitions very simple.

27:40We don't want you to move to a different stack,

27:42a different programming model.

27:44You can stay on the existing programming model

27:46and make minimal changes to switch into this next thing.

27:50And then I pass in a user message,

27:52and then I start printing these streams as I get back.

27:55So let's go ahead and quickly run this.

27:58And this is all real.

27:59So none of this is recorded.

28:01So I'm hoping this comes up.

28:05And as it's coming up, we can go quickly check what is happening

28:10with the other agent.

28:13So as you can see, this is the multi-agent

28:16that created a ledger.

28:17It created a plan on step one to gather data,

28:21step two to analyze the data, execute code,

28:24compile the report, and these would be the deliverables.

28:28So this is quite nice.

28:31And then you can also see

28:32that the Bing grounding tool was used.

28:34It was used to do a specific search about latest S&P

28:38and performance in May 2025, which is the month we're in.

28:42So you get a lot of observability through our stack.

28:44So you can see exactly how the tool interactions are happening,

28:47what inputs and outputs are provided to the tools.

28:50They all sort of show up directly in VS Code.

28:52You obviously could do this in the Foundry portal too,

28:56but this is where you are.

28:58So we just natively connect to that.

29:02And eventually, when all of this work is done,

29:04you can see there's a coder agent that was pulled

29:06up to actually do some coding work.

29:09In this case, it was given a bunch of different industries

29:12and sort of their performance over time.

29:15And a plot was asked to generate.

29:18And we used our code interpreter service, which is a hosted tool

29:22to generate that plot.

29:24And eventually, after all of this work was done,

29:27you get sort of the visualization.

29:30And ultimately, I think the work is still happening.

29:35So we'll leave it be for now and we'll go back to this.

29:40So now we're back to the two-way chat student-teacher.

29:45And I'm just going to say go,

29:46because it actually really doesn't matter what I say here,

29:49because the, and this is this idea of,

29:54having much more granular control over the flow.

29:58It really doesn't matter what I say, because the teacher

30:00and the student are directed to work in a very specific way.

30:03So I'm going to just start that.

30:05And you will see that the student comes up and says,

30:09I'm ready to answer your question.

30:10Teacher, what should I do?

30:12The teacher says -- this is all real.

30:16So maybe it's thinking.

30:19I'm not sure.

30:21We'll give it a few more minutes.

30:24All right, here's your question.

30:25Three apples, you've got two more apples.

30:26How many apples do you have in total?

30:28And then the student says, yeah, yeah,

30:30let's try figuring out this math.

30:33And it does that.

30:35It's like, five apples in total.

30:37And then it goes back to the teacher for correction.

30:40So the teacher then checks the math to make sure

30:43that the math is correct.

30:44And it is.

30:45So it says, yes, you're done.

30:47And then it deletes the user thread that I was on.

30:50And then I can finally hit enter.

30:52I can delete those two teacher agents and the workflow,

30:55which all of these are ephemeral entities.

30:58You can create them, use them,

30:59and then delete them whenever you're done.

31:02So that was the demo of workflow agents.

31:07So let's go back here just

31:09to make sure this actually completed.

31:13The task has not been completed.

31:14So it's working and instructing other workflows to do stuff.

31:17And you can see some of them will fail because, you know,

31:20I may have exhausted my tokens usage for the time being.

31:25But there's sufficient retries in place to try

31:29and complete the task.

31:32And then the last thing I wanted to show is

31:36that we've been also working very hard

31:41around interoperability around A2A.

31:44So Google recently announced a protocol

31:47for agent-to-agent communication.

31:49And to show that off, what I'm going to show you here is a demo

31:55where I've got an agent that I've created, in this case agent

32:05AS GZA that has some tools and things involved.

32:10And this particular project, if you look at its dependencies,

32:14has everything A2A, Azure identity

32:17to obviously authenticate with Azure services,

32:19but everything else is just A2A client, A2A HTTP.

32:24But because our agents starting today is going

32:28to be A2A compatible, what this means is you can actually

32:34interact with them through the A2A protocol.

32:36So these are all A2A constructs.

32:38You create a send task request,

32:40you wait for the response and you print it.

32:43So if I run this, I should hopefully see

32:49that I can interact with this agent using the A2A protocol.

32:52So I mean, we'll extend this further

32:54into our workflow service itself.

32:57So our workflow service will not only be able to interact

33:00with single agents that are designed by us,

33:04but single agents that live in other clouds.

33:07So you can then interoperate across the cloud platforms.

33:12And as you can see, the agent responded with, I'm doing great

33:16and thanks for asking.

33:17And this was all through the A2A protocol.

33:19So thank you.

33:24[ Applause ]

33:25And with that, I am going to bring over the JM Family

33:30to talk a little bit about how they've been using our

33:33multi-agent stack to provide value to their organization.

33:40[ Applause ]

33:40Welcome, John.

33:41Nice to meet you.

33:42Amit, nice to meet you.

33:44So tell me a little bit more about how you found value

33:48in using multi-agents within the JM Family organization.

33:52JOHN: Yeah, definitely.

33:52So interestingly enough, you know,

33:54one of our biggest use cases was actually

33:56within our own technology organization.

33:59So for those of you who might not be familiar,

34:01JM Family is a $22 billion privately held company.

34:05Our businesses include,

34:06we're the largest independent distributor

34:08for Toyota in the world.

34:09We do captive automotive financing for Toyota customers

34:13in the Southeast US and JMFE Group business has finance

34:17and insurance products that we provide and manage

34:20through auto dealerships across the country.

34:22Now, all of these business units are

34:24in massive technology transformations all

34:27at the same time.

34:28One of our biggest challenges was the inputs our engineers

34:31were getting to execute those technology transformations.

34:34So things like requirements, user stories,

34:37even though we have published standards and guidelines,

34:40the level of granularity, the form, the format,

34:44varied wildly across teams.

34:46This turned out to be a perfect opportunity for agentic AI.

34:51This was producing everything from defects downstream

34:54to just countless of hours spent going back and forth,

34:57trying to get clarity on what actually could have been more

34:59clearly outlined up front.

35:01And that's where our captive product development entity,

35:04Mosswall Technologies stepped in and built a tool

35:07to bring this capability through multi-agent

35:10to our product owners, our business analysts,

35:12quality analysts, as essentially a team of AI assistants

35:16to help generate the inputs

35:18to give our engineers what they need to do best,

35:20which is build great software.

35:22SALMAN QUAZI: That's awesome.

35:22And what were some of the return

35:24on investment done using the stack?

35:26JOHN: Yeah.

35:26I mean, not only, you know, did we see the process time

35:30to create these inputs shrink by 30 and even upwards of 40%,

35:35but just the time saved in meetings.

35:37And I know how much, you know,

35:38we as engineers absolutely love sitting in meetings.

35:41So people are probably heartbroken,

35:43but that time could actually be spent

35:45like engineering novel concept.

35:48SALMAN QUAZI: That's great.

35:49Awesome. And then any key learnings, any takeaways

35:52that you want to share, Amit?

35:53AMIT: Interestingly, there's a lot of key learnings.

35:57If I spend time, I can just keep talking about it.

36:00So starting from, you know, one of the learnings is you need to,

36:04when you start any of the use case,

36:06you need to be very precise what you wanted to achieve.

36:10Make sure you have clear rules defined for these agents.

36:14And, you know, you need to test these Agent Service

36:18as individually.

36:19Now talking about platform, someone has talked

36:23so highly about this platform.

36:25I must say, we have leveraged pretty much everything

36:29from human in the loop to going back to, you know,

36:33the agent interaction, four or five agents interacting

36:37with each other successfully, as you can see on the screen.

36:40Now, one thing I wanted to highlight

36:42about this platform is we have looked

36:45at various other platforms or frameworks like LangChain.

36:49And in fact, one of the things was AutoGen.

36:51One of the challenges we faced over there was,

36:54how do we host these services?

36:56It's always being a challenge.

36:58Managing that session, threads, data,

37:01and users is always in the issue.

37:03And thanks to, you know,

37:05Microsoft bringing this Agent Service as a PaaS platform

37:09for us, where we worry about business use case rather

37:14than worrying about the infrastructure.

37:17Human in loop has become very seamless.

37:19It was very easy.

37:21As you can see on the screen, there are four

37:22or five agents running around.

37:24My demo is not as fancy as the newness

37:27which has been introduced, but there's a lot of easiness.

37:31You worry about configuring FDL,

37:35which is a YAML type configuration rather

37:38than writing a code.

37:39JOHN: I would say just the last thing, you know,

37:41outside of the technology or in addition to the technology,

37:44just the multi-agent paradigm itself is a lot easier

37:48of a concept for our users to grasp.

37:50So historically, Gen AI retrieval methods,

37:53like it's been seen as a bit of a black box,

37:55but with multi-agents and having very specifically designed

37:58and intentional purpose-built agents that work collectively

38:02with our users, it's just opened up a much wider dialogue

38:06of how we can leverage AI in the enterprise.

38:08SALMAN QUAZI: That's great.

38:09Thank you.

38:09Thank you, JM Family.

38:10[ Applause ]

38:15All right.

38:15And with that, I'll hand it off to Shawn to show off a lot

38:18of nice Semantic Kernel stuff.

38:20SHAWN HENRY: All right.

38:20Thanks, Salman.

38:21Yeah. So let's move over from talking

38:24about managed Agent Services, and we'll talk

38:26about our client agent orchestration frameworks

38:29and how you can use those with our managed services.

38:33Oh, we're over here.

38:33That's right.

38:34So you know, within Semantic Kernel, we have a lot

38:38of the same capabilities that Salman talked

38:40about for building multi-agent systems.

38:42I already talked about, and Salman talked about kind

38:44of these LLM-driven applications where the LLM is in charge

38:49of flow control and decides which tools to use,

38:51how to execute the plan.

38:53But then also having the need for process-driven flows

38:59where you have a lot more control

39:00over which agent executes in which order.

39:02And, you know, these kind of things, you need to look at them

39:04and decide what is the right application for the problem

39:08and the solution you're trying to go after.

39:10And in cases, you may want

39:11to have composability between these.

39:12So you can see in the one on the right here where we have,

39:15you know, a process-driven flow that's very directed,

39:18but within it we have some agent-driven flow,

39:20some LLM-driven flow where you have this radar

39:22and critic agent talking back and forth.

39:25So all these things are capabilities that, you know,

39:27we have within Semantic Kernel,

39:29which is our open-source agent framework

39:32for building agent applications and orchestrating AI systems.

39:38And I will quickly get into a demo here.

39:40Let's take a quick look at how we build agents

39:44with Semantic Kernel on three.

39:47Here we go.

39:48Okay. So I'm going to kind of follow

39:52that same pattern that we had before.

39:55We're going to do first like a no-tool agent,

39:57the most simple agent that you can build.

39:59And you can see here it's just about three

40:02or four lines of code.

40:03So if we look here, building our agents, we create our agent,

40:09we connect it to our chat completion agent type

40:12within Semantic Kernel.

40:13And this is the case, you know, Semantic Kernel,

40:15we support connections to all sorts of different agents.

40:17This can be a chat completion agent,

40:19which means you're bringing your own model,

40:21you're bringing your own connections and your own tools,

40:24or it could be a managed agent, it could be an agent hosted

40:26in the Azure Agent Service, the Foundry Agent Service,

40:29it could be a Copilot agent that's living

40:31in the Copilot stack.

40:33You know, it could be a responses agent from OpenAI,

40:35it could be an agent from AutoGen,

40:37which is our research framework for multi-agent systems,

40:39or it could be an agent from something like Bedrock for AWS.

40:41So you can take all these agents and you can connect to them

40:44with Semantic Kernel and you can ask them questions.

40:46So in the case of this agent, you know,

40:48we're just asking something very simple.

40:49We're asking it to compose a haiku about Semantic Kernel.

40:54So we gave it these instructions and we asked it for a response,

40:58just a single response in this case,

40:59we're not having a conversation.

41:00So we're using this almost exactly like an inference API.

41:03And that kind of goes back to what I was talking about,

41:06in cases where you, you know, you start with an agent system,

41:10even when you're just doing responses, it means you can grow

41:12into more complex things, you know, as you go forward.

41:15So we've got a nice little haiku here.

41:17Code that learns and grows,

41:19patterns weave a mindful core, Semantic Kernel.

41:21Great. Beautiful.

41:24Let's talk about real agents.

41:26So building real agents, you know, with more capabilities,

41:33code very much the same in Semantic Kernel,

41:36but we've added a few new lines.

41:39So we've added these plugins, we've added a kernel object,

41:43and we've added arguments.

41:44So what are those?

41:45So first of all, our plugins, these are our tools.

41:47This is where we connect our tools and other capabilities.

41:50So in this case, I'm connecting with, you know,

41:53a tool that I've created in code.

41:54I could connect to OpenAPI from here.

41:57I could connect to MCP, connect to other agents.

41:59I'll show that in a second.

42:01This is where I put these in the plugin.

42:03In the kernel, I can figure things like telemetry

42:06and filters and things

42:10that I can see what's going on with my system.

42:12And then arguments, I tell the kernel how to handle that data.

42:15So let's look at menu plugin first.

42:18So menu plugin would be something where you normally,

42:20you would connect this to a database or a data store.

42:22And by the way, all these samples are in Python.

42:25I have the link at the end.

42:26We also have all this stuff in C Sharp as well.

42:28I'm going to go back and forth between.NET

42:31and Python a little bit.

42:31So don't worry if you're a.NET person or a Python person.

42:34I got you covered.

42:37We've created our menu plugin class here,

42:38and then we can annotate it

42:40with these kernel function annotations,

42:42which now tell Semantic Kernel to take these functions

42:46and stick them in the prompt that we give the LLM and tell it

42:49that your agent now has these tools to use.

42:53The other piece we're doing is with the settings,

42:55we're asking it to output in response format menu format.

42:59So instead of getting a natural language response,

43:01we're going to get a response in structured data.

43:04And then finally, we're adding this filter.

43:06So the filters are what tell us what tools calls we're making,

43:09what things we're asking the LLM to do, and ultimately,

43:11this is where we'll hook up our telemetry.

43:14So we'll run this agent, and this is an agent

43:17that answers questions about the menu, and we're asking it,

43:20what is the soup special and its price?

43:21So this is like one of those composite questions.

43:23There's two pieces to it.

43:25It needs to call back into our system twice,

43:27one to get the list of specials up here, clam chowder,

43:31cob salad, chai tea, has to identify

43:33which one is the soup special, then call back in and figure

43:37out what the price is.

43:40So we can see here, if we look at our output,

43:45our plugin was called with get specials.

43:48It's called with context.

43:50The response gave us three specials, return that string.

43:54And then our plugin was called again,

43:56this time with the menu item call, and it replied

44:00with clam chowder, $9.99.

44:03And then the LLM took those two responses together,

44:05and instead of generating natural language,

44:08which sometimes you want when you want to send that directly

44:10to a human, to a user, you want natural language,

44:13but sometimes you want structured output.

44:14So in this case, we're building structured output.

44:18Okay, so that's how you connect tools.

44:21You can connect all sorts of things to Semantic Kernel,

44:22and I'm actually contractually obligated to talk about MCP.

44:26So very easy to add MCP into Semantic Kernel.

44:30Actually, MCP is great.

44:30Probably the most important thing in the last three

44:32or four months that's happened in this world,

44:34in kind of the AI and agent world.

44:36And definitely something --

44:37SALMAN QUAZI: MCP versus agents,

44:39everybody's talking about.

44:40SHAWN HENRY: Yeah, which one was more?

44:42SALMAN QUAZI: Yeah.

44:42SHAWN HENRY: Well, we did the full SEO in this topic

44:44where we're talking about we got agents, we got MCP,

44:47we're going to do A2A.

44:47SALMAN QUAZI: We got all of it.

44:48SHAWN HENRY: Yeah, exactly.

44:49We got everything.

44:50That's why you guys are here.

44:51Okay, yeah, so MCP, you know, a great way to surface tools

44:57and build them and connect them to your agents.

45:00Very easy to do.

45:01I'll just run this sample real quick.

45:02This is connecting to the GitHub MCP server.

45:06So there's all these servers available.

45:07If you're not familiar with MCP,

45:09there's these services available.

45:10You can connect to them.

45:11Semantic Kernel can be a client.

45:12We can plug it in just like we saw before

45:14with that menu plugin.

45:15We can plug it in directly to our agent

45:18and then now it has access to all the tools

45:20that the GitHub MCP server has available to it.

45:25So I can now check up on my team and I can see, you know,

45:29what are the top issues they're working on right now?

45:32And we can look, we can see what's going on.

45:34Okay, look, we got a new issue.

45:34Oh, missing index and function.

45:36We got to get on that one.

45:38So this is all happening under the covers.

45:40I just had to give it a natural language request here.

45:42I asked it for what are the latest five issues

45:46in a Microsoft Semantic Kernel repo and it was able

45:48to return this without me having to figure

45:50out the API contract with GitHub.

45:53This is a really powerful way to use MCP.

45:55You've probably seen a lot of MCP being used

45:57with like Visual Studio Code and agent mode and GitHub.

46:01Sorry, and Copilot Studio agents as well.

46:05And now you can use it with Semantic Kernel.

46:07One of the things I do want to point out, I don't have time

46:09to run the demo for it, but what do you think is happening now

46:12with all these MCP servers?

46:14There are way too many of them and so your agents now have

46:17like thousands and thousands of tools to use and they break.

46:21Your agents can typically only handle, most models,

46:23you know, 10 to 20 tools.

46:25So something that we have a sample

46:26for in our Semantic Kernel repo I encourage you guys to go look

46:29at is actually taking all of your plugins

46:33and running a semantic search on them first.

46:35Put them in a vector store and then figure

46:37out what are the best set of plugins to give to your agent.

46:39Something that can really help

46:40with the performance of your agent.

46:41So there's a little nugget for you

46:42as you're building your agent systems.

46:44But you guys paid for multi-agents,

46:47so let's look at multi-agents.

46:51Okay, so as Salman mentioned with like connected agents,

46:54you know, very easy to add agents in Semantic Kernel.

46:57Here we have three agents we've created.

46:59We have a billing agent that focuses

47:01on kind of billing issues.

47:03We have a refund agent focusing on refund agents

47:04and then we have this triage agent that kind of looks

47:07at the incoming request from the user and decides

47:10which of the agents it wants to use to,

47:12you know, resolve that request.

47:14And the other thing you'll notice here is there's two

47:16different types of agents.

47:17So we have our chat completion agent which we've looked at.

47:20Well, that's not what I meant to do.

47:21There we go.

47:22We have our chat completion agent that we've looked

47:24at before and we've given it some instructions.

47:26But these agents up here we've only given them this ID

47:29and that's because we've defined these agents

47:32in the Agent Service.

47:34So I can hop over to the portal

47:35or I can use the Virtual Studio extension

47:37and I can see my agents here.

47:38I have my refund agent and I have my billing agent

47:41and I can manage these all within the portal.

47:43This is a great way to get started

47:44with multi-agent systems.

47:45You can put your agents in the portal.

47:47You can give them prompts.

47:48You can connect them to Bing grounding.

47:49You can connect them to OpenAPI.

47:52You can do all that stuff in a managed way

47:54and then you can build a multi-agent system

47:56with Semantic Kernel.

47:58So I'm going to run this but I'm actually going

47:59to run the UI version here.

48:01I really like using a tool for UI when I'm developing stuff.

48:06I really like to use ChainLit which is a nice tool for Python.

48:12And what you'll see is we've connected the agents

48:14up as plugins.

48:16So we have our triage agent and it's using those two agents,

48:19our billing agent and our refund agent as tools.

48:22And when this runs, I can ask it --

48:25I think I got a query here that I like.

48:31Here we go.

48:31So I'm going to ask it another composite question.

48:33You know, two things that need to be researched here.

48:36I'm going to say I was charged twice

48:38for my subscription last month.

48:40Can I get one of those payments refunded?

48:41So hopefully what's going to happen and typically happens,

48:44these are pretty good systems now.

48:45They're getting pretty good with their tool calls.

48:47We're now making tool calls to billing agents

48:49and tool calls to refund agents.

48:50These are both happening in parallel

48:52and we can see here the response that we get.

48:54So we've got an input into the billing agent

48:57from our triage agent and we've got a response

48:59from the billing agent.

49:00Same thing happening with the refund agent.

49:02We give it an input and we get a response.

49:05And now our triage agent takes those two responses

49:08and generates a human readable response

49:10that composes those two responses together.

49:14So a nice way to do that with connected agents

49:20and agents as plugins.

49:22But I want to talk about more advanced multi-agent systems.

49:28So I'm going to move over here to this one and I want to talk

49:32about AutoGen and Semantic Kernel.

49:34For those of you that aren't aware,

49:35we have a couple frameworks for building multi-agent systems

49:39at Microsoft client-side frameworks.

49:41We have AutoGen, which is our research framework focused

49:43mainly on multi-agent systems and we have Semantic Kernel,

49:46which is our AI application,

49:48our production-ready stable framework.

49:49So all the examples I showed here are in Semantic Kernel.

49:52But what we're doing is we're actually converging those

49:54two frameworks.

49:56And we've already done a bunch of the work

49:57to converge them kind of at that lower level

50:00and today actually the runtime for multi-agent systems

50:03in Semantic Kernel is actually the same runtime that's

50:06in AutoGen.

50:06So we've done that work under the covers

50:08and with the next versions of these libraries we're going

50:10to converge the API surface as well.

50:12So we'll just have one of these pretty soon

50:14and if you're starting your projects today,

50:16recommendation, use Semantic Kernel.

50:17That's our production-ready supported framework

50:20and we're bringing all the stuff from AutoGen into there

50:22and we'll have one place

50:23where everyone can build their multi-agent systems

50:26and orchestrate them from the client.

50:28[ Applause ]

50:28So you guys have been waiting, right?

50:32Thanks, Rory.

50:33Yep. It's been a long time coming but we're really happy

50:37with the work the AutoGen team is doing and we're happy

50:38to bring that in to Semantic Kernel

50:42and have a single library that we can use.

50:44So what does it mean though?

50:45So I talked about having a single runtime between the two.

50:51Well, what it means is we can take all those patterns

50:52that you may have been using already in AutoGen

50:54and we can now run them in Semantic Kernel.

50:57So we have a bunch of patterns that are kind of built on rails

50:59within Semantic Kernel.

50:59You can build your own patterns

51:00but these are ones that we supply.

51:03You know, fairly obvious what some of these do.

51:04Sequential, you have one agent talk to another agent

51:07and the results of one agent goes to the next agent

51:08and the next agent and you connect them together

51:10and they go in sequence.

51:11Sequential.

51:12Concurrent, that's what you just saw.

51:13The agents working in parallel

51:16and then the result comes back to, you know,

51:20once they've all completed.

51:21Handoff, a little bit interesting.

51:22You have like a shared context and so kind of the brain

51:24of the agent gets handed off between different agents.

51:26This is good when you kind of have a sequential flow

51:29but you want to have the same state between them.

51:32Group chat, this is a very powerful one to kind of sprinkle

51:34in your more complex processes

51:36where you have two agents talk back and forth.

51:38This is kind of what Salman saw, was showing us

51:41where you have these agents talking back

51:42and forth and iterating.

51:43And often you can do like an author critic pattern

51:45or a worker reviewer pattern where one agent comes

51:48up with some results and then you have a critic gives it

51:50feedback and you have these agents go, you know, two, three,

51:53ten, 20 iterations

51:55and significantly improve the results over time

51:57that you get out of them.

51:59And then two other more complex flows, Magentic, which I'll talk

52:02about in a second, and then workflow processes.

52:04So this is using our process framework directly

52:06to build custom workflows.

52:09So let's take a look at what that looks like.

52:11Let's dive into Magentic specifically.

52:14So Salman talked about Magentic.

52:16Magentic One is a pattern that was developed

52:18by Microsoft Research, by the AutoGen team.

52:21And really what it is, as Salman was mentioning,

52:23was it's a pattern where your coordinator agent gets a task

52:28and it generates a task ledger out of that.

52:30And it works its way through the task ledger and it checks to see

52:35if the agents it's working

52:36with is making progress towards completing that goal.

52:38And if it doesn't, it'll go back and it'll change the plan

52:41and it'll try something else.

52:43So you end up with these kind of self-healing processes.

52:45And what I want to show you, actually,

52:48Salman showed you a demo of that working.

52:50I'm going to show you guys a demo of it not working,

52:53which I think is really interesting.

52:54So this is a recording I did.

52:57This is an earlier.NET version

52:59of Magentic One running on Semantic Kernel.

53:03And what's really interesting here is you can see how these

53:08systems are made.

53:09We have these patterns that you can plug into.

53:12So I'm creating an agent.

53:12This one's got a coder agent and a researcher agent

53:15and an orchestrator agent.

53:16I think there's a few more agents in here as well.

53:18And we create this agent team.

53:20And then we register that team together.

53:21We put all those agents together.

53:23So we have our coordinator and then we have each

53:24of these sub-agents that are used by the coordinator agent.

53:27And so I'm going to run this.

53:29And this instance is where I ran it, I'm asking kind

53:32of a more complex question.

53:34Calculate the distribution

53:35of the top 20 largest cities in the USA.

53:38And we initiate that task.

53:39And so now what it's doing is what we saw in that flowchart.

53:41It's building that plan.

53:43It's figuring out.

53:44It's analyzed the task.

53:45It's kind of building a plan out of it.

53:47You'll see a couple responses going back

53:48and forth here with token usage.

53:50And then now that it's generated a plan,

53:52it's going to pick the agents it wants to use

53:54to accomplish that plan.

53:56And so it's now built this plan where it needs to search

53:57and retrieve the latest data from the web.

54:00It needs to do some math on that.

54:01And then it's going to generate a chart at the end.

54:04But what the Magentic system didn't know is

54:06that I didn't configure one of the agents properly.

54:09So there's agents here.

54:10There's an agent going up to the web searching for data.

54:13There's now a coding agent that's figuring

54:16out the distribution of the population of the cities.

54:20But the agent that is supposed to generate the chart at the end

54:24of it, the visualization,

54:25I actually didn't configure properly

54:27when I was recording this demo.

54:28So you'll see here it'll come back and it'll say, you know,

54:31please create a visualization

54:32for this population distribution.

54:34And it says, ah, I can't do it.

54:36And that was my fault.

54:36That was user error.

54:37I didn't configure the agent properly.

54:40And so it'll go back to the orchestrator agent

54:42and the orchestrator agent will try again.

54:44It'll say, you know, please,

54:46please can you create this visualization.

54:48It sends this back.

54:49It evaluates its plan.

54:50It sends it back.

54:52And our illustrator agent again says, hey, I can't do it.

54:54I can't produce the visual representation.

54:55I just keep getting an error code.

54:57I don't know what to do to fix this.

54:59So it's now tried twice.

55:01And so now it goes back to the orchestrator agent.

55:02The orchestrator says, okay, this agent is not working.

55:05I'm going to try a different plan.

55:07And so it generates a new plan here.

55:08And we'll see, hopefully in a second --

55:11well, I know what's going to happen

55:12because this is a recording,

55:13is that it actually asks a different agent

55:16to generate an ASCII visualization.

55:18So it didn't generate a chart.

55:19You couldn't do it with an image, but it was able

55:21to figure something out.

55:22[ Applause ]

55:23So that's a really powerful way.

55:25And I think it illustrates really nicely these cases

55:27where we have this LLM-driven flow.

55:29And of course, you want this often.

55:30You want this kind of creativity to come from your agent systems.

55:34But of course, sometimes you don't.

55:36Sometimes you want a process-driven flow.

55:40And so I will touch on that real quick.

55:41I know we're running out of time.

55:43So I'm just going to flip over actually

55:44to number two here real quick.

55:48So this is actually the sample that Scott Hanselman and I wrote

55:52for his session yesterday, where he has a process

55:56that is post-processing his podcasts.

56:03So if we look at this process here, very simple process,

56:09not as complex as the one Salman showed,

56:11but it's kind of going through.

56:12And one of the nice things when you're building stuff

56:14in process framework in Semantic Kernel is you can have some

56:17of these steps in the process be --

56:20What we have on the right here is our process going step

56:24after step.

56:26And if we run this -- I'll just do a.NET run here real quick.

56:30Oops.

56:33I found.NET run.

56:37Oh my goodness.

56:38Okay.

56:39So if you run this, it's going to go and it's going

56:40to post-process the podcast the same

56:42as we saw kind of yesterday.

56:44And one of the nice things with building a process framework

56:46with Semantic Kernel, there's cases where you need agents,

56:48there's cases where you need AI.

56:50So if we look and it's cases where you just want to run code.

56:52So you can see here each of these steps, we have some

56:54of them are agents, some of them are just services that we have.

56:59So we talk to our transcription service, which isn't an agent,

57:01it's just a service that we have available.

57:03And we're able to run all those things in code

57:06and have these very kind of complex business processes

57:08that combine agentic steps and non-agentic steps.

57:12And so we can see when this is running.

57:14The other thing I want to show that Scott didn't get

57:17to show you is being able

57:19to see telemetry coming out of the system.

57:22So if I do a search here, actually I'll just do the ones

57:24that are already loaded since the process is still going,

57:26we can hook up our process framework

57:28to Application Insights and if I click on this, we can see all

57:34of our agents running.

57:35This is one I did earlier today.

57:38We can see our timeline of agents running and we can see,

57:40okay, our processing or getting the transcript,

57:42that took a long time.

57:44Then our generate our nodes took a little bit less time.

57:46We can drill in and we can see all the data that is coming off

57:48of our agent system here in Application Insights.

57:51Okay, one last thing I want to show real quick.

57:55I don't know if I have time to show the whole thing,

57:57but we also have A2A support within Semantic Kernel.

58:00This is brand new and actually we haven't pushed it

58:02to main yet.

58:03We were kind of waiting for some things.

58:04I don't know if you guys saw,

58:05A2A actually did a big update today

58:07if you're following it closely, so we're going to fix that

58:09and we'll get it up into main.

58:11But you can now build A2A servers within Semantic Kernel

58:14and you can connect to A2A agents

58:16from Semantic Kernel as well.

58:18So that's a sample we have here.

58:19I don't think I have time to show it.

58:20We're kind of running out of time.

58:22But very easy to create your agents with Semantic Kernel.

58:28Your A2A endpoints, you can just use --

58:32if you're using ASP.NET Core, if you're familiar with that,

58:34you can map your A2A endpoint

58:36and now you can host those directly in Semantic Kernel.

58:38And of course, you can connect to them very easily as well

58:41by just specifying the endpoint.

58:44And we will go and retrieve -- where do I do that?

58:47Here somewhere, I think it's in here.

58:49And then we will go and retrieve the endpoints

58:52for that A2A server and then you can talk to them directly.

58:55Okay, I'm going to go back over to slides and we're going

58:57to wrap up real quick.

59:00Okay, let's do this.

59:02Okay, I do want to talk about a customer, ASUS,

59:05kindly provided us a case study where they've been able

59:08to build an assistant using Azure Open AI service and PTUs

59:14to provision their assistant

59:15and then now their next generation assistant they're

59:17working on uses Semantic Kernel and Agent Service

59:20and actually voice agents to create an interactive flow.

59:25Ooh, it's playing sound.

59:26Computer Voice: Gaming graphics cards

59:27that will suit your needs.

59:28Let me know if you'd like more details or have any questions.

59:32SHAWN HENRY: So you can see they're able to kind

59:33of connect these things together.

59:34They have a multi-agent system.

59:36You can see the diagram here on the left.

59:37One thing that I think is really interesting, too, that they did

59:40with their MCP servers is they put them behind API management.

59:43This is a great way if you're using these MCP servers

59:45and you want to control which agents

59:47and which processes have control of them, put them behind APM.

59:52Okay, wrapping up.

59:53We talked about single agents in the Agent Service.

59:55We talked about single agents, Semantic Kernel.

59:58We talked about multi-agents in the service.

1:00:00We talked about orchestrating them with Semantic Kernel.

1:00:02And we talked about the difference

1:00:03between process-driven flow, directed flow, workflows,

1:00:06and LLM, non-deterministic flow.

1:00:09And we had a demo fail.

1:00:11That was our non-deterministic flow.

1:00:13We had a connectivity fail.

1:00:15I think all our demos worked.

1:00:16I think we were pretty good there.

1:00:18And so to bring it all together, we saw Azure AI Foundry

1:00:23as a service to build your agents.

1:00:24You can select different models.

1:00:26You can select different tools.

1:00:27You can connect to different resources.

1:00:29You can store things in CosmosDB.

1:00:31You can use Azure AI Search.

1:00:34You can use grounding with Bing, which we showed.

1:00:37And then you can use Semantic Kernel to connect agents

1:00:40into the open agentic web with external API support,

1:00:43MCP servers, A2A servers, and then also connect

1:00:47into M365 ecosystem through the M365 Agent SDK, which allows you

1:00:51to now get your agents hosted from Foundry into Teams,

1:00:55into M365 Copilot, into Messenger,

1:00:58into Twilio, all these places.

1:00:59Okay, I'm about to be hooked off stage.

1:01:01Thank you, everyone.

1:01:02A bunch more sessions available.

1:01:04Thank you for your time.

1:01:05SALMAN QUAZI: Thank you.

1:01:06SHAWN HENRY: And one last thing.

1:01:08All the demos for today are available --

1:01:10oh, they kicked us off.

1:01:11There's a QR code.

1:01:12Let me see if I can go back.

1:01:13No, they're not going to let me.

1:01:14All right.

1:01:15Go to the Build site where you can see our feedback,

1:01:20and you can see links to our source code there.

1:01:22[ Music ]