Full transcript
0:00Hello everyone and welcome. Thank you
0:02all so much for joining us today for
0:04this live stream event. My name is
0:06Alexia and I am the program manager of
0:08the Microsoft Reactor Toronto. I will be
0:11sharing session resources with you in
0:13the chat. But before we begin, I'd like
0:15to quickly review two items, our code of
0:18conduct and event guidelines.
0:21First, please take a moment to review
0:23our code of conduct. Microsoft Reactor
0:25seeks to provide a respectful
0:27environment for both our audience and
0:29presenters. We encourage engagement in
0:31the chat, but please be mindful of your
0:33commentary. Remain professional and on
0:36topic.
0:39And secondly, our event guidelines. This
0:41session is being recorded and will be
0:43available on demand through the
0:44Microsoft Reactor YouTube channel in
0:46about 24 to 48 hours. I will be sharing
0:49the link in the chat for our on demand
0:51content. And if you have not been on a
0:54live stream to YouTube before, please
0:56note that you must create an account on
0:58YouTube in order to have access and
1:00interact in the chat. You can set that
1:02up now. And if you're unable to use the
1:04chat but have questions, feel free to
1:06reach out to us through social media or
1:08on our website,
1:12which brings us to today's session. I'm
1:13going to bring in our speaker here for
1:15today, Allesandro. Hi, Allesandro.
1:17Welcome. Welcome. Hi, Alexia. Hi,
1:20everyone.
1:22Yeah, we're very excited to have you
1:24here um on the reactor for the first
1:26time for your first session with us.
1:29Yeah.
1:34Well, thank you so much for being here
1:37with us today and uh without further
1:39ado, take it away.
1:42Thank you again and thank you everyone.
1:44So, good morning, good afternoon, good
1:45evening from you know wherever you you
1:47are uh viewing this session from. Uh I'm
1:51Alessandro.
1:52I'm a GBB specialist. So I'm part of the
1:56global black belt team of specialists in
2:00the digital and application innovation
2:02solution area. Um 10 years in Microsoft.
2:06I recently moved to this uh fantastic
2:09team and I'm based in the Netherlands.
2:12uh so I my let's say top priority is to
2:16make customer uh my customer make uh
2:20them successful okay uh not just in the
2:23Netherland market but in the inia so um
2:28today's session
2:30is about uh um AI agents we have a lot
2:34of customers asking for uh how to
2:38implement aic solutions what are agents
2:42how they can interact with each other.
2:44So sometimes very basic question,
2:47sometimes very complex question,
2:49sometimes complex questions about how to
2:52build a genetic solution. So today's
2:54session is about uh exploring the
2:57essentials of uh AI multi- aent systems
3:00in this case. Uh so we let's say we
3:03start from the very beginning. So we
3:05start from the definition of agents
3:08um and the key interaction patterns.
3:11Okay. So, uh we'll be focusing on the
3:14interaction, the strategies. Uh we'll
3:18discuss two use case or two case studies
3:22coming from a real world examples,
3:25marketing scenario, support center
3:26scenario. We'll we'll deep dive into
3:28into these ones and uh uh we'll actually
3:32deep dive into the code implementation
3:35of the support center scenario. So
3:41um when Genai and Open AAI came out more
3:46or less two years ago uh we started
3:50again uh talking about chat bots and uh
3:54building chat experiences. Um so we
3:59started redefining the chatbot
4:01experience by adding for example the
4:04rack pattern the retrieval augmentation
4:06generation um to actually fine-tune
4:10models uh LLM's models with the customer
4:13data and then following this um
4:16evolutions we had copilots and uh and
4:20actually we have copilots we have a very
4:23wide um copilot offering we have mic
4:26Microsoft 365 copilot, we have GitHub
4:29copilot, we have Azure copilot, you name
4:31it. Uh but uh um let's say all these
4:36type of solutions uh have been what we
4:38call uh reactive solutions, okay? Or ask
4:42solutions. So in the ask approach, there
4:45is a user that is asking for something
4:48and is getting something as a result. uh
4:51what is coming as next step and what we
4:55uh let's say we do see uh as a trend
4:58also from a customer's perspective
5:01across IMIA not just limited to IMIA uh
5:04are autonomous autonomous solutions um
5:07that are indeed able to do things on our
5:10behalf. So instead of having as I said
5:14an ask approach let's move on to the do
5:17approach. Okay. So instead of having a
5:20user that just asks something and get
5:22something, uh we'll ask our agents to do
5:27stuff on our behalf.
5:30So uh let's say let's start from the
5:34very uh basic definition of of agents.
5:38Okay.
5:40Um so what is an agent? Uh an agent is a
5:44local software component. So let let's
5:47focus on this. It's a local software
5:49component u a component in a generic
5:52system that has uh several additional
5:56components uh it is built upon. Okay. So
5:59as you can see in the slide we have an
6:01agent in the center uh but uh it
6:05consists of many things. It has tools
6:08states strategies and goals. Of course,
6:12it needs to address a specific goal or a
6:15set of goals, but it has several let's
6:18say capabilities or features in order to
6:20accomplish this goal or these goals. So,
6:24as you can see, tools uh tools um
6:27basically they provide different
6:29services, okay? Like accessing your your
6:32own data, recall the rag pattern. So uh
6:36adding data, additional data, customer
6:40data, your own data on top of these
6:42agents. Agents they have access to LLM
6:46of any kind. Even even not just LLMs but
6:50also SLMs. So not just large language
6:52models but also uh small language
6:54models. Uh they have access to internet
6:58so publicly available information. They
7:00have access to GitHub whatever. Okay.
7:04And then they have a state. And in this
7:07case we can either have um stateless or
7:10stateful agents more or less like
7:13microservices. Okay. Stateless or
7:15stateful uh microservices.
7:18Agents can then hold a memory of a
7:20specific section. Uh they remember what
7:24they did in the past and they have
7:26context of their environment. And then
7:29we have strategies. Uh now think about
7:32strategy as um a way that you define
7:38uh it's like a a type of communication
7:42between agent. So you define a strategy
7:44you define how these agents can
7:46communicate with each other and actually
7:48a strategy can be anything. uh a
7:51strategy can be just a very simple
7:53algorithm
7:55uh that provides deterministic results
7:57or you can rely on some prompt
8:01engineering techniques like these ones.
8:03So chain of thoughts, reflections,
8:05self-critic planning uh chain of
8:08thoughts for example is a a prompt
8:11engineering technique that uh enable
8:14complex reasoning capabilities uh
8:16through steps. If you uh recall the the
8:19latest open AI model so uh open a01
8:23openai actually implements this channel
8:26of thoughts. So there is a additional
8:29reasoning capability on top of that and
8:31you can actually author your agents so
8:33they can have this sort of reasoning
8:35capability but you you can actually
8:38create and define new strategies. You
8:42can have reflection, self-critic, you
8:44can have planning. So your agent can
8:47actually build a plan in order to
8:49accomplish that goal.
8:51And we actually see that in our support
8:54center scenario, we do use a planning to
8:58perform specific operations.
9:00So this is more or less what's behind an
9:05agent, a single agent local software
9:07component.
9:09It has also actions of course that uh it
9:13needs to perform in order to accomplish
9:16those goals
9:18and uh
9:22they can interact and communicate with
9:25other agents. So we started from the
9:27very basic definition a local software
9:29component but an agent can collaborate
9:32with other agents to accomplish a goal.
9:36Okay. So again they have independent
9:38memories uh they have independent
9:40reasoning capabilities they can work
9:42each other independently they have local
9:45goals but the combination of those goals
9:48is what they are trying to achieve to
9:50accomplish the overall goal. So if we
9:53recall the uh the previous slide we have
9:56a user that is asking for a very complex
9:59task. You can have a bunch of agents
10:02collaborating with each other. Each of
10:04them have they their own specific set of
10:06goals and they can actually accomplish
10:09together the what what their user is
10:11asking. Okay. So the original goal, so
10:15everything is so cool, great. But uh uh
10:19again, why agents? Um
10:22what what's in it for me? Okay, what's
10:25the real what are the real benefits of
10:27using agents? Uh first pillar, the need
10:31of uh building compounded system. So
10:35complex systems to get better
10:38performance. And actually um these uh
10:42these requirements came out when first
10:44LLMs like GPT were not as powerful as
10:48today's models and a very good prompt
10:51engineering was was needed to make AI
10:54models very effective. So we had the
10:57situation where a uh a user needs to to
11:00have a very to provide a very big uh
11:03complex prompt okay
11:06for a very specific ask okay but in this
11:10case leveraging on agentic system um
11:14let's say we could create an agentic
11:17solution built on top of different AI
11:19models to provide better results. So
11:22again instead of working on a single
11:25problem let's decompose this problem
11:28into smaller problems each of these
11:30problems can be handled by one agent or
11:34a set of agents and uh they can then
11:39perform and provide better results. So
11:42I'm going to provide you an example
11:44here. Um let's let's suppose that the
11:47the user is asking for uh writing a page
11:52with compliant content. So instead of
11:55building a very big prompt around this,
11:58I can have do two agents. I can have a
12:00writer agent and a reviewer agent. So
12:03once the first agent writes a page, the
12:06reviewer agent uh let's say the content
12:10gets reviewed by the second agent. So
12:13each agent is tied to a specific prompt
12:16with a very limited scope
12:19and you have much better control this
12:21way.
12:23Um the second pillar is to provide
12:26abstraction and this is more linked to
12:28the developer experience.
12:32Um depending on the agentic framework
12:35that you're going to use and we we we
12:37talk a little bit about these
12:39frameworks. So depending on the the
12:41quality of these agentic frameworks they
12:44can actually abstract the way these
12:46agents communicate with each other
12:49leaving you as developer just the let's
12:51say the joy of uh building the logic of
12:55your agents. So instead of
12:58focusing on the inner complexity, you
13:00just focus on modeling your own agent
13:03and the the agentic framework takes care
13:06of uh of the rest.
13:08And uh
13:11the last pillar is the most important
13:13one.
13:14You can easily map business processes,
13:18business roles covered by different
13:21personas into agentic workflows and the
13:24other way around. So you can start from
13:27a business process, you identify
13:29personas, you identify different roles
13:33and then you create your agentic
13:35workflow. Or you can actually do the
13:38other way around. given an agentic
13:40workflow for a specific business
13:42process. You can actually re rebuild
13:45your business process, okay, out of it.
13:48And this is very important when we need
13:50to decide whether to go or not with a um
13:54agentic solution. If we have a clear
13:57business process, we can clearly
14:00identify different roles, different
14:03personas
14:04and we are also able to decide the level
14:07of autonomy of our agents within the
14:10agentic solutions. Okay, it's good. We
14:13can go with agentic solution or we can
14:15try with something else. Um
14:20just to let's say two words about uh the
14:24agentic architectures.
14:26Um, of course we have we can just work
14:29with a single agent. In that case, we
14:31have a single agent architecture. If we
14:33recall that the chatbot slide,
14:37a user is interacting with a bot. That
14:39kind of experience is a single agent
14:42architecture, right? Or we can work with
14:46multi- aent architectures uh vertical
14:49and horizontal. that these categories
14:51are let's say they represent two ends of
14:55a spectrum where the most existing
14:58architecture falls somewhere between
15:00these these two extremes. So you can
15:02have mixed architectures but basically a
15:05vertical architecture um
15:08identifies one agent like a leader agent
15:12or a master agent or a proxy agent you
15:16name it really but it's a leader and it
15:20has other agents reporting directly to
15:23to him. Okay. And depending on the
15:25architecture, reporting agents um may
15:28communicate exclusively with the lead
15:30agent or directly back to the user. And
15:33we'll see an example of this. Um
15:38and then we have the horizontal
15:39architecture. Uh horizontal architecture
15:43uh it's it's a let's say um there is no
15:47different priority between agents. All
15:49agents are treated as equals and they
15:52they are part of one group discussion.
15:55Okay. Um so the communication between
15:58agents occur in the same thread each and
16:01every agent can see other messages. Um
16:05horizontal architectures in this way are
16:08generally used for tasks uh where
16:10collaboration where feedback and group
16:13discussions are key to the overall
16:15success of the task of the original
16:17task. So again depending on the use case
16:21and their complexity you may want to
16:23fall into a pure vertical, pure
16:26horizontal or just some something in in
16:29between.
16:32So
16:34again uh we see that different organi
16:38organizations are at different stages of
16:42uh their AI transformation. Some of of
16:45them are advanced, okay, and they are
16:48building the next generation of agents
16:50with autonomous capabilities, fully
16:52autonomous solutions. Others are still
16:55uh building out their AI agents to focus
16:58on retrieval and task task experiences.
17:01So bringing their own knowledge and
17:03automations into their agent experience.
17:06So depending on
17:09um the solution complexity you can have
17:12a very simple agentic solution just a
17:15pure retrieval okay there's a user who
17:17is asking for something and you have
17:20your agent uh retrieving the information
17:24out of um grounded data internet or your
17:29own data whatever up to fully autonomous
17:33agentic solutions.
17:36So a set of agents they operate
17:38independently they can dynamically plan
17:41they can orchestrate other agents. If
17:44you recall the horizontal architecture
17:46this is an architecture that uh can see
17:49you know many agents collaboratively
17:52working with each other. They can
17:54orchestrate okay the work with each
17:56other. They can learn and they can
17:58escalate. This is a more complex
18:00solution.
18:03So different complexities, different
18:06architectures,
18:08different use cases, right? Um very
18:12simple use cases like it help desk
18:15agent, how do I connect to the corporate
18:17network? So a user is asking for a very
18:20specific question is a very simple
18:21question. It's simple to address up to a
18:25customer support agent. So in this case
18:28you identify different personas covering
18:31the customer support scenario or like a
18:35call center scenario and you may have
18:37different agents um
18:41uh that need to collaborate to perform a
18:45specific set of tasks and uh to
18:47accomplish what the user is asking. So
18:49again we have many different use cases.
18:53Maybe for some of these an agent
18:55solution is not needed. It really
18:57depends on on on the on the use case. If
19:00we recall the why agent slide, think
19:04about the business process. If you are
19:07able to clearly articulate the business
19:09process, you are able to identify the
19:12personas, the roles and you may decide
19:15to go with a fully autonomous solution.
19:18Okay, maybe an agentic solution is um is
19:21a good fit for you. Okay. So now we'll
19:25focus on we'll be focusing on two case
19:28studies. The marketing case study. It
19:31comes from a real world example. So we
19:34built this use case um this solution for
19:38for a customer and uh there is another
19:42one that is called uh support center.
19:45We'll see two recordings for these two
19:48case studies and we'll deep dive into
19:50code implementation for the support
19:52center. So let's go first with the u the
19:56marketing.
19:58Again we started from a very basic
20:01statement the customer statement. We
20:04want to get a copilot to help us write
20:07marketing campaign.
20:09Um
20:11you may noticed you may know that uh
20:14behind building a marketing campaign
20:17there's a lot of effort there's a lot of
20:20uh people being involved but we started
20:23from a very basic role because for sure
20:26behind a marketing campaign we have a
20:28writer right a writer that actually
20:31writes
20:32uh the description of a marketing
20:34campaign. Okay, by the way, this is the
20:39uh the customer's background. Big
20:41customer, more than 500 brands,
20:43different territories, different brands,
20:45globally distributed marketing teams.
20:47So, a very big customer. Okay. So,
20:50that's what we did. We started from the
20:52writer role and then we identified
20:56uh other personas other roles because
20:59okay for a marketing scenario we have
21:02the writer that writes the marketing
21:05campaign okay the description but we
21:07have also the community manager that
21:10writes a post that is uh you know being
21:13shared to social networks like social
21:16media posts on X or LinkedIn but then we
21:20have The auditor the auditor needs to
21:24check whether the the brighter is uh
21:27let's say providing something that is
21:29compliant with internal regulations. The
21:32same for the community manager. And then
21:33we have the sales analyst.
21:36For example, the s the sales analyst
21:39may predict whether that marketing
21:42campaign will provide will uh let's say
21:44will generate some sort of revenues or
21:46not. And you name it, we have different
21:49rules here. You we have the the graphic
21:51designer for example that provides the
21:54the image or the multimedia content for
21:56the marketing campaign. So we started
21:58from uh these roles and then we said
22:02okay let's let's model them using
22:05agents.
22:07So let's model our agents
22:11specialized AI agents. Each of them are
22:14let's say uh again specialized for doing
22:17a specific set of tasks and let's come
22:21up with with a solution.
22:24So this is the uh marketing sample. On
22:27the left we have the the the front end
22:30the user is interacting with. The user
22:32is uh specifically asking for uh a
22:36marketing campaign. In this case is
22:38asking for a 25% discount on our uh blue
22:43microphones called whale. And on the
22:47right we have our agentic solution. So
22:50we have our um back end of agents again
22:55each of them are um working on a
22:59specific set of tasks. So everything
23:01starts from uh the user. The user is
23:04asking for something. This message goes
23:06to uh any agent. So any agent receive
23:12this user ask
23:14but then
23:17each and every agent is uh providing his
23:21own response to the user and that's what
23:24the the framework solves. We we're going
23:27to talk a little bit about agents uh
23:29agent agentic frameworks. Uh but in this
23:32case we used a framework that allows us
23:35to um independently update our front
23:40end. So as you can see the writer is
23:42providing a a a response. Okay. So a
23:46description of the marketing the
23:48marketing sample
23:50then the community manager the auditor
23:53uh the graphic designer. For example,
23:54here the graphic designer provides an
23:56image. Um,
23:59so let's let's see a live recording for
24:02this. So
24:05here we have a user
24:08that is asking for
24:10a new campaign
24:14on a yellow um remote controller.
24:20to make it appealing.
24:22They will get 25% discount
24:25the entire next week. So this is what
24:28the user is asking. Underneath we have
24:31our set of agents specialized AI agents.
24:34They are developed in different ways.
24:36Okay.
24:38And uh
24:40they can independently update our front
24:43end. As you can see here, the writer is
24:46the first one that replies to the user
24:49and it says, "Are you tired of always
24:52misplacing your remote and having to
24:54search high and low for it? Say hello to
24:57our new yellow remote controller." So,
24:59here we have the writer agent that uh is
25:03let's say is instructor to pro to
25:06provide just a um description of the
25:09marketing campaign. But then we have the
25:12auditor. Auditor says, "Hey dear writer,
25:16the discount you mentioned in your
25:17message is too large. Our company policy
25:20does not allows us to give discounts
25:22larger than 10%." So in this case, the
25:25discount
25:27should be more than 10%. And
25:30unfortunately the writer u provided 25%
25:34because it's basically what the user
25:36asked. But the auditor has been
25:38instructed to be compliant with a uh
25:41specific set of guidelines. In this
25:43case, this is for demo purposes. But one
25:46of these policies u was the fact that
25:49the discount wouldn't be more than 10%.
25:52Okay. So the auditor comes in and say,
25:55"Dear writer, please provide a different
25:58uh description because this is not
26:00compliant." And then we have the
26:02community manager.
26:04Community manager provides a social
26:06media post. So, as you can see, uh some
26:10emojis and hashtag are included into the
26:14message. So, say goodbye to the days of
26:16misplacing your remote. Okay? And so on
26:19and so forth. And also in this case, the
26:22auditor comes in and say, "Dear
26:24community manager, the discount you
26:26mentioned is above the 10% discount."
26:30And again, the the message is too large.
26:32And eventually the user can uh directly
26:35in interact with a specific agent if he
26:38wants to yeah uh
26:42say something else. Okay. So in this
26:44case okay the graphic uh designer
26:47provided a message on the right but here
26:50the the user it says hey community
26:51manager can you write the tweet post in
26:54hy so in a different language.
26:58So in this case the user is interacting
27:00directly with the community manager
27:03agent
27:05and uh in this case again
27:09the community manager would be the only
27:11agent that would reply to the user.
27:17Okay. So the post has been written in
27:20into a different uh uh language. So
27:25this is our first case study. As you may
27:28have noticed, this is a horizontal
27:30architecture. No explicit um let's say
27:35uh structure predefined structure
27:37between agent. Uh they are treated all
27:40treated as equals. Okay. And then we
27:43have the support center. Support centers
27:45is a completely different use case. A
27:48vertical architecture. Again we started
27:50from a very basic uh definition here. We
27:54want to enhance the support center
27:57productivity
27:59and of course we have many benefits for
28:02this uh efficient task delegation.
28:06Um because again you can set up your
28:09agentic solution with different
28:12specialized AI agents. Each agent is
28:15working on a specific task or set of
28:17tasks. So you can you can actually as a
28:19human you can actually delegate
28:21uh a set of your tasks to a specific
28:24agent. Okay. And this leads to a better
28:28customer experience because uh if you
28:31think about a call center scenario, a
28:34user doesn't have to wait for a human
28:37agent to to to come in and reply to the
28:39user because we have our agents working
28:43on on on our behalf. And of course
28:46targeted expertise, scal scalability and
28:48flexibility. This actually depends on
28:51the agentic solution and how uh we need
28:54it to to scale and to be highly reliable
28:58uh available so on and so forth. Okay.
29:00So the outcome was again as in the
29:03marketing sample a set of AIdriven
29:05assistants with a specific role with a
29:08specific domain knowledge which gets
29:10invoked for a particular task. This is
29:14the um the architecture for the support
29:18center. Again, everything starts from a
29:21user's ask. So the user is
29:24actually uh asking for something. I need
29:27support on XY Z. Okay. But in this case
29:31uh this message um doesn't go to all the
29:34the the agents. Okay. the we have a
29:39leader agent, a master agent or in this
29:41case this is called a dispatcher agent
29:43that is responsible to extracting the
29:46intent. So it's an intent extractor
29:49out of u the the user message and the
29:52decides the right agent for the request.
29:55So according to the user request it may
29:58redirect this message to the right agent
30:02to the right actually sub agent on the
30:05right. So we have the customer
30:08information. Customer information in
30:10this example is uh uh it get gets access
30:14to the Cosmos DB um
30:18um to actually a storage layer can be
30:20anything but in this case it's a Cosmos
30:22DB. So whenever the user is asking for
30:25okay I need to update uh my customer
30:28information I need to delete I need to
30:30add something on my customer profile.
30:33Okay, this is something that the
30:35customer info can address because the
30:37customer info has access to uh the
30:40storage layer and it can access the user
30:44users data.
30:47Um, if you think about an energy company
30:50here, um, maybe the user wants to get
30:54access to the last, I don't know, uh,
30:56six invoices or it wants to get the the
31:00amount of the total expenses for the
31:02last six months. In this case, okay,
31:05there is an invoice agent that is
31:07grounded with invoices for that specific
31:11user. Okay. So whenever the user is
31:14asking for something specific for his
31:16own invoices, we have the invoice agent
31:18here. Q&A. Q&A is specific for um
31:24questions around the company. So as a
31:27user, I want to ask something about the
31:29company. Okay. We have a Q&A agent that
31:33uh addresses the chat with your data ra
31:36pattern at the end of the day.
31:39and then a very basic conversation
31:41agent. So for any requests that uh uh
31:47don't fall into the the the the other
31:49path we have a conversation agent we see
31:52which is actually a very basic uh agent
31:55that works with LLM.
32:00Once um a sub agent provides a response,
32:05this response goes to the dispatcher and
32:07then the dispatcher can actually uh
32:09process this record response before
32:12going back to the to the user. Okay,
32:14let's see a recording also for um this
32:18example here. I'm impersonating a user.
32:29So, I'm asking in this case
32:34a very basic question. I'm moving to
32:36another city. I need to change my
32:38address. Can you help me with that?
32:40Okay.
32:43So, we get the first response uh from
32:47the dispatcher. As you can see on the
32:49left, the very first message come from
32:52the dispatcher D. The user request has
32:55been dispatched to the customer info
32:57agent. And then the customary info comes
33:00in CI. Okay, I'm working on the user's
33:03request the dispatcher. So it processed
33:06the the message, it extracted the intent
33:09and then it decided to forward to the
33:11customer info agent.
33:14So let's move on.
33:19Okay, sure. I can help with that. Could
33:21you please provide me uh with your new
33:24address? Okay, sure. But I actually want
33:27to check which is my current
33:30information.
33:32So as you can see here as a user I can
33:34uh
33:36engage a conversation with my agentic
33:38solution. Okay sure what is my current
33:40info dispatcher and then again customer
33:43info. Recall that agents have have a
33:46state so they can actually remember
33:50the the chat history. So, okay, the
33:52customer info replies with the user
33:54information.
33:56Okay. Um, so my current address is
33:59address 123, but my new address is my
34:02new address 46 uh 456. Okay. Please
34:06update my address again. Dispatcher CI,
34:10your address has been successfully
34:12updated to my new address. If we go to
34:16the Cosmos DB account, this is the old
34:19address, but the the agent
34:23um had access to to my DB to my specific
34:27item and it actually updated my my my
34:30address. Okay,
34:34so okay, I'm done with this. I don't
34:36need to create a new session. I can keep
34:40uh working in the same session. Okay,
34:42now I need to know a bit more about
34:44Northwind Health Plus. Northwind Health
34:46Plus is a fixious insurance plan, but
34:49here there is another agent that comes
34:52in. There is a Q&A agent because the Q&A
34:55agent is grounded with uh company's
34:58data. In this case, it has access to a
35:00bunch of documents and for example, the
35:03North Twin Health Plan. Okay. And I keep
35:07going. Great. Any additional information
35:09about that? Okay. Yeah. Q&A. Northwind
35:12Health Plus is indeed a comprehensive
35:14group health plan sponsored by blah blah
35:16blah. Okay. Any other plans?
35:20Okay. Please while I look in the
35:21documents,
35:29okay, we have other documents not just
35:31north wind L+ but also Northwind
35:34standard. Okay.
35:37So, and and we can keep going actually.
35:41So, we built these two use cases with
35:45the project or agent. Project or agents
35:48is a Microsoft IP is a net AI agents
35:52framework. Uh is not a production ready
35:55but it's a collection of uh best
35:57practices and patterns. Um and it's an
36:01accelerator. So you can actually use
36:03this to accelerate the adoption of uh
36:06agentic solutions. So you can it
36:08facilitates the creation of agentic
36:10solutions. Uh it is built on top of
36:13orans and semantic kernel.
36:16Um and uh if you go to the to the GitHub
36:20repo here, we provided three use cases.
36:24Two of these are the marketing, the
36:26support center that I just uh showed
36:28you, but there is also a dev team
36:30targeting a more more technical
36:32audience. It allows you to interact with
36:35GitHub and uh it simulates um like a
36:39development team. So, please have a look
36:42at this because it's it's very
36:44interesting. So, it's it's built on top
36:47of Orleans. For those of you who who uh
36:51don't know Orleans, Orleans is a let's
36:55say mostly referred to as the
36:58distributed.net. So it's a crossplatform
37:01crossplatform framework for building
37:03robust scalable and distributed apps.
37:06Okay, distributed apps of course because
37:09uh they can span more than a single
37:11process. Um and uh we have different
37:15primitives. I'm just want to recall them
37:18here uh because it it helps me to uh go
37:22deep in the code uh for the next 10
37:25minutes. So we have grains, we have
37:28cloos, we have clusters, a grain is a
37:30virtual actor.
37:32Um so this is the fundamental building
37:35block in any Orian application. Um so
37:40grants are let's say they basically
37:43uniquely identify an actor and guess
37:46what in this case an actor is a grain
37:52and is an agent okay um and then we have
37:56sillos okay that host one or more grains
38:00uh and silos coordinate uh uh with each
38:03other to distribute work so if you think
38:05about uh uh distributed application
38:08ations. You may have different uh silos
38:11in different clusters and orans which is
38:14a distributed runtime takes care of uh
38:17properly distribute grains across
38:20different uh clusters and then of course
38:22we have clusters which are collections
38:24of silos. Okay. So given this
38:29I want to show you
38:35the code here. So this is the the code
38:38for the support center. We have a front
38:41end and a back end. Whenever a user is
38:45uh getting um is sending a text
38:50uh it uh let's say it calls an API.
38:54Okay. So here we have the interactions
38:57controller and a post method that gets
39:01the the user's uh message. Okay. So here
39:05we get the user ID in the the example
39:08that has been provided in project or
39:10agent the user ID is a is a coded it's
39:12just one one two three four but we also
39:15get the user message. So here what we do
39:18as very first thing is to get access to
39:22a stream. A stream is another orian
39:24primitive. It allows you to uh make the
39:28communication happen between agents. So
39:31it uh allows you to enable the uh
39:34publisher consumer pattern. So whenever
39:38there is an agent that sends something
39:40to a stream there could be an agent or a
39:44set of agent that actually consume
39:46um that message in the stream. Okay. So
39:49in this case we are uh getting access to
39:52a stream
39:54and we are sending something. So in this
39:56case we are sending the user message.
40:02So we are building an event. We are
40:04sending this event onto the stream.
40:07Okay. And the event has an event type.
40:10This case is user chat input. It's
40:13really helpful for us uh because uh
40:18depending on the event type
40:20agents can behave differently. Okay. So
40:24user chat input and then of course the
40:26users user message.
40:29The logic behind agents is here. So we
40:32have the agents folder and we have one
40:34folder for each agent. So at the end of
40:36the day here an agent is a class. Okay.
40:40We have conversation, customer info,
40:42discount, dispatcher, the agents that we
40:45we just uh saw in the in the
40:48presentation, right? uh the invoice, the
40:50Q&A, the signal R here is an agent that
40:53is responsible to update the the user
40:56interface. So whenever we have a message
40:59that needs to be sent to the front end,
41:01there is the a signal agent responsible
41:04for for that. Uh but the other agents
41:07are are well known, right? So we have
41:09dispatcher
41:10and for each and every agent we have a
41:13class representing the the agent and
41:16then a class representing the prompt and
41:19eventually a state. Okay, we can have
41:22either stateless or stateful agent. So
41:25if we go to the dispatcher
41:28again a dispatcher is a class AI agent.
41:32AI agent is a agent. Okay, an agent and
41:37guess what is a grain? So agents are
41:41modeled as grains. So we are leveraging
41:43on orans grains uh concept.
41:47So here if we go back to the dispatcher
41:52every agent has the endle event. So here
41:55you can put your own logic. So
41:59uh
42:01here you can actually put uh the way
42:04this agent would handle these events. So
42:07in this case if you recall we have a we
42:11had a user chat input event. So the user
42:14sent a message. This is a user chat
42:16input message. And here we have uh
42:19different cases. Depending on the event
42:22type the dispatcher can behave
42:24accordingly. So if the user is rec has
42:28reconnected okay we are doing something
42:30but let's go to the user chat input. So
42:34when we get a user message we extract
42:38the intent.
42:40So we extract here the intent. Then here
42:42we have the intent.
42:44We publish an event a notification
42:47event. So if you remember we have a very
42:51first message from the agentic solution
42:53that is the user request uh has been
42:56dispatched to the intent agent. Okay,
42:58this is a notification um a notification
43:01message. When then we once we extracted
43:04the intent, we send dispatcher event.
43:07Okay, so here
43:10we publish an event, a new event. So
43:14it's the dispatcher agent that sends now
43:16an event
43:18with that specific type. Depending on
43:21the type,
43:23we may have the customer info agent uh
43:26working on this request. We may have the
43:28Q&A, we may have uh the conversation
43:31invoice, so on and so forth, right? Um
43:35the dispatcher is built upon a different
43:37choices here.
43:40We provided a basic description for each
43:42and every choice. So dear dispatcher uh
43:46if the customer is asking a question
43:48related to internal contosal knowledge
43:50base, okay, this is a Q&A. So you need
43:53to extract Q&A as intent. Otherwise, if
43:56the customer is asking for a discount,
43:58this is a discount uh this is something
44:00that the discount agent can handle. So,
44:02dear dispatcher, please extract discount
44:05as intent and then send a event to to
44:10our stream. Okay.
44:13So,
44:15if we see other other agents like the
44:18customer info customer info oh uh let me
44:22go back to the dispatcher again. Let's
44:24have a look at the prompt. The
44:26dispatcher prompt is this one. You are a
44:29dispatcher agent working with the
44:30support center. So this is a
44:33specifically tied to the dispatcher
44:35agent. So the dispatcher agent is
44:36working with this prompt. Uh if you
44:39don't know the intent, don't guess.
44:40Instead respond with unknown and
44:43programmatically will take care of this
44:45unknown case. You can choose between the
44:48following intents. Okay, the the choices
44:50that we we just shown and we provided
44:53also an examples and here we have the
44:56the user input. Okay, so please behave
44:59accordingly. But if we go to another
45:02agent customer info for example, the
45:05customer info leverage on the semantic
45:08kernel capabilities. We are leveraging
45:10on the semantic kernel planner to
45:12actually build a plan to interact with
45:15our storage system. And we need a plan
45:18because we need to get access to um to
45:22uh to that database. We need to get
45:25access to that specific uh users item
45:28and depending on the user's message, we
45:30need to add the delete, update or
45:32whatever. Okay. So here we have the
45:35customer agent
45:37that works with the semantic kernel
45:39planner. Okay. So this agentic framework
45:44allows us to uh quickly create our
45:48agentic solution because again at the
45:49end of the day agents are just classes.
45:52We are building uh our agentic solution
45:55on top of orans which is a distributed
45:57runtime and that it takes care of the
46:00replica the state replica. If uh our
46:02agents are stateful uh audience can
46:05actually replicate the state for us. Uh
46:09it takes care of the internal
46:11communication between agents. So the
46:13developer can actually focus on the the
46:16agents uh logic exactly as we did for
46:19for the dispatchers. So the
46:22everything lies into the end of event.
46:26Okay.
46:28So I encourage you to have a look at
46:30this um uh IP. Again this is not uh the
46:35only choice that that we have. Uh we
46:38provide different frameworks for for
46:40different use cases. There is no
46:42framework that fits them all. Okay.
46:44Depending on the use case, depending on
46:46the complexity, you may want to steer
46:48with
46:50um
46:52semantic kernel. You can uh have a look
46:56at the autogen which is a more
46:58experimental project but is going to be
47:02more production ready. You can use this
47:04accelerator. So uh I mean the the choice
47:07is is yours. So I'm having a look at the
47:11the questions here.
47:16Um is a multi- aent approach more
47:19expensive to run and maintain? Um
47:23it actually depends uh but you because
47:27um depend on
47:30the hosting platform. Okay. You may
47:32decide to deploy your agentic solution
47:36into uh hosting platforms like ACA,
47:40Azure container apps or build your
47:42agentic solution on top of a
47:45orchestration platform like AKS. But it
47:47really depends on on the scenario.
47:51Um these frameworks allows us to start
47:54with a very basic uh solution. We can
47:58debug it locally and we may decide to to
48:01go uh to to a yeah a hosting platform.
48:06But again it really depends depend it
48:09but also depends on the users workloads.
48:11We may have 10 1,000 1 million of
48:14customers. So depending on the on on the
48:16users's workload you you may have
48:19different set of customers they are
48:21hosted in different containers. So again
48:23it really it really depends on the uh on
48:26your agentic solutions complexity.
48:44So yeah, if we don't have any other
48:46questions again, thank you so much for
48:48for your time. Thank you so much for
48:50joining this session and um get back to
48:53you Alexia.
48:56Yeah, thank you so much Alessandra for
48:58this session. We can give it another
48:59minute or so for the audience um to see
49:02if they have any questions. In the
49:04meantime, I am sharing a great um
49:08resource or or compilation of resources
49:12that has been curated by um our internal
49:15team to learn more on various topic. I
49:18also shared the um repo that you um had
49:23on the screen as well in the chat. Um,
49:25but if you're in the audience and
49:27looking to learn more, uh, feel free to
49:29scan the QR code on the screen.
49:32And yeah, if you have one more minute,
49:36um, feel free to scan the new QR code on
49:39the screen, which will take you to our
49:41survey. We would love to hear your
49:43feedback on reactor sessions, what you
49:45enjoy, what you enjoyed a little bit
49:48less, or what we can do better. Any
49:50feedback is appreciated. The code for
49:52this session is 24283.
49:56And uh yeah, Landra, thank you again so
49:58much. It looks like everything is pretty
50:00clear and that we do not have any
50:01further questions. So, thank you again
50:04so much for being here with us today and
50:07um looking forward to the next one.
50:09Thank you, Alexia. And thank you
50:11everyone. Have a great day. Thanks
50:13everyone.