Full transcript
0:13May I actually actually ask you how many
0:15of you are already working with the AI
0:17agents? Let's see. Um Oh, okay. Oh,
0:20wonderful. Are you mostly Java
0:22developers here? Are you guys mostly um
0:25who are not not Java? Oh, okay. Oh, so
0:28still quite a few, but do you work have
0:31with Java as well? Are you familiar?
0:33It's just that currently Okay, that's
0:35great because my examples too um mostly
0:37right now Java because this conference
0:40started off more for Java developers.
0:42So, I've been assuming it's more for
0:44Java. So, but yeah. Anything I'm going
0:47to speak to will be applicable for any
0:49Yeah, anything. So, okay. So, the talk
0:52to this This one I sort of treated like
0:54any
0:55sort of an extension from what my talk
0:58from yesterday. I I'm doing two talks
1:00here, which I already did the first one
1:02yesterday. That was more introductory
1:05event streaming event you know
1:08what you call it event-driven approach
1:11for more multi-agentic type of workflow
1:14kind of systems. And I do have to admit
1:16too it's a bit
1:17exploratory at this point um because I
1:20don't feel the current state of the
1:23agent type of usages or development
1:26work, right? That that's being used I
1:28think in real production. It seems to be
1:31more about uh some single agent dev
1:34defining some steps and then you know
1:36you do certain things maybe just uh have
1:38some tool calling in the back and doing
1:40something. Uh don't get me wrong. I
1:41think it's still very good some of these
1:43agentic way of doing things. It The
1:46ideal situation is for the autonomy
1:49autonomous type of scenario automation
1:53essentially um and leveraging large
1:55language models, but what we're trying
1:57to address here is more about
1:59multi-agentic, which is more for I I
2:03kind of in my mind I'm more thinking of
2:05the true next generation, next phase of
2:08agents development. That's actually more
2:12catered for enterprise level type of
2:15application, which I don't think is
2:17there yet. What we're doing now is again
2:20more kind of single step, I mean, single
2:23agent at the way I look at it. So,
2:25so then this particular talk is a bit
2:27more exploring some design patterns for
2:30more production AI. That's how I named
2:32this title. So, okay, because I I
2:34actually proposed another topic and then
2:36at the last minute the organizer at Git
2:39here was asking, "Oh, I saw you doing
2:41this one." And so I have to admit this a
2:43bit newer topic and it's going to be new
2:46to. And I also got asked by a gentleman
2:49from my talk yesterday afterwards. He
2:51said, "Have you actually done anything
2:53with, you know, implementing
2:55multi-agentic?" And I do have to admit
2:57at this point I have not have not done
3:00any serious type of multi-agentic
3:02applications yet, but I do feel there's
3:04a need for us to talk about it and
3:07explore it. So, that that's why I'm
3:09going to talk about this. So, okay, so
3:11enough of that and
3:12this is the the plan for today.
3:15It is just I'm going to have some
3:17introduction talking some fundamentals
3:19first about AI agents just to make sure
3:21all of us are, you know, coming from the
3:23same point for this discussion of this
3:26talk. And then some
3:28agentic architecture and patterns and
3:30some code samples as well. And then also
3:32I'll discuss some challenges of GenAI.
3:35Sorry, seems like there's a bit of a
3:36static.
3:37So, do we know like why? It seems like
3:40it keeps having some static, right? Does
3:42it bother you? Yeah. Okay. Yeah, okay.
3:45But I think it's okay. All right, so and
3:47then that that's the plan.
3:50Okay, so who is Mary? And I our
3:53MC already introduced some so thank you
3:55very much and but again picture is worth
3:57a thousand words and how many of you
3:59actually came to my talk yesterday? I'm
4:01just curious too. Also a few of you.
4:03Okay, so I may have something that I'm
4:06repeating so pardon me for doing that
4:08but right now too is just a quick
4:10introduction just picture is worth a
4:12thousand words and so I'm also Java
4:14Champion and also Oracle Ace too and but
4:18at the heart of myself I'm still
4:20passionate advocate meaning I like to go
4:22and speak and also
4:24you know I will reach work and meet
4:26developers like yourselves like that and
4:28but right now too I'm also involved with
4:30this group called AI Collective and how
4:33many of you I'm curious to have heard of
4:35AI Collective or maybe attended any of
4:37their meetups. No, not yet. Okay, but
4:41anyway, just to give you an idea, you
4:43know, how Java we have Java users group.
4:46This one AI is not strictly speaking
4:48it's not really like users group so to
4:50speak but it's more a collective of
4:53people that are can be developers can be
4:56technical and also non-technical folks
5:00such as you know people who are
5:02investors, who are founders. Some
5:04founders are developers themselves too
5:06but also sometimes a lot of founders
5:08could be non-technical. They are
5:10marketing sales in that you know in that
5:13kind of fashion and also kind of the
5:15business strategies kind of people
5:18and so as such you know the the meetups
5:21too that we organize to around the world
5:23now is global
5:25and there's also a Bangalore chapter by
5:27the way and in India we have Delhi and
5:29also Pune and we're going to open up
5:31more maybe in Mumbai as well in Chennai
5:34all of these cities too. So anyway, I'm
5:36like responsible for the Western
5:37Hemisphere. So we do meetups like
5:40regular meetups like technology but we
5:42also do what we call pitch night and
5:44these are like for founders. So if you
5:46How many of you, I'm curious, are
5:47founders actually? You may be like a you
5:50know a small startup founders, no? Oh,
5:53one. Okay, that's cool. Yeah. So, yeah,
5:55I I think maybe some cities is like San
5:58Francisco and in the USA is a lot of
6:00founders, as you may be aware. So,
6:02founders, too, they like they need to
6:04bring their idea business idea because
6:07you want to raise funds or whatever, you
6:08know, reason you need to raise the
6:10visibility. So, we also offer
6:13um these kind of meetups that's called
6:15pitch night. So, these are founders uh
6:17coming together to pitch. So, usually
6:19it's like a high-level product pitch.
6:21So, so nothing like deep down technical
6:24dive, but still interesting in the
6:26product sense. So, people can do pitch
6:28pitches for like 5 minutes, and then we
6:30also have investors that come and try to
6:33evaluate, maybe will attract some of
6:35investors to invest in some of these uh
6:37startup companies. So, so that's the
6:39kind of thing we do. Um so, essentially
6:41getting into founders community as well.
6:44So, I'm the VP of the global uh for
6:46Western Hemisphere, but I also travel
6:48everywhere. So, that's why I'm in India
6:50as well, and I also was in Europe as
6:52well. So,
6:54okay, and I also help the Chicago
6:55chapter um too um as well cuz that's
6:58where I live. I'm from Chicago. I'm also
7:01I also run the Java Users Group in
7:02Chicago, too. So, I I'm for
7:05Ja- Java Users Group C jug. I've been
7:07doing it for over 10 years, just just so
7:09you know. So, okay, so enough of myself,
7:12but my interest, too, is really into
7:14distributed systems. How many of you,
7:16I'm just curious, work kind of call
7:18yourself more of a distributed systems
7:20type of person? You handling streaming
7:22Yeah, okay, quite a few, too. So, that's
7:24cool. So, as as you know,
7:27if you're doing work on distributed
7:28systems, they tend to be more
7:30complicated, too. I mean, because we're
7:32now talking about data that's going to
7:34flow through different uh network
7:36boundaries, so to speak, right? And
7:38also, too, in some cases. And that's
7:41what we want to kind of investigate into
7:43is like agent system currently are
7:45pretty kind of single-minded so to speak
7:48and that's maybe using tool calling
7:51using MCP whatever it is conforming to
7:53MCP type of thing but it doesn't maybe
7:56necessarily deal with multiple you know
7:59your data flowing through different
8:00network boundaries so there's a need for
8:03making sure that if you have a
8:04transaction that's open and the
8:07transaction is going to flow through
8:09different sub systems and then you need
8:11to make sure the data comes out correct
8:13and consistent so that's kind of like
8:15the big challenges one of the big
8:17challenges of distributed system but
8:19anyways so that's what I'm kind of
8:20interested into okay so let me kind of
8:23quickly get into fundamentals of agents
8:25first so just so get some understanding
8:27so basically to agent right the way it
8:30we are understanding it is is in today's
8:33world is a bit overloaded I I should say
8:36right it's like this big truck kind of
8:38everything's overloaded meaning you know
8:40it from a marketing point of view or
8:42frameworks or research I think the
8:44non-technical folks tend to think of
8:46agent is a bit more like magic you know
8:49that and then it's interesting when I'm
8:50doing AI collective is like when you
8:53talk about agents and people are
8:54fascinated because they don't quite
8:56understand how you know programming
8:58works you know that and they're more
9:00thinking of maybe product like that so
9:02it's a bit on the marketing level maybe
9:04thinking of that and then they're also
9:06like framework level two there's also
9:08agents and
9:10I'm sorry
9:11I'm sorry just push it there
9:13push it out okay
9:15okay
9:16sorry
9:17I noticed yeah it's yeah it's better
9:19right okay all right thank you maybe
9:20that's right sorry okay so oops
9:24kind of
9:25okay
9:26can you still hear me right okay okay
9:28that I think that's better if I pull it
9:30away okay so okay so agent again back to
9:33it agent can kind of be confusing in
9:36that sense
9:37but let's kind of get back into some
9:39fundamentals too of agents. I feel
9:41there's a need to kind of understand it
9:43and also too I wanted to point out right
9:45from computing point of view actually
9:47it's not new. I'd say you know if you
9:49are familiar or working with Unix or
9:52Linux systems it's pretty much is also
9:55using a lot of agents but I call those
9:57agents demons too. They are process that
9:59that's running right. So it's
10:01essentially they usually are focused on
10:03certain goals doing something very well.
10:05So let's kind of take a little little
10:07quick look. Fundamentals of agents so
10:09why do we use agent at all? And because
10:12some tasks you know work better when the
10:14whole I call it shebang are handled by
10:17some single party. So for example you
10:20know I'm going to use some examples
10:22right of an agent. It can be travel
10:24agent or insurance agent right? And and
10:27it's kind of likely same thing like for
10:29us here like without you know kind of
10:31mentioning AI if we are you know living
10:34in today's world it can be very
10:36complicated. So if we need to get a car
10:38insurance or house insurance or
10:40something like that.
10:42So we can actually go and purchase the
10:45the insurance ourselves right? Like
10:47travel especially like say
10:49travel agent. We want to travel here
10:51let's say from Bangalore to Delhi. You
10:54want to make reservation and you can
10:57book it yourself of course but then what
10:59about you know sometimes you could run
11:01into problems as the flights get
11:03cancelled a flight get you know moved or
11:05updated all these things or you need to
11:08try to find the best route and you don't
11:10have enough time to search for it. So
11:12things like that right? It takes all
11:14takes up so much time. So it may maybe
11:17worthwhile to kind of delegate that's
11:19that's the keyword to delegate to some
11:21other party to handle the whole thing.
11:23That's why I call it whole shebang and
11:25it's basically it takes care of the goal
11:27of taking you safely from Bangalore to
11:30New Delhi something like that. So that's
11:32an example of an agent and some things
11:34are better handled by agent. But, it
11:36doesn't mean everything is good using
11:38agents, too. So, that's also a bit of a
11:40takes a bit of like experience and
11:42skills to figure out, you know, agents
11:44are not for all situations, but in a lot
11:46of cases maybe it makes sense. So, I
11:49then you know, ideally what the duties
11:51for an agent is is basically first to
11:53understand the goal, you know, what are
11:55you trying to achieve first the goal and
11:57then essentially to once you know what
11:59it is then you need to break it into
12:01different steps, especially we as
12:03developers and engineers, we we don't
12:05just like say it's not magic. It's not
12:08like raising a wand and oh, go do this.
12:10It's more like we need to break down,
12:12see what is needed and into different
12:14sub steps, right? Sub kind of sub
12:17parts of that. And uh so, from there,
12:20too, you also need to make decisions,
12:22too. And so, when there are conditions
12:24change, um for example, like you're
12:26booking travel and you know, there's a
12:28weather uh problem coming, you need to
12:30rebook. Things like that, you need to
12:32immediately make some decisions, cancel
12:34your flight or update whatever is
12:35needed. Um and then also there are
12:37exceptions, too. Maybe like you know,
12:40completely a flight got canceled. Now,
12:41you're going to change your plans, you
12:43need need to fly from Bangalore maybe to
12:45Mumbai, Mumbai to Delhi, something like
12:47that. Handle the exceptions and then
12:49also to keep track of the progress, too,
12:51in a lot of cases. So, that's kind of
12:53like basic abstract sense what the
12:55duties of an agent is. So, examples
12:58already kind of quoted travel agent or
13:00insurance agent.
13:03Okay, so now then get in getting
13:04bringing AI into this conversation is
13:07basically AI is essentially autonomous.
13:10This autonomous um way of performing
13:13task and also specifically is working
13:15with language large language models.
13:18These are like the premier kind of you
13:20know, name for the kind of models that's
13:22working with the current state of AI,
13:24generative AI that we're working with,
13:26right? At this core of this AI agent.
13:29So, sometimes these agents are also
13:31called LLM agents, too.
13:33Um but, as we all know, LLMs are very
13:35restrictive in the sense because it's
13:37kind of bounded. It's depending on what
13:40data you use for training it. And that's
13:42all it knows. That that's, you know,
13:44that's the LLM knows about. So, the
13:45scope is actually as large as the data
13:48set is, it's still kind of uh bounded,
13:51you know, by what it's being trained on.
13:53And the reasoning is as well. Right now,
13:56as you kind of as we all understand, it
13:59there isn't any true AGI, you know,
14:01artificial general intelligence, which
14:03actually is about true reasoning in
14:05which, you know, then agent able to
14:07figure something out. As we all know
14:09right now, you know,
14:10as we have worked actually with LLMs,
14:13you know, since ChatGPT came out in
14:152023, it's not actually magic. Again,
14:18it's like uh Neil Ford, uh one of the
14:21speakers, right? The architect speaker
14:22from ThoughtWorks, he talked about uh at
14:25Venkat's conference actually at last
14:27week. And he said, "They are actually
14:30LLMs are like recipe finders." And think
14:32of yourself as like trying to cook food,
14:34and you need to like how do I cook food?
14:36And you ask the agent, it needs to look
14:38for some recipe for you. But, it in
14:40using an analogy, that's what LLM is,
14:43right? And that's what it is.
14:45So, basically, agentic technology will
14:47extend um the capability of LLM cuz
14:50because of this limitation of LLM. And
14:53the way it does it is, you know, via
14:55tool calling in the back end. And also,
14:57too, um it should be able to obtain some
15:00up-to-date information because LLMs are
15:02trained with old data because you it
15:04needs a lot of time to train it. So,
15:06usually, too, the LLM that comes out
15:09would contain data from, you know,
15:10couple at least couple months back, like
15:12for example, like that. And then also,
15:15the workflows, too, will have to be
15:17optimized. And also, agents ought to be
15:19able to create subtasks to all of these
15:22things and being able to be flexible.
15:24So, now let's take time to take a look
15:26for single agent, you know, and what is
15:28an AI agent? It's clear definition is
15:30basically AI agent, it knows it should
15:33be able to perceive the environment,
15:35too, and then make some decisions and
15:37take actions on it, too. So, it's not
15:40like a chatbot, you know, chatbot
15:41usually well, they are also you can
15:43think of them sort of an agent, but
15:45specialized case in which it is
15:46responding to some prompts. Uh usually a
15:49prompt will kind of guide what the
15:51chatbot should do, but agent sometimes
15:53is could be some background task, you
15:55know, like think of again, going back to
15:57Linux um kind of environment is you have
16:00cron jobs that does stuff, and those are
16:02kind of is actually agents to that
16:04handle handle some of the cron jobs, all
16:06of the steps, too, but this is kind of
16:08like the agent definition of agent. And
16:10then also some building blocks of
16:12agents, these are core components of an
16:14AI agent. It's very often it needs
16:16memory because it again, you know, LM is
16:19stateless, doesn't have memory, but
16:20that's what agent will make possible to
16:22be able to remember what it has been
16:25asked or some, you know, some like the
16:27context essentially. With the memory,
16:29you can build up a context, so then, you
16:31know, you kind of maintain the state for
16:34you your interaction with an LM. And of
16:36course, too, to a certain extent we call
16:38it reasoning, but it's not true
16:40reasoning, but in some ways it's like
16:42agent will rely on external tools, for
16:44example, to help with reasoning. And
16:46then also planning, too. Planning is
16:49essentially you know, we give it a
16:50problem and agent can should can and
16:53should be able to break down into
16:55sub-steps to to kind of tell how to do
16:58something, and also pro pro kind of
17:00provide a bit of an action interface,
17:02too. So, essentially AI agent is
17:04composed of multiple layers, too, that
17:06which are like these kind of four
17:09building blocks here.
17:11Okay, so we'll get into a bit of agentic
17:13architecture now. So, the current
17:15definition of agent is basically a
17:17system that first receives a goal and
17:20then two will decides what to do next,
17:22right? So, that part is the planning
17:24part. And then also then it needs to
17:26keep uh the context relevant. Keeps
17:29these relevant contexts and then it is
17:31essentially does all the execution and
17:33then repeats until it reaches a certain
17:35stopping condition, what you're trying
17:37to achieve, like that. And then what's
17:39kind of interesting, the planning uh
17:41when I was looking up IBM's definition
17:43is, you know, formal definition is
17:45planning. It's like determining a
17:47sequence of actions toward a goal
17:49alongside modules like, you know,
17:51memory, reasoning, and action. So,
17:53that's thing.
17:54And our current current understanding
17:56too, especially when we work with AI
17:58agents and it deals with like prompt
18:00chaining. As we all know, you know, you
18:02if you're working with prompts, these
18:03are essentially instead of using one
18:05single prompt, you could be breaking,
18:07you know, your prompts up into like
18:09different uh steps. So, then you guide
18:12guide it so then it's not just one chunk
18:14of prompt going in. It's more like you
18:16chained it up so the input, you know,
18:18from one or the output from one the
18:20first prompt can become input to the
18:23next prompt. So, and so on and so forth.
18:26So, it kind of makes you can kind of
18:28guide it like this and makes your what
18:30you know, achieve what you're trying to
18:32achieve essentially better, right? It
18:34kind of reducing hallucination in some
18:36sense. And then of course there are also
18:38tool calling assistants and these are
18:40usually um a kind of like tools uh
18:43working with any kind of external tools
18:45in the back end too. As well as also
18:48handling stateful type of workflows and
18:50these are essentially two um
18:53We we probably won't get into all of
18:55these yet, you know, like when we talk
18:57about stateful workflows and very often,
19:00I think if we think of enterprise level
19:02then it's probably dealing with more
19:04transactional type of scenario. But at
19:06least too, like workflows could be there
19:09are different workflows that carries
19:10with it data that you want to make sure
19:12the data is is being consistent between
19:14different steps as it step through the
19:16different, you know, flow type of
19:18procedure.
19:20Okay. So, now I kind of get into some
19:23agentic architecture. In this case too,
19:25like just now I talked more a bit more
19:27about
19:28single agent, but let's see, you know,
19:30in in kind of more enterprise
19:34level of
19:35execution and agentic type of ways of
19:38doing things and basically, you know,
19:41having single agents probably not going
19:43to help as much as basically having
19:46multiple agents kind of scenario. We are
19:48still using some multi-agent scenario
19:50now to do things too, but then the the
19:52fact is,
19:54you know, it's basically
19:57say for example, we're going to design
20:00some kind of airline reservation system
20:02completely guided by agents. Then in
20:04that case, you probably will need multi-
20:07multiple agents because you need maybe
20:09an agent first to interact with you and
20:11then from there, it actually, you know,
20:13the goal is to book tickets, you know,
20:16and and manage your flights and and
20:18probably then you have like different
20:20steps of booking your tickets and then
20:22eventually too, you also have to see if
20:24there are, you know, you have to
20:25interact with another booking system
20:27with all of the maybe another system
20:29that aggregates all of the flight
20:31information. You have to then, you know,
20:33figure out if the time would work with
20:35your personal calendar, things like
20:37that. And then later, you also have the
20:39agent interact, you know, with your
20:41payment system as well. And and so, all
20:44of these things will involve multiple
20:45subsystems along the way. So, it's it's
20:47going to be always going to be more
20:49complicated than one single agent doing
20:52everything too. You have multiple agents
20:54and some are doing certain task. So,
20:57basically in agentic architecture sense,
21:00then you can have like a vertical
21:01architecture, essentially having one
21:03agent be the leader too. And then, um,
21:07they're are also like what is called
21:08like horizontal architecture, too. So,
21:10then this case you can have multiple
21:12agents, each collaborating, but they are
21:14of equal kind of, um,
21:17what you call equal power, so to speak,
21:19that it doesn't it isn't quite a leader,
21:21but they all kind of collaborate
21:23together, and depending on situation,
21:25maybe one could be voted as a leader,
21:27like that. And then, also, too, then,
21:29um, in more complicated scenario, you
21:31also need someone essentially another
21:34agent be the orchestrator. So, the one
21:36orchestrator is the one that kind of
21:38overseeing, making sure all of the
21:40agents, whoever is supposed to do do
21:42some task, they are doing it in the
21:44proper sequence. As we all know,
21:46sometimes, especially say with financial
21:48type of situations, for example, you
21:50need to somebody needs to with I'm just
21:52citing a kind of a straightforward
21:54example, somebody needs to withdraw
21:56money, but it actually needs to have the
21:59transaction of him putting in money
22:01first before they can withdraw,
22:02otherwise you'll have a negative
22:04balance, for example, things like that.
22:05So, you also need, for example, in this
22:07case, maybe an overseer type of agents
22:10to be orchestrated to orchestrate all of
22:12these things, but of course, that's a
22:13simple kind of cases. In really
22:16complicated type of financial
22:18transactions, for example, making buying
22:20and selling of stocks and options and
22:23trade trading situation, maybe you
22:24really need a better orchestrator to
22:27orchestrate everything. Um, so and also
22:29I like to kind of make a the compare an
22:32analogy, too. Orchestration is
22:33necessary. It's like if you kind of
22:35think of it if you are like musically
22:37inclined and you listen to music and you
22:40know, in an orchestra or any kind of an
22:42ensemble of music, you know, kind of a
22:44performance, there can be multiple, you
22:47know, instruments playing, but you want
22:49to make sure the music comes out
22:50correct, too. So, very often, right, in
22:53symphony orchestra, you need an
22:54orchestrator, then that's their job is
22:57to conduct and making sure all of the
22:59instruments are playing music at the
23:00right time. So, then the music comes out
23:03correct to kind of a pleasant to the
23:05ear, so to speak. But it's kind of in
23:06that sense
23:08that agent should work like that. And of
23:10course, too, then it should also help
23:12with, you know, complex kind of task
23:14execution. It should kind of manage that
23:16and also the autonomous decision-making
23:19side. And right now, again, you know,
23:21agents are not fully, you know, 100%
23:24like able to reason, but we can kind of
23:27maybe have external tooling, that type
23:29of stuff to help to make it sort of like
23:32helping human in the loop type of
23:33reasoning as well and planning as well.
23:36And then also, too, one key kind of
23:39capability agent should have is
23:40basically a continuous learning and
23:42adaptability kind of capability
23:45capability, capability, too. It needs to
23:47be able to adapt to the different kind
23:49of situations. And essentially, too, we
23:52want true agentic architectures to have
23:54no human in the loop or human
23:57intervention. Basically,
23:59currently or I should say, currently,
24:01too, a lot of cases in which we're
24:03seeing all these systems still require
24:05humans to be in the loop. You know, you
24:07still need to check it. And the way the
24:09LLMs are today are still not quite You
24:12can't fully fully be trusting it. And
24:15I'd like to suggest let's not fully
24:16trust LLMs because they are, again, just
24:19recipe finder. They are based on fuzzy
24:21logic and they're non-deterministic. So,
24:24don't always trust it and always have a
24:26second pair of eyes looking at the
24:28results that it gets generated and
24:30verify. Either you you use another agent
24:33to eval, like eval agent, to make sure
24:35it's correct or better, you know, kind
24:37of closer to accuracy than it is, you
24:39know, with completely trusting it. And
24:42but also, too, while I'm here, I think I
24:44should also mention because as as
24:46developers, we tend to be we're more
24:48rational in our thinking. We we do
24:50things and I think we tend to be better
24:52in terms of not being fooled by any
24:54system. But let me tell you a story
24:57because I'm I also organize events, you
24:59know, so the AI collective in Chicago,
25:01we had an event that celebrating Women's
25:04Day, International Women's Day in March.
25:06And then I have some you know, mothers
25:08coming to a a meet-up. And not just
25:11mothers too, there are fathers too,
25:12other people, but it's basically
25:13celebrating women, but I just want to
25:15share this story and it's telling
25:18about how, you know, AI, I know it's not
25:20developer specific, but I think it's
25:22worth mentioning because the mom said
25:25that her son has autism and then was
25:27taking medicine medicine. And so the son
25:30actually quite smart, 15-year-old
25:32teenager, just like any teenagers,
25:34you know, used ChatGPT and then ask, you
25:37know, he even he was asking question
25:39about the medicine he was taking for his
25:40autism. And eventually, what he got back
25:43from the ChatGPT, I think through many
25:45conversations, he was asking different
25:47things, the ChatGPT came out and told
25:50him,
25:51"Do not trust your parents. They do not
25:53know what they're doing, you know, this
25:54medicine and blah blah blah." So the kid
25:57was like thinking, "Oh, that's what
25:59ChatGPT told me to do, so I'm not going
26:01to trust my mom." And so thankfully,
26:03because he did it at school, at the
26:05school computer, so the staff caught him
26:08kind of like asking ChatGPT these things
26:10and he was check she was checking, so
26:12she immediately notified the parents. So
26:14thankfully, so the mom was telling the
26:16story. So as you can tell, all of these
26:20you know, AI thing, they are just being
26:22trained with some certain set of data.
26:24If the data is bad, it is whatever it
26:27is, it's just spill out what it's being
26:28trained on. So it can be telling you not
26:30true things, things aren't true. So do
26:33not trust it right away. Always be
26:35skeptical, not to say don't trust it to
26:37the point of not being productive, but
26:40always kind of with the skeptical mind
26:42saying that, "Well, this is just AI as a
26:44tool, it can be wrong too." and all
26:46these things. So anyway,
26:48I just want to tell that story. I like
26:49to kind of mention that because even I
26:52have other
26:53Java developers who was talking to me
26:55that they were kind of using it to
26:57generate code, and they were looking at
26:59things, and they were like, "Oh, this
27:01looks like it's it it looks good." And
27:03they keep doing the the, you know, the
27:05code generation, the code assistant, and
27:07he this this friend of mine also was
27:09saying that, "Oh, I'm starting to
27:11believe that it is true." And so easy,
27:13right? That we feel like we can start to
27:15trust it, but just remind ourselves do
27:17not trust it that is the key. So, not
27:20all the way. Okay, so
27:22let let me kind of move forward. So,
27:24some fundamentals to of agents again,
27:26you know, basically agentic and
27:28basically the the agent is one who is
27:30authorized to act for on the place of
27:33another. And so, always remember this
27:36not trustable. It's just somebody you
27:38can delegate to do some work for you.
27:40And also agentic is basically use
27:43systems that use AI to pursue the goals
27:45and complete tasks on behalf of users.
27:48That should show reasoning and planning
27:50and memory and have a high level of
27:52autonomy to make decisions to and and
27:54adapt and all these things. So, that's
27:56kind of the general thing. Generally, I
27:58agents are basically software entities
28:00that orchestrate complex workflows and
28:03coordinate the activities of multiple
28:04agents, process logic, and evaluate
28:07answers. Multi-agentic systems are when
28:10multiple autonomous work agents working
28:13together, and they each specialize in
28:15some task, and they also will have
28:18intelligence to from agent-to-agent kind
28:20of interactions, too. And it needs to be
28:22able to divide complex tasks up into
28:25manageable subtasks and leverage the
28:27strengths of different AI models to
28:29increase robustness and fault tolerance,
28:31as well.
28:32Okay, so now I just quickly, too, there
28:35are also protocol that's being used from
28:37Anthropic is model context protocol,
28:39MCP, and that one is more of a lower
28:42level that interacts with resources, you
28:45know, infrastructure, that kind of
28:47level. And then there's also A2A
28:49protocols, agent-to-agent protocol, and
28:51that one is from Google, but now part of
28:53Linux found governed by Linux
28:56Foundation, too. But to kind of look at
28:58the two kind of relationship, too, the
29:00next diagram will show you that MCP is
29:02on the again, more on the
29:04infrastructural lower level, interacts
29:06with enterprise apps and doing all these
29:08things interacting with database, for
29:10example. Agent-to-agent is the higher
29:12level that's like between agents. They
29:15they can communicate among themselves.
29:18Okay. So here and and just a bunch of
29:20things, but I actually won't kind of
29:22going through all of it because I said
29:24we'll talk more about the patterns, too.
29:27So I just want to jump to this talking
29:29about agentic design patterns. And those
29:31of you, maybe just a few who came to my
29:33talk yesterday, so these are some of the
29:35repeats too that I talk about, but I
29:37wanted to talk about agentic design
29:39patterns. There are essentially like
29:41four things that are
29:43kind of
29:44you know, kind of can think in AI
29:46specific for agents, too. This one is a
29:49reflection and then tool use and
29:51planning and multi-agent collaboration.
29:55So and also actually orchestrated to our
29:57going next, but first let's talk about
29:59reflection first. And so this one I also
30:02talk about yesterday, so bear with me if
30:03you already heard it. So basically, too,
30:06we want to design these agents
30:09let's say, right? These days where it's
30:11common we're using coding agent. But at
30:14the same time, again, we want to be
30:16critical skeptical, right? About what
30:18gets
30:20what comes back from the LLM. We need to
30:22be skeptical in knowing that it isn't
30:24always trust you shouldn't trust it
30:26right away. So in this case we use a
30:28critic agent, which is also called an
30:30eval agent, to kind of be there kind of
30:33like looking over the back essentially
30:35with the coder agent. Coder agent will
30:37interact with your LLM, ask question and
30:39maybe you know, kind of first ask your
30:42LLM to say, you know, perform some task,
30:44do something, and generate me the code.
30:47And so, the critic agent will be there
30:49and immediately like criticize, you
30:51know, looking at the output and will
30:52find some problems with it. With that,
30:55it asks the coder agent to
30:57kind of re- re- send this
31:00information back to the LLM and ask the
31:03LLM to regenerate better code, like
31:05that. So, kind of so on so forth. So,
31:07it's like reflection is kind of a
31:09reflection kind of pattern, right? You
31:11just kind of have a feedback kind of
31:13loop. And eventually, you kind of do it
31:15enough times to kind of achieve that
31:17condition in which you know the the
31:19final generated set of code is the most
31:22accurate. And so, that's a reflection.
31:25Okay. And then, the next
31:28pattern is called tool use. So, tool use
31:30too. Now again, I didn't kind of mention
31:33this all these patterns
31:35I'm actually essentially to describing
31:37what Dr. Andrew Ng, he's like a
31:39well-known
31:41AI person, right? He started the
31:43deeplearning.ai.
31:45So, very knowledgeable, obviously,
31:47person in AI. So, I kind of am quoting
31:50what he says in here. And he has a lab
31:52that also does a lot of research, too.
31:55So, so this is what I'm just repeating.
31:57So, reflection is the first one and tool
31:59use is the second one in which this
32:01pattern, meaning that we know that
32:03sometimes LLM may not have the answer.
32:05So, in that particular case, you want to
32:07configure it, basically giving the agent
32:10some external help. And so, these are
32:13essentially using tools in the back end
32:15to kind of extend, you know, what the
32:17LLM can't give you back or it doesn't
32:19have an answer, then the agents should
32:22go and reach out to these tools in the
32:24back end and ask for, you know,
32:26additional help. And that's what is tool
32:29tool use, this pattern. So, example is
32:31that it could be, you know, you you
32:33giving it help and saying that, you
32:35know, if the agent doesn't get a proper
32:37answer, you can always use for example,
32:40go to Wikipedia, go to some sites and
32:42other website or another tool, whatever
32:44it is, to look for additional help, you
32:47know, to get better answers from that.
32:49So, that's essentially what it is.
32:52And then this one too, I'm using an
32:54example. I thought just mention it
32:56because there's a quite a popular
32:58agent development SDK called Crew AI.
33:02Maybe some of you have worked with this
33:03Python essentially. And but Crew AI too,
33:06their model is is more of a rule-based
33:09type of
33:10agent agentic kind of approach of doing
33:13things. And so, they actually have this
33:15particular way of doing things and they
33:17have Crew and you can have different
33:19agents and and you kind of define the
33:22tools for example, and then they're also
33:24process that helps, you know, define how
33:26the agents will actually work together
33:28to how tasks are designed, all these
33:30things. Because because it's
33:33That's weird that sounded me.
33:35And um
33:37So, okay. So, they they're also the Crew
33:39AI again, it's essentially the same kind
33:41of idea using tool uses reaching out to
33:44get external help. And then essentially
33:47too, then they're also tasked and task
33:49can actually essentially override the
33:51agent tool with specific ones too. And
33:54so, they're also specific like agent
33:57that can tackle it too. So, it's a it
33:58gives a little bit of a different flavor
34:00to this tool use type of interface to in
34:03Crew AI. And then it will generate you
34:05the outcome. So, I have also a link in
34:07here too. So, if I get to the point, you
34:09can if you like I'm sharing this slide
34:12deck, so you can also have access to
34:13this information. Okay.
34:16Now, this third pattern here is
34:18basically very interesting. It's called
34:20planning. So, in this particular case,
34:22you actually want the agent to do a bit
34:25more work, a kind of more thinking. So,
34:27essentially too, planning is, you know,
34:29using an example like this Um, is you
34:32can just say give give a picture an
34:34image, right, to the LLM and saying
34:37that, well, you know, this is a little
34:39boy on the left-hand side is riding a
34:41scooter. And what I want to achieve is
34:43to get a another picture with a little
34:46girl in the same posture, but the little
34:49girl is reading a book. So, in this
34:51case, yeah, you're not really giving too
34:52much kind of hint to the agents, really
34:55letting the agent be the driver to
34:57figure this out. So, so in this
34:59particular case, you know, being an
35:01agent, you know, I would kind of look at
35:03it, what is the goal? Let me kind of
35:04start breaking up the steps, and then it
35:06has to look, you know, and use
35:08techniques to kind of figure out the
35:09image and all of these positioning,
35:11whatever it is using, that's what the
35:13power of LLM can come in. Looking, you
35:15know, using a visual based kind of LLM,
35:18look at this, and then it determine the
35:21the
35:21posture, and then using some model to
35:24kind of determine the posture, and then
35:27essentially, too, using the posture to
35:28look for an image of a little girl, and
35:31then in this particular example, maybe
35:33using a Google VIT kind of model, for
35:36example, and search for a girl, and then
35:38also look for another image that's
35:40reading a book instead. So, then it
35:43should then generate you that output on
35:45the right side like that. So, so as you
35:47can see, I'm skipping over all of it,
35:49but essentially, the agents will have to
35:51determine all the posture and then
35:52figuring out the steps in order to
35:54achieve the final image in this
35:56particular case. But I want to also
35:58point out is that from this article that
36:00I read and which I'm sharing in here,
36:02Dr. Ing was saying that his lab at the
36:04time, maybe 8 months ago, it it isn't as
36:08it wasn't as accurate at that point as
36:11as, you know, they tried different ways
36:13of using this approach or kind of trying
36:16to use images and generate the end
36:17result, but it's not not very good yet
36:21at that point. But I believe, you know,
36:228 months later, maybe these days the
36:24models are better at a better state, and
36:27also agents are better in terms of able
36:30to figure things out. You know, this
36:32kind of abstract input, it should be
36:33able to do a better job. So, Does that
36:36planning?
36:37And then also then the fourth pattern in
36:39here that it describes is called
36:40multi-agentic collaboration. So, this
36:43particular one is a bit kind of
36:45not in real life yet, but there's there
36:48is actually a GitHub too that they are
36:51developing this. It's sort of like a
36:53game game-like kind of scenario. So, if
36:56you're interested, I also have shared
36:58this particular
36:59link to to its article. But, it's very
37:02interesting as you can see too. It's
37:04kind of mimicking an IT project using a
37:06waterfall model. And this agentic design
37:10patterns is essentially multi-agentic
37:12collaborative collaborations too. So,
37:15you can have different roles, you know,
37:17in your in in your code or in your in
37:20your particular system that they are
37:21different roles that are handling design
37:23designing things or coding, testing, or
37:26documenting all these things. And there
37:28are different, you know, interaction
37:30between the different
37:32kind of roles among themselves. And then
37:35it should be able to then help to
37:37produce a project. But, that's again,
37:39you know, it's not real life yet. But, I
37:40think it's an interesting kind of
37:42concept that they are experimenting on.
37:44So, if you want to, you can look into
37:46that too. Okay. So,
37:49these are examples of a few
37:50multi-agentic libraries like AutoGen
37:52from Microsoft or CrewAI. And I also
37:55mentioned to and on Java side there's
37:58LangChain, FourChain, of course. And
38:00then of course LangChain by itself is
38:02Python.
38:03LangGraph is also is graph-based and
38:06LangGraph 4J. And there's also some code
38:08example I'll show you shortly too. It's
38:11using LangGraph to doing
38:13better position the you know, the to to
38:16do like orchestration type of work. And
38:18there's also Strands from AWS or Agent
38:20SDK from Google. And on Java side of
38:23your Java Quarkus LangChain 4j as which
38:26is based on CDI also has some examples
38:29too which in I've given in here.
38:31Okay, there just a bit this page is just
38:34giving you some additional code examples
38:36and there are also more too. So,
38:39yeah, so if you if you miss it I think
38:41so I see some people taking picture but
38:43you can also yeah, get the link to this
38:45code. Okay, so just a quick example
38:47then. So, here LangGraph 4j some example
38:50here is just a quick one. It's not like
38:53you know, I'm not really showing
38:54anything
38:55working so to speak but just to show you
38:58using LangGraph 4j you can use a state
39:00graph and essentially too it's
39:03basically help because you can then
39:05using graph and you can actually add the
39:08node and add add the edge. So, node is
39:10more like the entities and then edge is
39:12essentially describing transitions right
39:15between between nodes for example and
39:17then you can then use a state graph and
39:19do the compile and in this particular
39:21case it's just simply print out these
39:23items to in this graph for example. So,
39:26so this is just again an example but
39:28wanting to kind of
39:30kind of point out to you is basically
39:32helps with a bit more explicit like
39:34state and orchestration to using
39:36LangGraph 4j.
39:38Again, you know, this might be something
39:40you can consider using if you're
39:41interested because
39:43especially if you're thinking of
39:45thinking of your code in terms of nodes
39:47and transitions and deterministic flow.
39:50So, this state object help with making
39:53like memory and intermediate outputs
39:55more explicit and then nodes will
39:57represent the units of work or decisions
39:59and the edges define like all the
40:01transitions too. So, essentially too if
40:04you're doing orchestration you can maybe
40:06consider using LangGraph 4j to help you
40:09with it too. And LangChain 4j also has
40:12some higher level building blocks too
40:14for tool calling and using chat memory
40:17rag, and agents as well. So,
40:20Okay. So, here are some challenges. I
40:21thought I have more too. Let me see.
40:23Okay, so challenges again before I get
40:25to the example, just wanted to quickly
40:27point out to, you know, using still, you
40:29know, with agents and doing like AI, I
40:32think it's exciting and everything, but
40:34just always remember that
40:36there are always some challenges,
40:38especially, you know, these are complex
40:40and we have to deal with agent memory,
40:42context management, and you have to
40:44ensure the consistency among, especially
40:46if they're dealing with multiple
40:48distributed agents to the data
40:50consistency. And there are also failure
40:52situation, partial results, and also
40:55latency and resource usage, you know,
40:57how do you store certain data, all of
40:59these things. Yesterday I talked a bit
41:00about data engineering side too that
41:03that type of stuff, the pipeline, the
41:04data flowing through different network
41:06boundaries. And also too, ultimately
41:09too, the LLMs too are quite expensive as
41:12we all know, uses a lot of tokens.
41:14Sometimes, you know, when when you use
41:15it, you know, if you don't think about
41:17using it you know, before you use it, it
41:19can actually use up a lot of tokens like
41:21that. But there are also Yesterday I was
41:23actually went to a DevRel Jam. It's
41:26called DevRel Jam meetup too. That's
41:28offsite. And and Brian Benz, who is from
41:31Microsoft, he talked about there's a I
41:33I'm less of paying attention to some of
41:36these chip maker, all these things, but
41:38I think there are new ways of kind of
41:41chip that enables you to use less tokens
41:44to do certain things too. So, those kind
41:47of things, yeah, there are certainly
41:48there are advances to make in terms of,
41:51you know, reducing pollution, so to
41:53speak, and all that stuff. But still,
41:56you know, as we all know,
41:57you know, it can use up a lot of tokens,
41:59which means the data center is
42:00generating a lot of heat too. From what
42:02I know is that I think the the demand is
42:05quite high that we're using all of these
42:07AI agents and tools and they're burning
42:10up a lot of energy. That that's the you
42:12know, that challenge is still there. I
42:14think we're advancing, but it's just not
42:16quite there to make it more
42:18kind of commoditized to make the cost
42:20lower in that sense. And of course too,
42:23LLMs too, one thing again is is
42:25non-deterministic.
42:27It's a fuzzy logic, so there are still
42:29hallucinations. We can't just trust it
42:31right away. And also one more thing,
42:33right, is basically the very important
42:35is AI safety and responsibility to the
42:38ethics side of things. Um and we just
42:41have to make sure, especially if we're
42:43builders, developers, developing code,
42:45we also have to be aware of the fact
42:47that what we're going to do, what we're
42:49going to produce is not going to be
42:51misleading. And also we also as
42:54developers, I feel we are all advocates
42:56too. We should go out and educate
42:58people, right? Especially sometimes the
43:00non-technical people, they get a bit
43:02nervous. Oh, what is this? What is that?
43:04They just don't have the idea of how to
43:06do coding. They thought this is really
43:08like magic that can that can cause the
43:10world to end or something, right? We we
43:13know it's not going to be that simple of
43:15just able able to you know, end the
43:17world. I actually read about something
43:20about this famous computer scientist, I
43:22think Yoshua Bengio, a Canadian
43:25computer scientist. And he made a
43:27comment that oh, this is very scary. AI
43:29can actually
43:31you cannot even shut it down or
43:33something. And I was thinking, why is it
43:35not able to shut down? Anyway, I saw
43:37that like a interview. And I don't know
43:40why they're saying stuff like that
43:41because I'm thinking it can have danger,
43:43but it's not about not able to get shut
43:46down. It's it's more about people
43:47misunderstanding and being like deceived
43:50by things that they think is real. But
43:53somehow he was saying that because I was
43:55immediately thinking, why can't you shut
43:56it down? You can turn off the power, the
43:59power supply and everything will shut
44:00down. So, I don't know. But anyway, I
44:02got into some more talking with some
44:04colleagues and stuff and they were
44:05saying that, you know, sometimes these
44:07is so-called like celebrity computer
44:09scientist, they think they are the ones
44:11who they are the only ones who know the
44:13secret
44:15hold the secret key to how things work.
44:17Maybe they are just thinking, you know,
44:19like a little bit like too arrogant to
44:22think like that. So, they are like
44:23scaring everybody. Maybe, yeah, I don't
44:25know, but I don't know if you've heard
44:27of that. But, anyway, to me I feel like,
44:29you know, nothing to scare about.
44:30Although, there are also things in which
44:32we are I I'm sure everybody maybe in
44:35India is not as much an issue, but in
44:37the US I feel sorry for some of the
44:39junior developers. They are not They
44:41just graduated or about to graduate.
44:44They are having trouble looking for jobs
44:45and because right now a lot of
44:47companies, too, they are cutting costs.
44:49So, they are
44:50they're basically saying that, well, we
44:51don't need as many programmers because
44:53we can five code and get the job done.
44:56And while five coding can help you, but
44:58I've done it before, but I feel that I
45:00still need to spend quite a bit of time
45:02to to make sure my code work correctly,
45:05right? Or doing the what I wanted to do.
45:07So, it's not completely and plus I'm I'm
45:09just generating something quite simple.
45:11If I kind of go back into doing
45:12enterprise level of work and interacting
45:15with database, that's no way.
45:17You can five You can't five code it. And
45:19I don't know why. Maybe the whole
45:21society, maybe especially in the US,
45:23there are lots of companies in San
45:25Francisco, you know, Silicon Valley as
45:27we all know innovation is going crazy.
45:30They are just trying to make it sound
45:31like, "Oh, this is, you know, we're
45:33going to take away jobs and creating a
45:35sense of fear maybe among people." And
45:37then the rest of the companies are
45:39responding, "Oh, we better do something
45:40to so that we are not losing money." And
45:43we kind of show the world that we can do
45:45better without you know, without being
45:47so expensive by not hiring people, then
45:49we can we can be ahead because we get
45:51value valuation higher, something, I
45:54think. So, anyway, but my thing I feel
45:57is that what I tell the junior people
45:59who come to my meet meet up and
46:02and their concern I'll say, "You know,
46:03just continue." And I think there's
46:05still
46:06a lot of valid reason why as developers
46:09we need to keep these skills going and
46:11there's so much
46:13you can also say there's also a lot of
46:15fun too, you know, that how I got into
46:17programming is because when I was
46:19younger, right? When I was going to
46:20universities because I really enjoy
46:22coding. There's also the joy in doing
46:24coding is a good good for your mind and
46:27of course we get into working. We need
46:29to do things faster, but that's when we
46:31can use AI
46:32code assistant to help us to achieve
46:34results faster in certain cases, right?
46:36But it's not going to replace us is what
46:38I think. So, anyway, but that's just
46:40what I want to share too. I feel that we
46:43are, you know, while we're talking about
46:44tech stuff we need to be aware of some
46:46of these other things too, I think.
46:48Okay, so
46:49these are resources, but I I'm going to
46:51show some code too, but before I go I
46:53can
46:55kind of share with you this slide deck
46:56first and so
46:58otherwise sometimes if I show code and
47:00then folks started leaving and maybe you
47:02won't get to it. So
47:04Okay.
47:05So, okay. Yeah.
47:08Okay, cool. Yeah. And if you miss it
47:11don't worry because
47:13or I'll also show some of the thing, but
47:15there are also some links too, but I
47:17again will show some more stuff and I
47:18also by the way have a
47:20live streaming Twitch too on Twitch and
47:23then on YouTube as well. So, I suggest
47:25if you're interested you feel free to
47:27join or even if you can join it live
47:30I'll try to schedule it so the timing
47:31would work for your time zone because
47:33I'm based in Chicago, but let's see what
47:36to do because if you're doing it live if
47:38you respond back, right? I can actually
47:40invite you to join my stream if you want
47:42to, you know, kind of like speak too.
47:44And I I have done it before. When I was
47:46at IBM I I did it and it's kind of fun
47:48too. There are people talking. I said,
47:50"Why don't you come on to my stream?"
47:52People will be chatting and for a while
47:53too. And and it's kind of fun because
47:55we're doing live coding and trying
47:57different things out. So, yeah, so
47:58that's that. And then there's also the
48:00the link of all of my stuff. Although, I
48:03I don't know why there isn't that one uh
48:05slide, but I will then go back to uh
48:07showing some code at this point, too.
48:08So, okay.
48:10Anyway, you can find me, too, on
48:12LinkedIn of these things. So,
48:14so over here, I just again, you know, I
48:16have to say my examples isn't so much
48:19about the multi um multi-agentic uh
48:22pattern. Oh, I actually forgot to talk
48:24about one thing before I get here, of
48:26course, yeah. Because I talk about the
48:28uh
48:29I think I remember I skipped this one
48:31thing, which is uh about orchestrator.
48:34Uh just one more thing before I get to
48:35that. It's basically agentic
48:37orchestrator design patterns, too. Yeah,
48:40because I just talked about the agent
48:42design pattern, but orchestrator design
48:44pattern. So, orchestrator again, you
48:46know, it's a dealing more with
48:47multi-agents kind of scenario, and it's
48:49basically there essentially like four
48:51things I just quickly kind of talk about
48:54it. Orchestrator worker. So, in this
48:56particular case then, it essentially
48:58having a central agent that would
49:00decomposes a task and delegates these
49:03subtasks to specialized worker agents,
49:06and then it aggregates and validate
49:07results, too. And of course, it's easier
49:09to set than done, you know, it's it's
49:11but the pattern is how it works, you
49:14know, orchestrator worker kind of
49:16pattern, a bit more like the vertical
49:18type of architecture. And then there are
49:20also hierarchical agent, too. They These
49:22are organizing layers, too. So, it it
49:25depending on your example, you know, of
49:27your uh um
49:28your uh system, your application, I
49:29should say. So, you can have like, for
49:31example, a manager, a specialist, and
49:34ex- executor, something. And they will
49:37enable abstraction, delegation, and
49:39error handling across different levels,
49:41too. So, these are like using
49:42hierarchical type of agents, too. And
49:45then there's also one is called
49:46blackboard. It's basically agents will
49:48contribute to and react from a shared
49:51blackboard kind of workspace. We have
49:53separate workspaces. Basically, it's a
49:55bit more loosely driven, a bit more like
49:57the event-driven type of collaboration I
50:00talked about yesterday, too. It's
50:02basically
50:03they can have a workspace and then talk
50:05about what needs to be done and you can
50:07use a messaging kind of infrastructure
50:09to help too for messaging like pops up
50:11type of architecture. So, that's kind of
50:14what it is. And then also another one is
50:16interesting is called market base. It's
50:18a bit more competitive because you are
50:20allowing agents. They are autonomous
50:22participants in this system and they
50:24will actually be negotiating and bidding
50:26and competing for tasks too. So, that's
50:28kind of an an interesting kind of
50:30concept in here as well. So, okay. So,
50:32that was the one thing I forgot about
50:34that before I go to the
50:36example. So, in my example here
50:39it's basically
50:41less about all of these patterns yet
50:44because I do have to admit I didn't have
50:46enough time. I I came from
50:48This is a actually a new talk and then I
50:50became from
50:52Frankfurt too. I was in the another
50:53conference called J Con. It's a Java
50:55conference and then I was doing mostly
50:57my event-driven talk, but I also was
50:59developing this and then I was running
51:01out of time. I have to admit because I
51:03was kind of travel a lot of travel for
51:05me past few weeks. So, pardon me, but I
51:08wanted to quickly show to
51:10this particular example.
51:12Okay, let me see. Why is it
51:14doesn't Okay. So, over here,
51:17but this is the the read me. I I will
51:19publish my GitHub short shortly too, but
51:22I just haven't check in yet, but
51:25Okay.
51:26For some reason it's not uh
51:29Yeah, dependent Oops.
51:31Up. There There we go. Okay.
51:34So, basically I I just have a simple
51:36example. It's just event-driven alert
51:38intelligence demo. So, right now, can
51:41you see okay then or maybe I can make it
51:43a little bigger.
51:46You can see okay, right The the fonts?
51:48Okay. Okay. Okay. Yeah. All right. Okay,
51:51good. So, okay. So, this one too again,
51:53I just wanted to experiment, right? Like
51:55event-driven with agents. And this one
51:57again, not [clears throat] like too many
51:59agents in in this particular case, but
52:01essentially using an alert sender. So,
52:05less of calling it an agent, but you can
52:07call it an agent. So, this one will be
52:09the producer that actually will send out
52:12information. I'll act as um uh let's say
52:16a problem or a reporter, like a feedback
52:19reporter that that comes comes in and
52:22publish some request. And then I
52:25actually will then have an analyzer. The
52:27So, the analyzer is the one that
52:29interacts with LLM is a AI enrichment
52:31service. And then also I have a
52:33notification service. So, that's another
52:36is basically will notify whoever needs
52:39to to get the notification, too. And so,
52:42essentially that's just how simple it
52:44is. And basically it it is a Java 25
52:46project, event-driven demo using Spring
52:48Boot services, Kafka, Pub/Sub, and a
52:51local Ollama model in this particular
52:54case.
52:55And so, the thing is because I for demo
52:57purpose I didn't I don't make it too
52:59complicated. So, I'm using local Ollama.
53:02If you're familiar with Ollama is local,
53:04you can run your model locally.
53:06And then also with Kafka in this
53:08particular case I'm using Kafka Kafdrop.
53:11And that one is also like a I have
53:13Docker image and then can bring it up
53:15Kafka local, too. So,
53:17but essentially too, if I run it is
53:19basically I can do Docker compose and
53:21bring up my Kafka topic and it will
53:23listen on 9000. It will be the the one
53:26that's, you know, receiving messages.
53:29And then on my Ollama side I can just
53:31start up my Ollama serve and then pull
53:33in the Llama 3 in this particular case.
53:35And then I can then run Now, again there
53:38are three components to this. I have a
53:40alert sender. So, I have a Spring Boot
53:42app that publishes messages and these
53:45are just text messages and they will
53:46describe a problem. For example, like in
53:49this particular case, I can test it. I
53:51call it water leak, you know, some is
53:53found in a building in certain building
53:55B and all that
53:57and near whatever like that. So, in this
53:59particular case, I'm using then I can
54:01actually use an alert and analyzer that
54:04will actually work with LLM because it's
54:06natural language is a spoken language,
54:08so it can analyze as an LLM to figure
54:10out oh, what is the complaint in here?
54:12So, then it will then come back to me
54:14and tell me, "Okay, there's water leak."
54:16So, then it should know to notify the
54:18facility
54:20you know, coordinator so they can look
54:22at the problem. So, it's just very
54:23simple. And then the notification
54:25service is the one that is another
54:27Spring Boot app that will actually be
54:29running the notification to send
54:30notification to
54:33to who needs to be. So, so that's kind
54:35of just how simple it is to and the
54:37alert center basically send the messages
54:40to this topic called alert raw and then
54:42the analyzer again consumes this event,
54:45right? And analyze it and figure out
54:46what it is and basically using Ollama
54:49interact with my LLM and then the model
54:51will will then return, you know, invalid
54:54JSON or if the analyzer fails to fails
54:57back all of these things too. And
54:59essentially too, the notification
55:01service will then consume this alert and
55:03rich you enrich the messages and then
55:06get sent to the center whoever needs to
55:08be notified. So, that's kind of just
55:10simple very simple kind of cases, but it
55:12makes yourself event-driven kind of
55:14techniques too. So, we'll kind of just
55:16really quickly
55:18look into this example. The First of
55:21all, I'm I talked about why is it not
55:24here? Okay, here. The sender.
55:26Yeah, can you Actually, I I want to make
55:29it a little bigger. I think there's some
55:31some ways of view.
55:34Let me make it a little bigger.
55:40How can I make it bigger?
55:42Um this is IntelliJ, but I thought I
55:44could just
55:46number
55:47Huh?
55:48Command plus.
55:49Plus, right? That's what I thought.
55:50That's what I was doing. But for some
55:52reason, it's not doing. But I just
55:54thought I should make it a little bigger
55:55for you. But I'm sorry if you can't see
55:57it too good. But I will share with this
55:59I'll I'll be checking in. So,
56:01over here, but I can quickly kind of do
56:03it, too. I have my Java file
56:06um that basically you can look at
56:09a tell
56:10sender.
56:11Yeah, again, I'm so sorry. I wanted to
56:14try to make it bigger. But anyway, so
56:15it's a Spring Boot app and is again just
56:18standard alert sender application. And
56:21then I have my model that basically
56:25my topic. I I named it raw alert.raw.
56:29And then the raw alert is basically the
56:32another record that just kind of you
56:35know, have information about the event
56:36ID, the correlation ID, what is it
56:39created at, and source system message,
56:41and reported by some
56:43kind of basic thing. And I look at that,
56:45right? And then the request that comes
56:47in. The request will have the source
56:49system, the message, and the report by.
56:51And then
56:52and then the alert provider in here, the
56:55producer, I should say. So, it's a Kafka
56:57template and I basically will produce
56:59and then essentially just get this Kafka
57:02template and I publish this and
57:04publishes to the topic called raw. And
57:07then it gives the alert ID and the
57:10the information and the controller.
57:13Same in here. It is basically too. It's
57:15very simple, right? It just have that
57:17message like that. And so so this is my
57:20sender. And then over here is my uh
57:25Okay, sorry. And example here, I have my
57:29analyzer. So, my I have my analyzer in
57:32here and then I also have my service,
57:34too. So, service in here
57:37as you can see it's just have an enrich
57:39alert that will analyze it. So, this in
57:43this particular case I have my client
57:44that takes in the prompt and you are an
57:47alert triage resistant and read the raw
57:49alert and return only valid JSON from
57:52these exact fields, right? All of these
57:54fields are severity, category, location,
57:57crucial info, audience recommendations.
57:59Severity, which can be must be one of
58:02low, medium, high, and critical.
58:05Critical info and audience must be a
58:06raise of strings and keep category
58:08uppercase with underscore and do not
58:11wrap the the JSON in mark markdown.
58:13That's what my message is.
58:15And basically over here to the this one
58:19and is basically, let me see, over here.
58:22And and then yeah, so some of these are
58:25just the different helper
58:27um
58:28messages in here, too. But, I also then
58:31will kind of identify the severity based
58:34on the type, you know, the text if it's
58:35contained. I'm identified, too, if if
58:38the text has electrical or fire that's
58:40severe. If there's leak and water,
58:42they're high, but it's not, you know,
58:44it's a just an incident and like that
58:46and you know, so on so forth. So, so
58:48again, this is just my um
58:51you know, kind of my my service element
58:54in here. And then I have my model, too,
58:56and the messaging
58:58and the controller and the config, too.
59:00So, my Kafka, too, will already been in
59:02here that I have configured my topic
59:05alert raw, but there's also alert enrich
59:08is after I analyze the the input, then I
59:11basically determine, you know, from my
59:13LLM that what the problem is and then it
59:16will then generate that
59:18notification over to send to the
59:20consumer, too, at that point. So, so
59:23that's what that's what the this this
59:25works and I do realize I'm running out
59:27of time, but I do have to say I haven't
59:28got time get to the point of actually
59:30able to
59:32uh run it yet. So, but I again will
59:34share the GitHub so then you can all get
59:36to it, but I just want to point out just
59:38three simple services and then of course
59:40there's also the notification service in
59:42here as well that you can see as again
59:44these are Spring Boot application. I
59:46have notification service that uh
59:49handles the routing part um in here and
59:53also
59:54the subscriber registry, for example. It
59:57is true. Sometimes when we do Java, we
59:59tend to have a lot of classes and handle
1:00:01different things, right? The types of um
1:00:03all of the messages, but it's very
1:00:05organized, too, you know, I I do have to
1:00:06say. So, there's also the enrich alert
1:00:09consumer, that type of stuff. So, okay.
1:00:12So, over here, too, if you look at your
1:00:13Docker um kind of uh example, the YAML
1:00:16file that will just start up with my
1:00:18broker on here on listening at 1992 and
1:00:22uh so all of these things and uh yeah.
1:00:25So, so that's what it is and so you can
1:00:27actually basically try to run it and
1:00:29again
1:00:30um
1:00:31let me kind of go go back to my readme
1:00:33file.
1:00:34Where is it? My readme? Nope. Uh set up
1:00:38here. Yeah, so if you kind of do the
1:00:40readme, then you should be able to then
1:00:42run it and then start up your Olama and
1:00:44then uh basically then
1:00:46have your alert sender send run run that
1:00:49to send your messages and use the alert
1:00:51analyzer to analyze the input and then
1:00:54use the notification to basically notify
1:00:57your those who are interested in it. And
1:00:58if you need to test it, just use a curl
1:01:00command to send the messages over here
1:01:03and see what happens. And so, that's
1:01:05essentially this particular example
1:01:07making use of uh event notification and
1:01:10and uh Kafka type type of topic to
1:01:12illustrate how it would work, too. And
1:01:14it's not fully like all agent driven,
1:01:16right? In some sense, but still we are
1:01:18combining different technology together
1:01:20to kind of do what we need to do. So,
1:01:23they they um the LM part to help to
1:01:25analyze things and we still use Kafka
1:01:27topic to handle the events. So, so
1:01:30that's the idea. But again, from here I
1:01:32will I'll put it onto the slide deck
1:01:35that particularly the GitHub once I
1:01:36check it in then then you can take a
1:01:38look at the code and we can talk about
1:01:40it too and my information is there if
1:01:42you want if you're interested yeah,
1:01:43contact me and we can maybe do a twitch
1:01:45stream together something like that. So,
1:01:47yeah. Okay. So, yes and I know it's time
1:01:50is up too. So, but thank you very much.
1:01:52I do appreciate you attending my call
1:01:54and my calls
1:01:56my presentation that he sometimes is
1:01:58doing too much of virtual thing. So,
1:02:00thank you so much and I appreciate and
1:02:02please let me know what you'd like to
1:02:04hear more and what I can help you with
1:02:07in in your work too. So, thank you.
1:02:09Yeah, thank you. Yeah.
1:02:16>> [music]
1:02:24[music]
1:02:31[music]