Full transcript
0:00So,
0:01as opposed to most all of the
0:04presentations you will see
0:06at this conference, this one is not
0:09nearly as polished
0:11or prepared
0:13and probably won't make a lot of sense.
0:16So, just apologizing ahead of time. Just
0:19a really
0:21unorganized
0:22person.
0:24As Stefan knows. So, we'll make it fun.
0:27Interrupt me whenever.
0:29I've done a lot of
0:31kind of building
0:32and
0:34playing around. So,
0:37just ask me questions and and we'll have
0:40a good time. Okay? But if you need
0:42something really well polished and it's
0:45going to bug you if I don't know what
0:48I'm really
0:49like which way I'm going on a slide,
0:51then maybe find another session.
0:55All right. Anyways, so who is this? This
0:58is me, Thomas Dyer. Who's ever talked to
1:01me, heard about me? Okay, great. Pretty
1:03good number. I've been at InterSystems
1:05like 7 years.
1:07And now my role is in the developer
1:11relations
1:13department. And I am manager of our AI
1:16platform and ecosystems. My main role
1:19now is to help customers and partners do
1:23AI
1:25with InterSystems on our tech.
1:27And I was a product manager for vector
1:30search and integrated ML before this.
1:35So, have been around the kind of machine
1:38learning and AI space for for quite a
1:40while and around InterSystems for a
1:42little while. So, happy to talk to
1:45anyone about their projects and how they
1:48can
1:49kind of make life better with AI,
1:52which is not necessarily a foregone
1:55conclusion.
1:57So, I just saw this today.
1:59Kind of the cost of compute far beyond
2:01the cost of employees.
2:04Um, great.
2:07As we're seeing a lot of money going
2:10into AI, it may be crowding out people.
2:14And so, uh, as well, this, uh, data
2:19center
2:20um, kind of explosion.
2:23Uh, that's a nice one. 7,000
2:26people in a town, they planned a whole
2:27bunch of data centers. They wanted
2:30adding 51 Walmarts. This is nuts.
2:33Uh, and so there will be some changes in
2:36the world, right? And, uh,
2:38agentic AI is part of like what this
2:41hype is about, um,
2:43because really agents are and the models
2:46are getting so good at
2:48taking instruction and working for a
2:50long time on something
2:53that, uh, a lot of investors, I think,
2:56see that they could potentially get rid
2:59of those pesky humans.
3:01Or at least give them a new job title.
3:04Uh, maybe it's agent operator is going
3:06to be the new
3:08thing, not AI engineer or prompt
3:10engineer or data scientist or whatever
3:12you want to call them.
3:14Sit there and babysit all of the agents
3:16that are taking in on the left-hand side
3:18all of your data, all of your activity,
3:22and deciding what to do with it.
3:24Research,
3:25report, all that kind of thing. So,
3:28interesting times.
3:31And really, where did this like kind of
3:32come from? It's basically in the last
3:34year, this is my take on it. Uh, there
3:37was a really nice blog post that kind of
3:40started, you know, uh, this, uh, last
3:43April, uh, from Anthropic. And then
3:48they were talking about like let the let
3:50the model kind of figure things out. Uh
3:53that was when they were first getting,
3:56you know, kind of smart enough to do
3:57that. Before then, uh the models would
4:02take like some memory, like you give it
4:04some memory of what it had done before,
4:07and say, "This is what worked in the
4:09past. Uh
4:11use this as a pattern." And the models
4:13would just like kind of repeat it back
4:15verbatim. They didn't really know what
4:16to do
4:18with that context. And of course,
4:21context is all the power, all the all
4:24the thing that matters for LLMs.
4:27So then, there was a jump in the
4:29training methods,
4:31and also this kind of idea of agentic
4:36use of where the models can call tools.
4:40They can A tool can be a file read or
4:43something like that, and they can take
4:45this information that they get from
4:47those tools, and they can really do
4:49stuff with it. But still, uh I don't
4:52know if uh how many people have heard of
4:54the movie Memento, or seen it.
4:57So this guy has a brain disorder um that
5:01uh you can't create new memories. And so
5:04he's trying to solve this uh murder
5:06mystery, I think, of his wife or
5:07something like that. And the only way
5:09that he can like make progress, every
5:11day he wakes up and it's like a whole
5:13new world. He doesn't have any memories
5:15that he's
5:16uh supposedly, you know, normal people
5:19develop over over a day, and then
5:21they're able to kind of remember. He has
5:24to He's figured out he has to actually
5:26make tattoos, and then he reads the
5:28tattoos to to figure out what he uh
5:31needs to do for the day. So that's how
5:33LLMs are. They still don't have memory
5:36inside of them. They have to use scratch
5:39pads. They have to basically write
5:40tattoos. They write files, and then they
5:43can read from the files. And so then
5:45that allows them to progress, but they
5:47are dependent on that external world.
5:50Um
5:51and really uh that's become this kind of
5:54files or memory and skills. And skills
5:59are like memory or like
6:01uh instructions to the to the agent
6:04maybe from a past self of how to do
6:07something and they can be collected over
6:09time and that's really like what humans
6:12intelligence and cultural evolution
6:14really is all about.
6:15Is we've externalized all of our
6:18experiences and then everyone can learn
6:20from them and they get shared.
6:22Um and it's uh it's quite interesting uh
6:25to see this uh get recapitulated
6:29uh essentially within agents and within
6:31AI.
6:32Uh they're becoming cultural a a actors
6:36um
6:38as we as they progress.
6:41Any uh any questions or comments? Like I
6:43said, I wanted to make this uh
6:45interactive because maybe you can fill
6:46in some gaps for me, but uh
6:49uh I find this uh completely fascinating
6:52world. There there was also um uh an
6:55interesting paper couple papers by
6:57Google
6:59showing that the way that the the
7:01training methods have have evolved in
7:03the way the thinking. So right, you just
7:05kind of let the models talk to itself
7:08and that's kind of the thinking.
7:11They get better if they are
7:14talking amongst themselves essentially
7:17setting up you know, kind of devil's
7:20advocates for argumentation and letting
7:23different uh trains of thought run and
7:26argue with
7:27itself within the same model within the
7:29same
7:31uh trace. So that's part of that kind of
7:34cultural evolution of the way that uh
7:37the way that uh intelligence works.
7:40And the way that uh we all evolve.
7:43Very interesting stuff.
7:45But wait.
7:47We've already had some of this. So, I
7:49wanted to bring out this is kind of an
7:50interesting thing also that you see
7:53actors. And the actor pattern is
7:55something that
7:56is something from a while ago in
7:58computer science.
8:00Who has heard of Erlang?
8:03Right. So, that's a very famous
8:06programming language that that
8:08personified the actor pattern,
8:11which is just a way to
8:13kind of separate the acting from the
8:18um
8:19from the memory of from from the from
8:21the system and and do that in a in a way
8:24that that uh
8:25that gives you very deterministic
8:27results. Well, the interoperability
8:29system within Iris is the actor pattern
8:32implemented. And so, all of these kinds
8:35of things in AI
8:37are really
8:39you can see them as echoes of things
8:41that we've been doing for a long time.
8:44Of course, we've added vector search.
8:47And there at the bottom is something
8:49that I've been working on, which is a a
8:51graph database, a graph engine. It's
8:53open source. It's not product. It may
8:56never be product. But it is something
8:59that is really uses Iris and and
9:01integrates not only vector search
9:04but graph
9:06graph database
9:09perspective. I'll talk a little bit more
9:11about it.
9:15So really,
9:16you know, all that stuff that we've had
9:17for a long time
9:20Iris interoperability
9:22and you know, new multi-agent
9:24frameworks, there's a big overlap. But
9:27the thing that's new
9:29is of course the LLM brain. So, I'm not
9:31taking anything away from that. It's
9:33really amazing and it really does make
9:36uh, make it possible to fully automate
9:39stuff.
9:40The other kind of perspective that we
9:42have is, uh, kind of a data fabric or
9:45just kind of having all of your data
9:48together and doing your,
9:50um, your governance on that data
9:53together.
9:54Uh, that really, um, works well with
9:57kind of the agent framework and enables
9:59and this is kind of an ingredient for
10:01enabling multiple agents to act
10:04together, uh, on the same data sources
10:08and, um,
10:09and, uh, collect that information, build
10:13on that information, create memories,
10:16and create new, uh,
10:18new data and then feed that loop and
10:21keep going around and around. And we're
10:24going to be
10:25as, uh, Inner System we're we're
10:28building more and more capabilities for
10:31agentic AI on our existing kind of data
10:35platform.
10:36And to that we have now an AI Hub. Uh,
10:39who here has heard of
10:42the AI Hub?
10:43Okay, quite a few. It is in early access
10:46program now. Um, Benjamin Dubois is the
10:50product manager.
10:52Um, there is it's really an open EAP
10:55now. There's actually GitHub repository
10:57with a lot of documentation on that.
11:00And, um, really it's all about kind of
11:03enabling Iris to act within these agent
11:08environments as act as an agent
11:10environment as part of the agent
11:11environment. So, on the right hand side
11:13it's kind of like an AI SDK that allows
11:16you to easily set up connections to LLMs
11:20and other systems like MCP server. So,
11:23MCP stands for model context protocol
11:27and it's really just a USB plug for all
11:30agent tools. Instead of every agent and
11:33every tool having like a point-to-point
11:36connection, it's a standard so that you
11:38can just plug into an MCP server that
11:41acts as a central hub and gives access
11:44to tools.
11:46Um and so in the middle, of course,
11:48you're going to be able to within Iris
11:51uh make tools to do some retrieval
11:53augmented generation or some custom
11:55agents with the
11:56uh with uh object script or or even
12:00Python now, other languages.
12:03On the right-hand side, there's also
12:06an MCP server. So, Iris is will act as
12:10an MCP server.
12:12It's actually a little sidecar that sits
12:14outside of Iris at this point and
12:18connects to Iris, allows you to inspect
12:21or turn any object script
12:24class or function into a tool.
12:26So, then any agent that comes from
12:29either your cloud desktop user and you
12:31just uh can use those tools and expose
12:35as a developer expose any act uh inner
12:38um
12:39functionality that you have in Iris as
12:41an MCP tool and allow it to
12:45uh partake in all of the cool fun stuff
12:48that all the cool kids are doing.
12:51Uh so, you can imagine within Iris just
12:53all of these capabilities becoming
12:56agents. And you'd have multiple
12:59agents.
13:00Any questions on that?
13:03Anything?
13:04Come on.
13:05Nothing? Yes, please.
13:16At at this point, it's really kind of
13:18the core that is Well, so so the road
13:22map is that there will be uh official
13:25tools that will come with the kind of
13:28MCP server. So, there'll be things that
13:31really kind of expose
13:33um you know, in a in a very, you know,
13:36standard way like how to
13:39how to interact with Iris. Be SQL.
13:42Be all the classes and then just have
13:45some good tools that that get exposed by
13:47the MCP server.
13:49That the agents are also things that um
13:53will have plenty of examples because
13:55that's part of the object script um
13:59class hierarchy is like percent
14:02AI.agent.
14:04And so, it gives you that.
14:05And then we'll have examples and we may,
14:08in addition to just plain MCP tools,
14:11we'll have agents, but that's road map
14:14kind of thing.
14:27Yeah.
14:33That That That's very interesting. Makes
14:35sense. You know, it just That's kind of
14:38part of I think what all what I'll be
14:40driving at here is that that's
14:42that's kind of
14:44It's that It's that gap thing. Gaps
14:46happen because
14:48partially you want to make
14:51each agent have a particular role in a
14:55particular kind of constrained ability
14:58so that it's not getting confused as
15:02easily. Cuz still it's like context
15:04management, context engineering. You
15:06want to keep the role very crisp and
15:08defined and that can cut down on
15:10hallucinations.
15:12And then if you have multiple of those
15:14agents that have little gaps and they're
15:16talking together, but each of them don't
15:18hallucinate,
15:19maybe you get magic then. Um yeah, I
15:22think I get your drift and I think
15:24that's
15:25that's it's brought on.
15:28So, you know, the whole idea of the AI
15:31hub is not only is, you know, as always
15:34kind of mentioning,
15:36enabling AI development on Iris,
15:38accelerating your agent development,
15:42but then also
15:45enable making Iris something that that
15:48is able to be uh, used by other tools.
15:56So, uh, kind of overview of what you can
15:59do, uh, what the AI hub is now is
16:03unified tool calling uh, within Iris.
16:06Object script, Python, or even Rust is
16:09underneath, but of course within the EAP
16:12we're really focusing on object script.
16:15Um, eventually we'll we'll kind of
16:17broaden that out. There's an MCP server,
16:21um, an MCP client, uh, coming soon.
16:25There's, um,
16:27a policy layer so that all the roles
16:30that you define within Iris can then it
16:33can be exposed or used as enforcement
16:36and I'm going to talk a little bit about
16:38that.
16:39Uh, have LangChain and LangChain for J
16:42um, kind of integrations.
16:45And so, what you can build with it, tool
16:49using agent with auth authorization and
16:52auditing and just a
16:53few hundred lines of object script.
16:56Policy gated tool exceptions so you can
16:58set up policies for your agents and uh,
17:02gate the tools that they can use based
17:04on those policies.
17:06Uh, and expose all that, um, by MCP so
17:10that makes it very,
17:12um,
17:13very useful to make a uh,
17:15an interoperable kind of pluggable
17:17solution together.
17:21So, um so then I'm going to switch a
17:23little bit to to kind of things that
17:25I've been building.
17:27Um so I built a
17:29a graph engine as I mentioned.
17:31And uh just uh recently I made it so
17:34that it's open Cypher compliant. So what
17:36that means is that there is an uh Cypher
17:38language which is very much like SQL,
17:41but it allows you to just define kind of
17:44uh uh
17:45more graph related uh queries. And
17:48there's an open standard for it. And um
17:52my graph engine um can support all of
17:56it. So another popular graph database is
18:00Neo4j and that's that's one that also
18:02supports most of open Cypher. Uh so any
18:06kind of graph applications that you have
18:08should be easily able to be ported. Um
18:11it's very fast. It can also integrate
18:13our vector search as part of the graph
18:16um engine which makes it very good for
18:19agent things. You often have memories or
18:23complicated
18:25uh
18:26you know, structures that you want to
18:28store with agents or have agents walk
18:31over graphs of memories and things like
18:33that. You'd also want them to be able to
18:36do vector search. Now you have a toolkit
18:38that that puts those together.
18:40And it's very fast.
18:42Um
18:44I've been doing some benchmarking and
18:46it's it's
18:48it's comparable to most of the stuff
18:50that's out there. And that's mostly
18:51because of the way that Iris is um
18:55is architected. And so
18:58general flow of the whole thing is that
19:01you have a a query parser on the left.
19:05Um it gets translated. And that's all
19:07done in in Python, but it gets
19:09translated mostly to SQL. Most all of
19:12the graph
19:13um structure is in SQL and that allows
19:17you to use all of the standard and
19:19wonderful indexing that that Iris
19:21provides.
19:22But for things that you can't quite do
19:25uh with SQL that uh makes sense, you can
19:28just drop down into globals and I have,
19:31you know, custom indexes that that uh
19:34that then fill in the gaps for a graph
19:37um that make things like breadth-first
19:39search fast. Um make uh PPR stands for
19:44personalized page rank. It's like a page
19:46rank algorithm that Google of course
19:48made popular. That's something that SQL
19:51doesn't do particularly well. So, you
19:53need to uh do some custom coding there.
19:56Then for really speeding up some things
19:58that still don't do very well with
20:01ObjectScript, we have now in product a
20:04Rust um bridge. It's very similar to our
20:07embedded Python bridge. It gives you
20:09in-process access to between Python and
20:12ObjectScript. Now, there's RZF which
20:15does a similar thing with Rust. How many
20:19people uh either know about Rust or have
20:22actually programmed in Rust?
20:24Here. Hey, some people. Awesome.
20:27So, Rust is a really uh great system
20:29programming language um and um
20:33and I found it it it can be very fast.
20:36So, I'd love to talk more about there's
20:38there's definitely a lot lot in it as
20:41far as how how I've been using it. And
20:43there's actually a tech exchange uh
20:46that's going on this week that's going
20:47to be talking more about that. And I've
20:50given my example of how I've used RZF uh
20:54to the developer that that develops RZF
20:56and he he might talk about about it.
20:59So, these are some of the things I can
21:01do
21:02uh within this graph. Graph traversal,
21:04vector search,
21:06RDF and named graphs.
21:08I've built uh you know, kind of the
21:10ability to do security and
21:12observability. I'm going to talk a
21:14little bit about about that. I think
21:16there's a big uh benefit or there's big
21:19um possibility for um
21:22for using graph with agents. I'll talk a
21:26little bit more about it and show it. Uh
21:28but um
21:30but also for kind of the larger world of
21:34observability.
21:36Um and using graph databases and using
21:40Iris in particular for observability
21:42data and um it just makes sense to have
21:46everything all in one box in some ways.
21:49And when you're dealing with security
21:50concerns, uh if you have it all in one
21:53box, you you you can really search the
21:56whole thing.
21:57And so that really fits with our our
22:01kind of philosophy overall within
22:02InterSystems.
22:04Um so so the example that I'm going to
22:07talk about with the the multiple agents
22:09which I see uh
22:12uh someone here.
22:14Just trying to Enrique.
22:17So he knows a little bit about uh Yoki.
22:19Uh Yoki was an internal project and I'll
22:22just just give a
22:24brief description of it. And I've
22:27actually presented uh it's been about 2
22:29or 3 years now. Uh we've talked about
22:31this. So the idea of Yoki was to give an
22:35intelligent assistant over our IRIS
22:37service, our track care tickets.
22:40And we've gone through several loops as
22:42you see, different uh
22:44use cases,
22:46bringing in different data, using
22:48different tools, LangChain, used
22:51different LLMs, started using Google,
22:54went to Anthropic, and you know, using
22:56more OpenAI models to the right. So
22:59we've gone over several years doing more
23:02and more kind of agentic
23:05application internally to kind of use
23:09the tools, use our own tools, and and
23:12learn about about um
23:14about AI and how to do it.
23:17And so
23:20as
23:22we've developed this, now this gets to
23:23where I'm like, okay,
23:25didn't quite get a good
23:27uh
23:28presentation together, but kind of going
23:30back to that idea about doing your
23:33security and your observability all in
23:35one box.
23:37We're also uh I've been exploring this
23:39idea within the Yoki project to provide
23:43a security
23:45overlay on what the agents are doing.
23:47Each agent having some role, having
23:50things that it can and cannot do.
23:53And then that makes it a really nice
23:55system together. Typically, if you have
23:58all these things separate, you've got to
24:00do some correlation between systems,
24:03you have your authorization system and
24:05your audit system might be in Splunk,
24:08and you've got to figure out how to put
24:09those together.
24:11Um and then eventually, you know, the
24:13data system is where things have been
24:15happening. That's where transactions
24:16happen, but there's some latency to get
24:19that that stuff into Splunk, and then
24:21also um
24:23right. So, there's definitely
24:25problems with separating everything. It
24:27can be. And so, if you just have your
24:29domain data, your identity and your
24:32roles together, and all your audit
24:34trail, it's uh it simplifies life. And I
24:37think it could be more secure.
24:42So, back a little bit to the memory
24:45concept as well.
24:47Uh all these agents, one of the the the
24:49great things about agentic AI and about
24:53how uh you know, as agents are working,
24:57doing things, they can record what they
25:00do,
25:01and then you can mine that data at any
25:04point. You can have the agent look over
25:07its past history and determine if it
25:11should do things differently this time.
25:13Um and so that in in kind of a
25:16psychology
25:18frame of mind, these are different forms
25:20of memory. You have your working memory,
25:22this like your instant your context of
25:24like right now, what's going on, uh and
25:28what you're doing right now. You have
25:30episodic memory, you might want to
25:32commit some memory to a store, like a
25:35vector search would be nice to then be
25:37able to search over past memories.
25:40Then
25:41you can do a little bit longer-term
25:44mining of that of all of those episodes
25:47and the evolution of working memory
25:49that's in the past,
25:51and determine, okay, what do I know
25:54about this system? What maybe can this
25:58all of this that I'm seeing together
26:00tell me about how this works.
26:03Um and develop that kind of semantic
26:06knowledge. It's not specific to a
26:08particular point in time or a particular
26:11experience, it's kind of like just you
26:14knowing about how the world works and
26:16how to do things and those
26:18procedural and semantic are that kind of
26:20memory.
26:21Um
26:23and down in the bottom I see that
26:24there's definitely some kind of new uh
26:27some papers that have come up that that
26:29that kind of take that perspective and
26:32that improves search, improves agent
26:34behavior over time by developing that
26:37memory, by remembering things and then
26:39mining it, using the same large language
26:42model capability to reason
26:44by reasoning over past things that it
26:47sees.
26:48Any uh questions on any of this so far
26:51or comments.
26:54It's
26:55So,
27:03So, one of the ways that that kind of uh
27:06gets applied then to the Yoki project
27:10is that if you have agents
27:12that are helping our support advisors,
27:15which is one of the use cases that we
27:17were thinking about,
27:19you can have agents remember particular
27:23interactions with particular advisors,
27:25what they're doing, what their
27:27particular problems are, the customers
27:30that they're interacting with. That's
27:32going to be slightly different. We have
27:34such a wide range of customers and uh
27:38problems that that go with them because
27:41you might have a customer that that has
27:4320 hospitals or you have a customer that
27:46just has one Trackster installation. Uh
27:49that's just the context that that can
27:52make the difference for an advisor to
27:54see something learning for itself,
27:58learning for that advisor, and making
28:00his life better by really understanding
28:03that context.
28:05Then, across advisors, you can have um
28:09often times you need to hand off
28:11different shifts, and you want a
28:13consistent kind of view, and you also
28:17can then develop memories and and
28:19capabilities in that way.
28:21And then the agent itself, if you have
28:23multiple agents running and they each
28:25have their own particular role and their
28:27particular memories, they might work on
28:30at different customer sites, so to
28:32speak, uh or you can separate those, it
28:35depends. You have a lot of options
28:37there, but then it can evolve and get
28:40better at things that it does by looking
28:42specifically at it. So, all of these
28:44kind of memory
28:47um
28:48perspectives are really important. And
28:51it's uh it's just a a kind of a magical
28:54thing of being able to
28:55um make uh AI do better
28:59by using its own capabilities.
29:04Um so, what I had uh worked on
29:08is
29:09kind of putting a bunch of different
29:11agents in a role
29:14uh in different roles. And you know,
29:17that you'd have like maybe this uh
29:19question you're coming at the top. Find
29:21me laboratory error tickets. And walk
29:25the graph of look over my graph and find
29:27the top result. Find top related
29:29tickets.
29:30And make sure you don't uh use any uh
29:33PHI. Actually, this is kind of system
29:36prompt that comes in. So, one of the
29:39things that's uh interesting within uh
29:41TrackCare is that you need to be careful
29:45that there's no PHI in a ticket. Often
29:48times, it can happen. It's not supposed
29:51to be there, but uh patient information
29:54can be there. And so, we had to take
29:55great pains to to uh strip it out
29:59anytime uh that might be the case. And
30:01then only show a public LLM or even um
30:05you know, an uh enterprise kind of LLM
30:07that we would have um you know,
30:10very secure access to still would not be
30:14um
30:14kosher for there to be any PHI to be
30:17sent to it. So,
30:20typical run would be this you know,
30:23thing. The AI would be thinking about
30:25it. It would decide to use the semantic
30:26ticket search and do a vector search
30:28over laboratory errors. It would get a
30:30bunch of results from that. It would
30:33then walk It would be thinking about
30:35doing that graph walk and and looking
30:37over the the graph database where a lot
30:40of different tickets were associated
30:42with each other
30:44uh to find related tickets.
30:46And then you could find eventually these
30:49kinds of
30:51results to be like, well, this
30:53laboratory, this module affects this
30:57ticket
30:58exhibits this error typically, you know,
31:01and so there would be relationships that
31:03we would
31:04that would get mined.
31:06And eventually go through all of these
31:10uh
31:10these kind of steps. All right, well, I
31:13hope
31:14I hope you have a good
31:15ready and
31:17like I said, I'll be at the tech
31:19exchange most all week.
31:21And happy to talk to you about any
31:23projects and things you want to work on.
31:26Thank you.