AI Agent Orchestration: How to Automate Enterprise Workflows Safely

0:00Hi everyone, and welcome. My name is

0:02Rutuja, and I'm a senior product

0:03marketing manager here at Box. Today's

0:05enterprises aren't automating existing

0:07workflows. They're creating new ones

0:09with agents that can read, write, and

0:11act on secure content. Today, I'm joined

0:13with Nirmal Ganesh from Box and Aaron

0:15Diamond from AWS to talk about

0:17automating agent orchestration. It's

0:19great to have you both here with us.

0:22My first question to you, um, both is

0:24when your team say AI agent internally,

0:27what are you describing? Is it the same

0:29thing? Is it different? What does your

0:31definition depend on?

0:34>> You want to do the first?

0:35>> That's [clears throat]

0:36sure. No, it's great great to be here.

0:38It's an It's an interesting question.

0:39When when we say we give people a lot of

0:41latitude to let them define it. Uh, I

0:44personally I think we personally have

0:45some ideas, but more importantly, we

0:47want we want people just creating and

0:49doing and experimenting and living it.

0:51So, we give them a lot of freedom, right

0:53or wrong, to define it.

0:54>> Yeah. And that's very synonymous with

0:57how people see it as well. If you ask an

0:58engineer what an AI agent is, it'll

1:00likely be an LLM model with tools. If

1:03you ask a lawyer what an agent is,

1:05they'll say, "Hey,

1:06it gets my job done better."

1:09But in general, how we see AI agents,

1:12uh, are are somebody

1:14is an agent who has an objective, who

1:17has access to like a knowledge base, and

1:21has the ability to plan, act, access to

1:24tools, and drive like multi-step tasks

1:27without human intervention. So, at Box,

1:30uh, we kind of see them as more as

1:33uh, a player in a workflow step who has

1:35access to govern content, uh,

1:37who can reason,

1:39>> [clears throat]

1:39>> uh,

1:40with guardrails, you can who can reason

1:43with like limits, uh, and then take more

1:46downstream actions which are,

1:49uh,

1:50driven through confidence, driven

1:52through governance, uh,

1:55and and we see them as

1:57like a key player in driving decisions

2:00without humans. That's how I think we

2:02see uh

2:03agents in general.

2:05>> Yeah, so I I like your definition. I

2:06think we see it similar. We

2:08we constantly debate do we use the word

2:11automation or not? And is automation

2:13help make it better? We even sometimes

2:15have even taken the word agent out. Um

2:19but as long as people are doing that,

2:20we're we're we're pretty happy with the

2:22end result.

2:23>> Yeah. And sometimes it's a little

2:24confusing here, what's an agent versus

2:26here, what's an automation, right? Like

2:28is is automation a multi-step agentic

2:31workflow which just chains one agent to

2:35another or is it like one agent that

2:37does everything? And the definition kind

2:40of varies based on what your use cases

2:42looks like, how are you orchestrating

2:44them, and so on.

2:46>> Yeah, that's phenomenal.

2:47>> Awesome. I guess maybe switching gears a

2:49little, something that we find maybe

2:51might be synonymous with the word

2:52automation is orchestration. When you

2:54picture the ideal multi-agent workflow

2:57for something complex with an enterprise

2:58use case at AWS, Aram, you know, what

3:01does each of your organizations need in

3:03order to contribute to make that

3:05end-to-end workflow work?

3:07>> Yes, I I I can speak both to AWS and

3:10then to our our customer base in in

3:12general. And so

3:14just like you said, people want to

3:15accomplish a task. They they want to to

3:18do something. And so in the end, take

3:21all the buzz out of the IT is what what

3:23problem am I solving? What am I doing?

3:25What's the day in the life of my job,

3:28the week in the life of my job? And so

3:30that orchestration layer can be

3:32everything from top of the funnel, I

3:34have a question, down that I want to get

3:36a task done, and then end-to-end. And so

3:40again, automation, orchestration, if

3:42you're solving a business problem,

3:44that's what matters. For us

3:46specifically, we're playing around with

3:48it. Um we're using a a with CRM data.

3:52Uh we're going in and we're finding

3:53things, things that would be stagnant or

3:55static in CRM. Like, what do we actually

3:57do? How do we find something that's

3:58blocked or an escalation? And then we

4:01use a lot of content. It's a big doc

4:02writing culture. So, we put everything

4:04into documents. We have mechanisms. And

4:06so, we want that orchestration

4:09to go get us to that mechanism where we

4:10can go review the business and then take

4:12action on it. So, taking action is

4:14everything. We always want an action to

4:16come out of whatever data we're looking

4:17at.

4:18>> I think for us it's very similar to,

4:20right? Like how you mentioned about,

4:21"Hey, you've got customers who want to

4:23get jobs done." For For us,

4:24orchestration is also

4:27how can we help our customers get job

4:30done? So, if if you take if you take a

4:31job like an onboarding workflow or like

4:34a knowledge management use case,

4:36what do you need? You need

4:37infrastructure. You need the tools and

4:40models to convert unstructured data into

4:44structured data. You need uh

4:47tools and capabilities to combine agents

4:50and human beings to get the job done.

4:53And at the end of the day, all of this

4:55together is how is how we see uh

4:58orchestration. And

5:00we want to be able to do it in a way we

5:01are able to do

5:02orchestration across multiple agents to

5:05get the job done for our customers.

5:07>> It's so funny you said onboarding. I I

5:09came into AWS about a year and a half

5:10ago.

5:11And my onboarding experience was so

5:13different compared to now because we

5:15didn't have agents right then. We were

5:17just at the forefront of it. And so, now

5:19when we onboard someone, it's a

5:20completely better experience because

5:23that orchestration layer actually meets

5:25them and helps them get something done.

5:26And then kind of nudges them along the

5:28way. Hey, don't forget this. Why not do

5:30that? Um my own onboarding experience,

5:33you you end up having a lot of

5:34questions. Even though you complete a

5:36task, you need other resources to go do

5:38your job. So, I think the onboarding

5:41experience of any company when you use

5:42agents, right, is it's a game changer.

5:44>> I I I it's a very good example to talk

5:47about like how can agents help with such

5:50task, right? Like if you break down what

5:52an onboarding or any other use case

5:54looks like, any other use case which

5:56involves content looks like, right? Uh

6:00invoice processing, contracts. At the

6:02center of everything is like documents.

6:03It's a process. You have an intake of a

6:05document. You read through the document,

6:08extract some data from it, based on that

6:11data apply some business rules on top of

6:13it, a route to the right person, and

6:16then approve, and then you have a

6:17welcome kit. It's a It's a It's a

6:19blueprint for most of the document entry

6:23use cases. And that's where we think

6:25there's a lot of scope for automation.

6:27Like

6:28today models have become very effective,

6:30right? Like

6:31what used to be manual work where

6:33everybody has to see all the employee

6:35documents and then approve, an agent can

6:37get it done faster today and with very

6:40high degree of accuracy.

6:42An agent can drive decisions, too. It

6:45can loop in You can have confidence

6:46level today and say, "Hey,

6:48if the agent can do it with less than

6:52with higher than 60% confidence, auto

6:55approve. If not, escalate it to a

6:58uh onboarding specialist. So, this is

7:00where orchestration sort of plays a

7:02role. Like there's there's good ways

7:04where models are very good today

7:06and there's good ways to combine models

7:09and humans in a workflow where you can

7:11streamline them for higher productivity

7:13with with still very good amount of

7:16reliability.

7:17>> I I don't mind admitting to you and and

7:19the folks listening that one of the big

7:20things I had to do at first was learn

7:22our expense policy.

7:24What [clears throat] can I expense? How

7:25do I do it? It seems so trivial, but the

7:26the traditional way was this expense

7:28policy, which was pages and pages long.

7:30You never really read it unless you had

7:32an issue, and I just wanted to see what

7:34could I do and put that in a workflow

7:35where help me go build my next trip.

7:37What could I do? And so, it sounds so

7:39simple, but no one wants to do something

7:41wrong with expenses.

7:43Our expense policy now and our expenses

7:45in general and our travel is so much

7:46easier.

7:48>> And and and to and it's gotten to a

7:50point where you can actually have your

7:53expense policy as a knowledge base and

7:55say, "Hey." And then ask questions. You

7:58don't have to read through documents.

7:59You can just ask a question. And to a

8:01point where you can even

8:03map it to a business rule. Like every

8:06time you book a travel, "Hey, just go

8:08check my travel policy and see if it if

8:10it's in alignment with the travel

8:12policy. If yes, go book ahead. If not,

8:14just flag it and you can just go and

8:16modify it." It just takes so much of our

8:18effort out of the way.

8:19>> I think every head of HR and finance

8:21just love that we taught that here.

8:23>> [laughter]

8:24>> Yeah, I actually want to pull on a

8:26thread that you talked about there

8:27around like compliance and auditability.

8:29Um as we know, enterprises running on

8:31AWS you know have some serious

8:32compliance requirements and audit

8:34requirements. You know, how do you think

8:36about auditability as like a shared

8:37responsibility between

8:39um you know, the humans in the loop, the

8:40agents, and the stack? And then I guess

8:42like how does Box think about it as

8:44well?

8:45>> Yeah, like I'll I'll answer it. Everyone

8:47always asks me this. They always say,

8:49"How do I not screw up my agentic

8:50project?" So, I'll I'll touch on the

8:52three things and I'm happy to go back

8:53and forth. Like I said, so the first one

8:55is absolutely security. You know,

8:56security matters now. Identity

8:58management security is cool. You can You

9:00can do almost anything wrong except for

9:03break trust on security.

9:05And so, like security is is is a big

9:07thing. So, that's why I tell people

9:08number one. Second is this idea of

9:11flexible workflows. You really have to

9:13think about the day in the life of the

9:15knowledge worker.

9:17Because if you make agentic too brittle,

9:19too strict, how do you know if you have

9:2010 steps today that 20 is too much? Or

9:23maybe the 10 should go to five. So,

9:26people can really blow it with agentic

9:27if they don't know the workflows. And

9:29lastly, they need a cost model, right?

9:31This has to be cost contained. It may

9:34work well, but but if the cost are in

9:35control, it's not worth it. So, full

9:38back with security and auditability,

9:39which was the the first part of the the

9:41question, it is absolutely the core

9:44fundamental thing that matters. And I

9:46think with security and double down in,

9:48it's it's that auditability of who gets

9:51access to that content and do that.

9:53Who's authorized? And the thing is,

9:55right now, everyone's relearning it

9:57because the technology never existed

9:59before. So, no one has ever thought of,

10:02well, he or she can do this or not do

10:04that with it. It was always what could

10:05they access? And now it's what can they

10:07do? So, that's what most companies come

10:10to us when we talk through that. I don't

10:12know if you see the same thing.

10:14>> you're absolutely on target, right? We

10:15call them like governance native AI.

10:18Basically, if a user does not have

10:20access to content, neither should the

10:22agent have access to it. If the user If

10:25the user cannot see something, neither

10:27does the agent agent, right? And that's

10:28one of our first principles for our

10:31agents. Uh so,

10:34and it's very key because that's where

10:36you can build trust with your agentic

10:38models.

10:39I think I really like something you

10:41touched upon is how do we know that

10:44a four-step or 10-step process today is

10:47a 10-step or a four-step process

10:49tomorrow, right? Like, that's where we

10:51think the ability to redesign your

10:53workflows for AI-first world makes a lot

10:56of sense. And because

10:59today, a 10-step process could be

11:01broken/manual.

11:03You You need not add AI to a single step

11:06and make the broken process faster or

11:09manual, right? Like, that's That's the

11:11fundamental thesis behind why you

11:12redesign your workflows. And as you do

11:14it, we've covered the

11:16uh governance native AI. What's also

11:19very important is having an auditability

11:22and trace across the entire AI-first

11:26process, including your workflow steps

11:29and agents, which means you need a trace

11:32of

11:32what's the input. What was the agent

11:36did? Why did the agent make that choice?

11:39What's the confidence level? What's the

11:41audit log across the end-to-end process

11:44is really what gets to a state where

11:47your agentic process is secure and

11:50governed not just for your IT teams, but

11:52everybody. Like we've covered, we've

11:54made sure the HR is taken care, we've

11:57made sure the travel specialist is taken

11:59care. Uh it's also very important for

12:01our security and governance friends to

12:03ensure our entire agent workflow is

12:06secured and governed end-to-end.

12:08>> It's where you know with an AWS, what

12:11we're trying to do is really put out to

12:12the edge where the individual

12:13contributor, no matter what their role

12:15is, can can build things and do things,

12:17not just consume.

12:19And so, it's both culture and security,

12:21but we we have an ops team. We love our

12:24ops team, they're not going anywhere,

12:25but we don't want ops to build

12:27everything and push it down. We want

12:28people to build themselves. That's an

12:29interesting security identity. How much

12:32power do you allow someone to build

12:33something

12:34and work on behalf of others or push it

12:36out and share.

12:38Um

12:38we're letting people go and have fun. I

12:40mean, we're definitely putting

12:41guardrails in, but we want more

12:42innovation and we want to go faster

12:44internally, but it's it's it's a

12:46balance.

12:47>> Yeah, as long as you have the right

12:48security policies and the guardrails in

12:51place, I think everybody uh should be

12:54able to create workflows for either

12:57their knowledge base or their teams or

12:58their departments or across the

13:00enterprises. Uh and that's how I think

13:03agentic workflow should be.

13:05>> Yeah.

13:06You guys both touched on uh maybe like

13:08the background player in all of this,

13:09which is content. Um how much of an

13:11agent's real-world usefulness comes down

13:13to the quality and accessibility of the

13:15content or the context from the content

13:17that they have? And does this show up as

13:19a bottleneck anywhere in the processes

13:21that you guys see day-to-day?

13:24>> Uh

13:26I like to think about does customers

13:29want to access their contract or the

13:31right contract, right? I think that's

13:33the difference between having access to

13:35the right content versus having access

13:37to the content. So, I think about it in

13:39three ways, right? Do we does does

13:43does your content uh have the right

13:45structure for your agents to read and

13:48understand and what to do with your

13:49content?

13:51If they have the if they if they have

13:52the right structure,

13:54uh is it connected with other aspects of

13:58your workflow?

13:59And then finally, is the content

14:01governed, right? Like is is the content

14:04uh has the right governance and security

14:06controls in place, so it's secure. So,

14:08if you combine all the three, that's

14:10when you can drive and a good sort of

14:14workflow. So, in my opinion, the three

14:16principles are is your content

14:17structured

14:19uh for agents to read and figure out

14:21what to do with it. Is your content

14:24governed so that you have access to the

14:26right content, right permissions, right

14:28users?

14:29And then, is it connected

14:32across your workflow so you can transfer

14:34context from one step to the other based

14:37on content and how are you seeing it

14:39across your customer base?

14:41>> I wish I I wish I'd answered first cuz I

14:43don't think I can make that better. But

14:44now, I mean my my um my take on it has

14:47evolved over the last year. I mean, a

14:48year ago I would have told people go go

14:50do something AI, don't worry about the

14:52content, it'll catch up. And I still

14:53fundamentally believe that.

14:56But I think the main answer to the

14:57question is yes, content and context

14:59matters, the data foundation matters.

15:03I think what's happening is now, you

15:05know, these agentic AI use cases are

15:06really showing people what's lacking in

15:09their content.

15:10If if it's dirty, if it's bad, if it's

15:12not the right context, it's really hard

15:14to do any things. And so, when I work

15:16with with companies, I say don't get

15:17discouraged when that comes up because

15:20it will. Embrace it, realize if it

15:22wasn't for the agentic AI use case,

15:24there was a lot of human intervention

15:26that was bypassing what was missing in

15:28the content. So, anyone that's going

15:30down this journey has to be ready that

15:32that they're going to have to redo or

15:33modify their content and the context out

15:35of it, but absolutely matters. And

15:37that's that's what most people struggle

15:39with.

15:40And so, we had an an internal meeting

15:42the other day, and I just used the word

15:43data foundation and use that as sort of

15:45the catch-all phrase. If the data

15:46foundation isn't part of the project, it

15:50will fail.

15:51But if you get it right, it's it's a

15:53beautiful thing.

15:54>> Yeah, and and we also believe so, right?

15:56Like we talk about structured versus

15:57unstructured content.

15:59Uh like we have something called as an

16:01extraction capability, which basically

16:03reads through your content on box and

16:06converts them into very structured

16:08content. It I It updates metadata. It

16:10makes sure you have the right naming

16:12structure.

16:13And what that really unlocks is you go

16:16from having a black box or a piece of

16:18paper on your cloud to more readable

16:21content.

16:22Based on which you can drive decisions.

16:23You can Hey, you can read if if the

16:25contract value is over a million

16:27dollars, you can write policies that

16:29says, "Hey, if it's over a million

16:30dollar, automatically route it to

16:32somebody." Or if it's over a million

16:34dollars, "Hey, please involve a human in

16:36the loop review before you do something

16:38with it." So So you So that really

16:40unlocks what you can do with it.

16:42But that's just not enough. Like like

16:44you need you need to make sure, "Hey,

16:46you apply the right

16:48uh

16:49governance on top of it. Hey, if if if

16:50it's something a human cannot read, make

16:52sure the agent also cannot read." You

16:54know? And then that just unlocks a lot

16:56of value from

16:58a lot of content that you already have.

17:00>> And there's a a feedback loop, too. I

17:02find myself going back more into the

17:04content I already have and re-looking at

17:06it, re-purposing it, re-using it,

17:08modifying it, which I never would have

17:09done if I wasn't in an agentic model.

17:11I'll give you an example. We're We're

17:13doing a customer event at a summit, and

17:14so we have these dinners, and there's a

17:16whole area where the dinner is in the

17:18content and it's not a real

17:20you know, attractive document, but

17:22there's so much goodness in there. And

17:25so as we put it in an agentic workflow,

17:27we're modifying the document. Hey, why

17:28don't we put this we're missing and you

17:30you realize how many things were living

17:32outside of the document if you could put

17:34it all in. So it's that feedback loop

17:36and you keep modifying things.

17:38>> So I'm a big fan of documents in

17:40general. It's a very unpopular opinion,

17:42but like I I I tend to say hey think of

17:45anything in the world, it's based on

17:48like a document. Like you talk about

17:49like expense policies. Hey, it's based

17:51on it's based on content. It's based on

17:54what exactly is your expense policy for

17:56flights. It's it's a it's a field inside

17:59the document. You you talk about tax

18:01documents. You talk about loan

18:02applications. Everything is all based on

18:04content, right? The minute you unlock

18:06what's inside a content you just make it

18:08easy for somebody to speed up the

18:11process. Just it just makes life easier.

18:14>> Yeah.

18:16>> Awesome. I'm going to switch gears a

18:17little bit and talk about partnerships,

18:19which I think are top of mind for you

18:20guys. Where in your organizations like

18:23agent building capabilities are they

18:25in-house versus in partner? Where do you

18:27choose and who makes that call?

18:31>> So many people ask me this. I'll talk to

18:33the the market in general and then maybe

18:35a little bit of AWS and so I I

18:37personally am a big fan of a

18:39combination. I don't like to get into

18:41the black or white, you know, build it

18:43buy it, but I think

18:45things in general, software companies,

18:47ISVs have done a great job with

18:49workflows, with content platforms and

18:51applications. I think naturally that

18:54agentic AI use cases, other AI use cases

18:56are just going to go on top of it. So

18:58what I say to companies when they ask me

19:00is I say look at if you have a general

19:02ledger, if you have an HR, if you have a

19:04CRM, like what sort of AI capabilities

19:07are those providers building on? Because

19:09why would you want to go do that

19:10yourself?

19:11So first and foremost, I think that's

19:13that's the right path, but I think it's

19:14good when companies

19:16augments along the way in some

19:19capabilities. And so, with AWS, we we

19:21have different partnerships. We have

19:23different folks. They're building in

19:24some of their own AI capabilities. We we

19:26look through it. And so, internally,

19:29when we look at our own use cases, we go

19:31back and we say, "What again what are we

19:32trying to solve? What do we get already

19:34out of the box in that application or

19:36platform or widget or whatever it is?

19:39And what are what are they applying? And

19:41so, I think the ISV market right now, as

19:43we see, is being the most innovative,

19:46Box included in this,

19:48of building out and putting agents

19:49extending in the offering that that's

19:51already there. So, that's how I think

19:53the market is going to go. Um I don't

19:56think people are going to go rewrite

19:57their own general ledger. How are they

19:59going to go rewrite their own, you know,

20:00benefits package? And so, I think it's a

20:03good thing to let the ISVs

20:06go be innovative and extend out the

20:08capabilities with AI.

20:10>> I think I I think of it very similar.

20:12Like, when I first saw what customers

20:15were doing,

20:17uh they were trying to rebuild some of

20:20the things from scratch, right? Think

20:22about

20:23they were trying to build their own rag

20:25models. They were trying to build their

20:26own vector databases. They were trying

20:28to

20:30write the chunking logic.

20:32Make sure it has the right permission

20:35retrieval. Uh build an OCR layer on top

20:38of their existing content workflows.

20:41That's where my perspective was

20:44it's it's very hard [snorts] to build uh

20:47govern governed content layer for like

20:51customers, right? Like, it's it's

20:53especially at like enterprise scale uh

20:56with the right security models and

20:58governance models, it's very hard. So,

21:01my perspective is I think I I think we

21:03should uh

21:05customers should focus on

21:07like

21:08building the right business logic on top

21:11of

21:14content.

21:15And that's where

21:17a lot of their energy can be focused on.

21:19Instead of using your engineering

21:20dollars to

21:22uh, build your content layer, I think

21:24customers are better suited to build the

21:27right prompts for their documents, uh,

21:30build the right confidence levels for

21:32their workflows, build their business

21:34rules on top of content layer, uh, build

21:37even build even like more of uh, audit

21:41policies on top of content models, like

21:43build their integrations. But use Box

21:45for your secure storage. Use Box for

21:48uh, permission permission native agents.

21:51You have We've got metadata. We can

21:52convert all your content into usable

21:55uh, con- structured content for your

21:57agent. So, my my philosophy is very

22:00similar. Use Box for content as your

22:03content layer. And customers should

22:06focus on building like their business

22:08logic on top of their content layer, so

22:11they could accelerate their use cases.

22:14>> Now, I'll say because everyone always

22:15asks like what does AWS do? So, so we're

22:17talking about this.

22:18>> I I think it's the same for

22:19infrastructure layer, too.

22:20>> 100% but it's it's I say this anecdote

22:23for us and it sounds so simple. We use a

22:25third-party video conferencing

22:28software partnership.

22:30The amount of note-taking follow-up and

22:33the capture of AI that we're using is

22:36unbelievable. I didn't realize how bad

22:38we were

22:39at note-taking internally. And then all

22:41of a sudden we get off these conference

22:43calls, we have these summaries. We move

22:44very fast at AWS. So, frugality is one

22:46of our leadership principles.

22:48We're not going to do a meeting in 60

22:50minutes, we're going to do it in 30

22:51minutes. We're not going to do it in 30

22:52minutes, we're going to do it in 15

22:53minutes. But just the ability to take

22:55traditional video conferencing now that

22:57has an AI layer in it to summarize and

22:59help with next steps, absolute

23:01game-changer. And so, you wouldn't think

23:03that that's a big use case, but for us

23:04internally, it it's been absolutely

23:06huge. So, those are the little things

23:08that we're getting all the time from

23:09video to content to the CRM. Everything

23:13we're doing, we're trying to put an AI

23:14layer in, whether it comes from, again,

23:16an ISV partner or we're augmenting it

23:18with ourselves.

23:19>> And and you have the flexibility and

23:21freedom to like customize how you want

23:24your notes to look like. I think that's

23:25where a lot of value is like you can you

23:28can benefit a lot of that by adding that

23:31layer on top of the existing stack,

23:33right?

23:34>> I don't know if I'll ever build another

23:35pivot table again, thank goodness.

23:36[laughter]

23:37>> Can Can will you ask your agent to build

23:39a pivot table based on your expense

23:41policy?

23:42>> I I will not use the word pivot table,

23:44but if I can get any of that data

23:45extracted out, I mean, we we joke about

23:48it, but um I didn't want to do pivot

23:50tables 5, 10 years ago, let let alone

23:52now. Um there'll be people out there

23:54that say, "Hey, you know, you'll never

23:55get away from pivot tables." And I'm not

23:56trying to make that argument against

23:58that or some of those tools, but um the

24:00way that I want to consume data, my

24:02teams, other people,

24:04I want to I want to just ask it

24:05questions. I want to go explore the

24:07data. I want to get the data out. Um and

24:09I want to constantly be asking

24:11questions, cuz every good question leads

24:13to an answer. Every good answer leads to

24:14two or three more questions. By the way,

24:15I took that line from someone else, but

24:17it was easy to say before we were

24:19slicing and dicing dashboards or coming

24:21through documents. Now, you can actually

24:22real time do it.

24:24>> Ask. Yeah. I'm sure uh at least my kids

24:27would not know what a pivot table is.

24:29>> Yeah, thank goodness. I don't think my

24:30kids know what a pivot table is, either.

24:32>> Yeah, I mean, listen, if an agent can do

24:34a VLOOKUP for me, I'm happy.

24:35>> [clears throat]

24:35>> Yeah.

24:36>> [laughter]

24:36>> I've got one last question for the both

24:38of you. So, what's something you

24:40believed about AI agents maybe a year

24:42ago that you no longer believed, and

24:43what changed your mind?

24:47>> Well, it's how easy it is to to build

24:49them. I I don't want to say her name. Um

24:51she's within AWS, just for

24:53confidentiality, but um she's she's a

24:55very smart person, but she's not someone

24:57I thought would ever build one. And she

24:59came into a meeting and she had built

25:01her own agent.

25:03I I couldn't have built what she did.

25:05And so, I always thought it was people

25:07would consume agents. I didn't realize

25:10that the average person could go build

25:12them.

25:13Now, again, I'm not saying that we

25:15should just let everyone go crazy build

25:16agents. Yes, there's going to be

25:17governance and operations teams.

25:19Um, but I missed a year ago how easy

25:22it'd be for people to go do. And that

25:24And that was a pleasant surprise, but

25:26I've had to personally rethink how the

25:28world is is changing

25:30and that people will do that. And the

25:32other part of this I was actually at my

25:34my kids play competitive water polo.

25:36And uh when they do the tournaments, you

25:37have all the data and sheets of where

25:39they're going, the games, the stats.

25:41One of the parents, non-technical, built

25:43an agent to scrape the data to go see

25:47what place the team was in and what time

25:49the next game would be. It sounds so

25:50simple, but this is someone who just

25:52spun up a model and and and a parent

25:54that did it on their own. So, my eyes

25:56have been opened about just the average

25:58person how they're going to go build.

25:59>> Yeah. I I think for me the answer is

26:01very similar. Uh

26:03like a year ago I thought it was

26:05impossible to use a model to

26:08look into a document and get the data

26:10out. Like, I thought it was not possible

26:12to just have your have a parent go

26:15through the go through all the Excel

26:16sheets of the of your water polo and

26:18then

26:19and then tell our and then give us some

26:21data out.

26:22I think I'm wrong.

26:23Uh I I think today the models exist.

26:26They're actually much much much

26:28efficient in

26:30like going through hundreds of documents

26:32and giving you the right answer.

26:34Uh I think what I kind of underestimated

26:36is our our ability to like build systems

26:40around it. Like

26:42for the example you said about building

26:43agents,

26:45uh I think it's incredibly hard to like

26:48uh

26:49build the tooling for these agents to

26:51work at an enterprise scale with the

26:54right security and governance model. I

26:56think that's what I think is was was a

26:58lot more difficult than than the models

27:01itself

27:02extracting data.

27:04Uh so, that's that's my take on what I

27:07think I got wrong. Uh

27:10I think what we're all focused on is

27:12focusing on building that infrastructure

27:15for agents to enterprise agents to scale

27:18at the enterprise level, right?

27:20Build the right permissions, build the

27:21right audit trails, build the right uh

27:24capabilities to have human in the loops,

27:27handoffs where agents and humans work

27:29together. Uh that's where I think the

27:32effort's going to go in the future.

27:34>> This was where similar I think we're

27:35both optimists optimists in that it's

27:38it's easier, it's now, it's not going to

27:40be as heavy of a lift. It's not

27:42something out in the future, it exists

27:43today right now. Any anyone can do it.

27:46Um it's never going to be easy, but it's

27:48this isn't a 5-year-out thing. It's a

27:50right now. How do you do it?

27:52>> Yeah, and how do you do it? Uh in a very

27:54reliable way, too. Like uh

27:57I understand agents can drive a lot of

27:59productivity,

28:00uh but if the process is too risky or if

28:04the process is very important,

28:06uh there's ways to do it

28:10with high level of reliability. You can

28:11bring in the human whenever you need.

28:14You can define confidence levels. You

28:16can you can define risk monitors. So, I

28:18think it's getting to a lot better state

28:21where you can actually safely deploy

28:24enterprise for your agents for your

28:26critical workflows in the enterprise.

28:28>> I know we're coming up on time, but I'll

28:30I'll just squeeze in that I think that

28:31that

28:32I probably missed how much culture

28:33impacts this. You know, organizations

28:35will do this right will really have

28:37empathy

28:38of what their people do.

28:40Um and I intellectually knew that, but

28:42then as I've seen it, I've worked with

28:43more companies that the ones that are

28:45doing well in this space have great

28:46culture, and understand what their

28:48employees need. And the healthier the

28:50culture and that curiosity of helping

28:52them with the tools and process to their

28:54job, the better and faster it goes. So,

28:56I would have thought a year ago it was

28:58more of a technology play.

29:00Culture number one.

29:01>> I I think I think organizationally

29:03that's the that is where I think

29:06that is what will define the success of

29:08an agentic deployment. If your HR person

29:12knows that using agents will help uh do

29:16their jobs 50% faster, 90% more

29:19accurate, I think they will be your

29:21advocates.

29:22I think we're getting there where we

29:24need to work, we need to define roles,

29:26we need to like define center of

29:29excellence for AI across the org. Uh we

29:31we define what the metrics look like for

29:34success. It's no longer the old approval

29:36rates. So, uh

29:38I think

29:40the more and more

29:41we see where AI how we can drive AI to

29:44success, it's all about

29:46how we can drive our organizational

29:48rigor to ensure

29:50uh we can drive that towards success.

29:54>> Awesome. Well, thank you both. It was

29:55great having you today. That's all the

29:57time we have for now. Thank you for

29:59joining us, and we'll see you next time.