It's 10pm. Do You Know Where Your Agents Are? — Kim Maida, Keycard

0:01[music]

0:12Okay. So, when I was a kid growing up in

0:15the '9s, we'd be out late all summer

0:17riding our bikes off in the

0:18neighborhood, playing with friends. And

0:20from the 60s to the 80s, there were

0:22these public service announcements on TV

0:24where celebrities would come on and they

0:27would say, "It's 10 p.m. Do you know

0:29where your children are?" Because

0:30apparently our parents at that time

0:33needed to be reminded that they had

0:34offspring they were responsible for. And

0:36I feel like in this phase of AI where we

0:39are right now, um, we're entrusting

0:41agents with more and more

0:42responsibility,

0:44but we still kind of need that public

0:46service announcement that says it's 10

0:48p.m. Do you know where your agents are?

0:54So, say you as a user want agent to use

0:57an MCP server or API to accomplish

1:00tasks.

1:01Now, we know agents without access

1:04aren't useful. So, we give them an end

1:07file and we give them some API keys and

1:09we let them run off and go do their

1:10thing. And this is fine until it's not.

1:14Uh, so you've heard the horror stories,

1:16right? Or maybe even experience some of

1:18them yourself. Uh, so let's see what

1:21this looks like in practice.

1:36this way.

1:45Okay. So, I have an agent running here

1:48that is it's basically an incident

1:50management agent. It is

1:53late night and there is a human user but

1:56they're probably half asleep. The agent

1:58is responsible for triaging issues that

2:00are coming in. Right? So if we

2:04look at the first one here,

2:07I can find the mouse.

2:10So the first one is the backup power

2:12supply failed in the server room.

2:16Now the agent is going to use an API key

2:18to read the system that is bringing in

2:21the reports. It's going to evaluate what

2:24is written there and it's going to

2:25decide that it can't do anything about

2:27this, right? Because it's a it's a

2:29physical fail failure.

2:31So it's going to escalate that ticket

2:33for the morning and there's really sort

2:34of no problems yet, right?

2:38So now the second one is the certificate

2:41is expiring soon right for TLS

2:46agent is going to use that same API key

2:48again to read the ticket contents and

2:52then it is going to decide that it

2:54should renew the certificate and it's

2:56going to use another API key to call the

2:59cloud hosting service to uh renew the

3:02certificate and then it's going to use

3:06the API key from before to make the

3:09report for the morning team.

3:14So, here's where it kind of gets fun,

3:15right? So, this one is the billing

3:18database is broken and payments are

3:20failing.

3:23Agent's going to read that

3:27it sees that the solution is pretty

3:29clear. It says that the documented

3:31recovery is to delete the database and

3:33then restore from what the restore from

3:35backup happen automatically. Um, so it

3:37has the Postgress connection string. So

3:40it goes ahead and it drops the database

3:42and then it doesn't have a way to check

3:43to see if it was backed up. So it just

3:45escalates that for the morning.

3:48And this has really happened, right?

3:50Like it's happened to high-profile

3:51companies.

3:57Now if we go to the next one, this one

3:59is the main server processes are frozen

4:02and they're not recovering. doing

4:04something about it's going to take prod

4:06offline for a brief amount of time. And

4:08you can see the the agent decided that

4:10it should do that. And it's using the

4:12same API key now that it did to renew

4:15the certificate, right? Because that API

4:17key is a kitchen sink. It can do all of

4:19these things with it.

4:23And then finally, we have sites failing

4:25for one in three users.

4:28The recommended solution is to scale up,

4:30right? And this is going to incur some

4:32amount of spend. And then it goes ahead

4:34and it does that because again it can

4:37use that same API key to do this as

4:39well.

4:52Right? So agents with API keys are

4:54indeed outpass 10. They're

4:56overprivileged. So this means they are

4:58able to act freely on decisions that

5:00they make that you may or may not agree

5:02with. And they can do this even with

5:04your supervision. So you might be

5:06familiar with the panic of uh trying to

5:08stop an agent mid task because you told

5:10it to maybe read a project and it read

5:12the project and it found something it

5:14thought it should fix and then it starts

5:15writing. Um agents do that even while

5:19they're supervised. And this is becoming

5:21even more of a problem because we have

5:23more and more agents that are running

5:24unsupervised and that only makes it

5:26worse because agents want to be helpful.

5:28they're going to use all the permissions

5:30that they have access to in order to get

5:32the job done.

5:35And we can't just solve this with human

5:37in the loop. We spent decades solving

5:40access management for humans. So just

5:42blindly trusting a human who might be a

5:44little bit consent fatigued uh or who

5:46might be tired enough at night, this

5:48isn't really going to be enough.

5:52So in order to see where we can

5:54introduce security and access control,

5:56we have to take a look at the agentic

5:58execution path.

6:02So we have a user who wants to use an

6:04LLM to interact with a resource. Now an

6:07agent is a control loop that calls an

6:09LLM and often we have an MCP server in

6:12between that provides tools that the

6:14agent can call and then it connects

6:15directly to the resource.

6:18Now an MCP client takes the agent's

6:21proposed tool calls and it dispatches

6:23them to its MCP server.

6:26And then we have a runtime. Now this is

6:28a process that runs the agent loop and

6:30executes the calls. And this runtime

6:33might be a CLI like cloud code. It might

6:36be an SDK like AI SDK or provider agent

6:39SDKs.

6:41Or it might be an app like cursor or

6:43codeex.

6:45So let's follow a prompt through the

6:47execution path. The user submits the

6:49prompt to the runtime which calls the

6:51model which sends it to the LM. The

6:54model is then going to propose tool

6:56calls which are dispatched by the MCP

6:59client to the MCP server which then

7:01executes the tools and it calls the

7:03resource API.

7:05The API then responds to the MCP server

7:08and the MCP client delivers the results

7:10to the runtime and then the model will

7:14be called and loop repeats until the

7:16model is satisfied and at that point

7:18it's going to return the final answer to

7:20the user.

7:22So there are a few places in this path

7:25where we could implement real access

7:27control and we can actually do this with

7:29open standards and as kind of a spoiler

7:32it's not just OOTH.

7:36So RFC8693

7:38is token exchange and this is an RFC

7:40that extends OOTH 2 and I'm going to

7:43show you how this spec can be used to

7:44address agent access.

7:49So first I want to recap the problems

7:50that we're actually trying to solve

7:51here. Right? So if you remember looking

7:53at the audit log as it was going by in

7:55the demo, we could see API keys were

7:57being used to call endpoints, but we had

7:59no idea who was using the API keys. So,

8:02we have credentials that are being used

8:04that aren't attributed to a user or an

8:07agent identity. We have an agent that

8:10has unrestricted access to any and all

8:12permissions that are in an API key. And

8:15finally, we know that we can't just slap

8:17human in the loop everywhere because

8:19humans make mistakes too. And also, many

8:22agents run autonomously.

8:26So, we can address this with an

8:28authorization server called a security

8:30token service.

8:32Now an authorization server verifies

8:34identities and issues tokens. So

8:36identity providers like Google, Octa,

8:38Ozero and so on provide authorization

8:40servers. And if we want an identity

8:43chain in our agent execution path, then

8:45we have to be able to log in first.

8:49So the authorization server is then

8:51going to prompt the user for their

8:52consent to delegate access with a subset

8:55of their permissions. And this is the

8:57first narrowing of access, right? We're

8:59only delegating some of the user's total

9:00permissions to the agent.

9:04The authorization server issues a token

9:06that identifies the user and also

9:08contains their level of access. And so

9:10right now already we're doing better

9:11than the first demo because we actually

9:13know who the user is and what they're

9:14allowed to do.

9:17So this token identifies the subject on

9:19whose behalf the agent is going to act.

9:23In order to support token exchange, we

9:25need an OOTH client that's capable of

9:27executing code. So, this might be a

9:30gateway between the MCP client and a

9:32thirdparty MCP server. It might be your

9:35own custom agent app or a CLI wrapper

9:37around an off-the-shelf coding agent.

9:41And we take the prompt and we take the

9:42subject token and we send these to the

9:44OATH client. Now the agent loop runs and

9:47the model proposes a tool call

9:51and then the runtime is going to

9:53authenticate with the security token

9:54service using its ooth app client

9:57credentials or workload identity and it

10:00also sends the subject token that

10:01contains the user's identity and level

10:03of access. It creates token exchange

10:06request and this request is asking for

10:09permissions to access the MCP server for

10:11that tool call but only that tool call.

10:14So now we have three key pieces of

10:16information that we're missing from the

10:18API key demo.

10:22We know the identity of the agent that's

10:24requesting access. We know the identity

10:26of the user on whose behalf is acting.

10:29And we know the delegating user's level

10:31of access as well.

10:33So now we need to decide if the

10:35requested token should in fact be

10:37granted. And we can do this using

10:38governance policy

10:41which is evaluated against the requested

10:43access and who's asking for what

10:45resource on whose behalf.

10:50Now if the delegation chain and the

10:51requested access are within policy then

10:54the security token service issues an

10:56access token for the downstream resource

10:58and this token has an audience declaring

11:01that only this target MCP server is

11:03allowed to use it to make requests. It

11:05should be short-lived uh often expiring

11:07within a few minutes and it's also

11:09ephemeral meaning it should never be

11:11stored.

11:13So this token is sent to the MCP client

11:16which makes tool call using it as a

11:17bearer credential.

11:19The MCP server validates the token and

11:22then goes and calls the resource and

11:24again it never stores the token and it

11:26discards it as soon as the call is done.

11:30So the result flows back up the loop and

11:32then it repeats until the model returns

11:34the answer to the user.

11:40So if we come back to the demo now we're

11:44going to use the same agent

11:49only now

11:52we have token exchange.

11:57Okay. So the first thing we have that's

11:59different already is that we have an

12:01operator sign in right. So we have

12:02authentication

12:05and I'm going to authenticate with

12:07Google as myself.

12:13So we have the same tickets. We have the

12:15same agent. It's got the same prompt.

12:18And now we can see what it's going to

12:19do. Now the first item is probably going

12:22to be exactly the right because this was

12:23a hardware failure. there's it's going

12:25to decide after it reads it that there's

12:26nothing it can do. But as you can see

12:28kind of the audit log filling up, we've

12:30got a lot more information now. Um we

12:32know who the agent is acting on behalf

12:35of. We know that the agent is calling a

12:38prod infra MCP server and we know that

12:40it's going to contact certain downstream

12:42resources. Right? So we have a hardware

12:45monitor that is the source of this

12:47incident and then when it decides that

12:49it can't do anything about it, it uses a

12:52right scope to talk to the pager

12:57uh resource in order to escalate to the

12:59morning team.

13:07So for renewing the certificate, right,

13:09this is actually a pretty safe action

13:12and it's going to specifically ask for a

13:16scope to only renew the certificate,

13:18right? So it's talking to this cloud

13:20host where before we had this API key

13:22that could do a ton of different things,

13:23but this time it is only asking for

13:25permissions, being granted permission to

13:27do this one thing.

13:33So now with the billing database right

13:35like the billing database is broken. It

13:37uh pretty clearly documents that you are

13:40supposed to uh drop the database here

13:44but no agent should be able to drop a

13:46database. So what happens is when we

13:49make this call it's being eval the

13:51policy is evaluating the request against

13:54all of the permissions that the user has

13:56and uh it sees that there is actually a

14:00restriction in place that prevents

14:01agents from doing this and this

14:04credential never even existed. So the

14:06policy evaluates before the credential

14:08is minted, which means you don't have an

14:10overprivileged credential that's just

14:12floating around then that uh you were

14:14supposed to then prevent the entity from

14:16receiving. Um it just doesn't exist. So

14:19there's nothing to leak, there's nothing

14:20to replay, and there's nothing to steal.

14:27So this one was the one where it wants

14:30to restart prod.

14:32So there are things that agents should

14:35probably be allowed to do and things

14:36that maybe they shouldn't be allowed to

14:38do and then there's some kind of you

14:39know something in between, right? So

14:41it's going to ask me as the user for my

14:44approval as human in the loop. I say

14:47that it can do that but there's another

14:49policy here that says that the human

14:51user needs to have a specific role in

14:54order to be able to do this. And I

14:56actually do not have that role. So it's

14:58going to prevent me from being able to

15:00allow the agent to do this. uh even

15:03though I approved it. So we can prevent

15:06kind of people from just consent fatigue

15:08clicking over and over just to get

15:09things done.

15:14>> And then this is the scaling one, right?

15:17So this is something that maybe the user

15:20does have permission to do.

15:23So if I say approve on this, I do have

15:25permission to do this and I was able to

15:27tell the agent that it is indeed allowed

15:30to and the policy approved it because I

15:33am allowed to do it also.

15:52So the agent access problems that we had

15:54discussed they have solutions now we

15:56know who the user is and we know who the

15:59agent is as well. The agent also has

16:01task scoped short-lived ephemeral access

16:06and human in the loop actually has

16:08access control that is backed by real

16:10policy. So an exhausted person can't

16:12just accept everything that happens.

16:17Now another benefit of using open

16:19standards like token exchange is the

16:21ability to continue to support emerging

16:23technologies. So this works with

16:25off-the-shelf agents. It also works with

16:27custom agents that you might build

16:28yourself. It works with the CLI. It

16:31works with thirdparty as well as

16:32proprietary MCP servers, MCP gateways,

16:35agent to agent, uh any OOTH identity

16:38provider. It works with OpenClaw and

16:39basically anything that might come out

16:41next week.

16:45So, it is in fact possible right now to

16:47be that responsible parent and to say

16:50that yes, you do in fact know where your

16:53agents are.

16:58So my name is Kim Maida. I am the

17:00founding GTM engineer and head of Devril

17:02at Keycard which is a standardsbased

17:05platform for uh providing a security

17:08token service and policy governance. I'm

17:10going to be at the Keycard booth for

17:12kind of the duration of the event but

17:14we're also running a workshop tomorrow

17:15on building and securing an MCP server.

17:20You can scan this QR code to connect

17:22with me and I really appreciate your

17:24time today. So, thank you very much.

17:27[applause]

17:31>> We do have time for a couple questions.

17:34>> Uh, so if anyone has some questions for

17:36Kim, um, we can answer those right now

17:38and then we got about five five minutes.

17:40So, we can probably do about five short

17:41questions if anyone has them.

17:55Um, so it seems like you put the

17:57security bar bear barrier

18:00uh at the um between MCP to

18:06uh resource or is it uh just maybe to

18:11clarify that is it at at both places?

18:14I'm wondering if there was like any

18:16decision there or like what would

18:18motivate you to choose you know where to

18:20put these sorts of barriers.

18:22>> Uh yeah so the the authorization server

18:25is

18:27sitting in between let me find the slide

18:30actually.

18:44So the runtime authenticates to the

18:47security token service and identifies

18:49itself. And this is the point at which

18:51we have the request that the agent

18:53generated. Um and we also have the the

18:56scopes that it's asking for to get the

18:59token to call the next thing in line,

19:01right? And the next thing in line in

19:03this particular case is the MCP server.

19:06So the downstream resource is like one

19:08step farther down. But if you think

19:11about like an OOTH token for a user,

19:14right? So an OOTH token for a user is

19:16going to have all of the grants in it

19:18that the user accepted when it when we

19:22were presented with like here's what

19:23access you're going to delegate. But an

19:25agent, you don't want an agent to be

19:28using any of those grants that it wants

19:31on every single tool call. So this the

19:35service sits in between that so that we

19:38can say if the things it's requesting

19:41are kind of beyond what we want to allow

19:44for the specific tool call then we never

19:47send that ooth token down right so they

19:49get that oath token only if they are

19:52within requesting something within the

19:55scope of what what we want

19:58does that make sense

20:08based

20:25enterprise systems. Uh there may be some

20:29resistance to actually adopt this new uh

20:32protocol, new openspec. Have you

20:35encountered that? and what are the ways

20:37to uh get past that?

20:40>> So, it's not actually a new spec, which

20:43is, you know, it it's kind of one of

20:46those things like there was this period

20:49of time where people were like, oh, you

20:51can just use OOTH for for this. Um, we

20:55don't necessarily need a new spec for

20:56this. This spec has actually existed for

20:58a little while already. Um, so there's

21:01not kind of that fear of, oh my gosh,

21:04we're introducing something completely

21:05new.

21:07There are new specs that are coming out

21:09almost daily right now. Um, but they can

21:13be combined essentially with this uh

21:16with token exchange.

21:19I think we have time for one more

21:21question.

21:27I might be asking a big question. So if

21:29you tell me to go look at the thing,

21:31that's fine. Um, I'm in the situation

21:34where we know that we don't have enough

21:36OOTH scopes defined yet in an MCP server

21:38sub just like what you have. And one of

21:40the reluctant things that we've got is

21:42like how fine grain do we get with the

21:44scope definitions because we know that

21:46we're also having to define downstream

21:48services that will have a certain number

21:49of these things and someone has to do

21:51the authorization check somewhere.

21:52What's your recommendation for getting

21:54started with defining your own scopes so

21:56that you can realistically manage this

21:58thing when you haven't defined enough

22:00yet and you're worried about ongoing

22:02management over time?

22:05>> Well, if your resource server already

22:07has like specific scopes, that's going

22:09to be kind of your place to start

22:11because the downstream token, the one

22:13that was the OS token for the user is

22:16going to have the scopes for the

22:18resource, right? Because it's the user's

22:20access to the resource. So those are

22:22kind of like the baseline like those are

22:24those are the ones you know that you're

22:25going to have and then if you want to

22:27have scopes additionally for that kind

22:31of govern tool calls if you have like a

22:33custom MCP server or something like that

22:35then you can layer those on top or you

22:37can just pass them through.

22:39>> Yeah. Okay. Thanks.

22:41>> All right. Thank you Kim.