Full transcript
0:00Hi, thank you so much for tuning in. I'm
0:02Ravi, I'm one of the co-founders of
0:03Scale Grid.
0:04Today, I'm going to talk about how you
0:07need to think
0:08architecturally from the ground up about
0:10building your applications APIs, your
0:13MCP servers for agents,
0:15and how the human-focused architecture
0:18doesn't scale well for agents.
0:20So, a while back, we were looking at our
0:22performance and latency numbers, and one
0:23thing that kind of jumped out at us was
0:26how our latency was spiking every 15
0:30minutes in a rhythmic manner.
0:33Nothing harmful, but just a curious
0:35thing for us to analyze. What we noticed
0:37was very interesting. So, in our
0:40identity and authentication
0:41infrastructure platform, we have this
0:42little timestamp that we mark for every
0:44user to say, "Hey, when was the user
0:46last seen?" or "When was the user last
0:48active or last acted in our system?" so
0:52that we can predictively say, "Hey, this
0:54user is an active user. This user is not
0:56so active." But, one thing that we
0:58realized was this system was primarily
1:00built for humans, but
1:02when agents started hitting our APIs in
1:05the last 12 months or so,
1:07we realized that this
1:10last seen update is happening
1:1260 times faster than what it would, and
1:15that is creating unnecessary pressure in
1:18our DB write system. So,
1:20of course, it's a harmless thing. We
1:22were able to fix it very easily. We
1:24would just batch the update at a second
1:26level and not at every single time we
1:28had to update it.
1:30That kind of took us down a rabbit hole.
1:32So, the assumption that broke was how
1:34often would our system have to update
1:36this timestamp on every row, and that's
1:38okay. It's just about speed, it's about
1:40latency, etc.
1:42But, what I was worried about is, "Hey,
1:43what if some of our assumptions that we
1:45made about authentication, authorization
1:48need to be rewired and rethought
1:49completely
1:51when it comes to agents because we would
1:53have designed earlier for humans as
1:55actors in mind?"
1:56Now, just to give you a context, I
1:59worked on identity and authentication
2:01operations for the last 10 years
2:02building an identity platform at
2:04Freshworks,
2:05which is being used by millions of daily
2:07users, hundreds and thousands of
2:08customers all over the world. But, this
2:10is predominantly human users, right? Or
2:13at best, APIs. But, the way I think
2:16about it is APIs are also accessed by
2:18machines that are written by humans.
2:22That's not too bad, right? But, what I
2:25realized is the fundamental picture has
2:28changed drastically in the last three,
2:30four years or so.
2:32We have a unique ringside view to see
2:35how developers nowadays are building
2:37agents and how they're giving context to
2:39these agents with data from third-party
2:41applications like Salesforce or or
2:44Databricks or HubSpot or Notion. What we
2:46have realized is
2:49most of the agents our customers are
2:51building have
2:53way too permissions and scopes than the
2:57agent's responsibility or the agent's
2:59job is. Again, it's not because the
3:03developers who are building the agents
3:04are careless,
3:06but somehow
3:07this became a default pattern of giving
3:09[snorts] the agents what they need
3:11access to, and the existing primitives
3:13that we have don't let us give extremely
3:16fine-grained permissions to the agents.
3:18Now, I'll tell you how we ended up here,
3:21right? We predominantly have two slots,
3:24and neither of the slots was built for
3:26agents in mind. There's a human who's
3:28acting the application, either a web
3:30application or a mobile application, or
3:32their own little script that they wrote,
3:33and they give it their API key so that
3:36their program can access data from the
3:38application. This is all
3:40the fundamental principle here is
3:44it's the same user who is
3:45authenticating, and it's the same user
3:47who's acting, right? And the second slot
3:49is the traditional service account
3:52scenario or end-to-end account scenario
3:53where
3:54you create a service account, you give
3:56it certain permissions, and then say
3:57this machine has its own identity.
3:59That's where the likes of SPIFFE and and
4:01OAuth and all of that came into picture,
4:03but you would give them certain
4:05credentials and say, "Hey, now this
4:07machine has access to whatever data that
4:10it needs at any single point of time."
4:12And this is the existing pattern, right?
4:15So, the fundamental philosophy that
4:17we've always maintained is whoever is
4:18authenticating is the one that is
4:21acting. Every action the program or the
4:23human takes is based on fixed set of
4:26permissions that actor was granted at
4:29some time. If you take traditional
4:31authentication mechanisms for humans,
4:33including password, you just say, "Hey,
4:35if an identity has the same password
4:37that it was set at the time of
4:38registration, if they come back and if
4:41they present the same password again,
4:42then you say, "Okay, this is how I
4:44validate the identity. This is how I
4:46authenticate the human." And every
4:48action subsequently is tied to that
4:50human identity. Again, the same is the
4:52case with API key or the same is the
4:54case with web session tokens or even the
4:56same case for service account. You
4:59define the permissions at the time of
5:01registration, and then every single time
5:03it acts based on the registration time
5:06permissions and scopes.
5:09Now, this is okay all this while because
5:12for decades the service account and
5:14OAuth principle even
5:16is working fine even though there are
5:20their own problems, but it's still
5:21working fine because these machines are
5:25using a program
5:28in a deterministic way by the way the
5:30human developer wrote that program.
5:33So, there is
5:34absolute guarantees about what the
5:36program could or what the program won't
5:39do, but it is still intentional based on
5:42what the human wrote, right? In this
5:44particular case, again, if it is using
5:47API keys, then that actor and the
5:49principal is the same, then there is
5:51some sort of a delegated permission for
5:53the program to act based on what consent
5:55the user has granted. But, the second
5:57one is the most important part, which is
5:59it's a deterministic program, and it
6:01always stays in its own lane. It can
6:04never do what it was not programmed to
6:07do. And you could inspect the code to
6:10say, "Okay, is the program doing what
6:12it's supposed to do?" Even if you apply
6:14for a Google developer account, and then
6:15ask for client ID, and
6:16you need to access these scopes, you
6:18have to go through a security review.
6:19So, what they're doing is they're
6:20looking at your code base to see, "Are
6:21you doing enough checks? The appropriate
6:24practices in place?" So, these programs
6:26are deterministic. These programs behave
6:28the exact same way a developer
6:29programmed them to work.
6:32But, agents fundamentally break this
6:34assumption.
6:35Right? First of all, in the case of
6:37agents, the principal is not the same as
6:39an actor. You need to give delegated
6:41access, so that the agent can act on
6:44behalf of the user. Agent can access the
6:46user's Gmail. Agent can access the
6:48user's Salesforce data, and whatever the
6:50case may be.
6:51But,
6:52again, unfortunately, not a lot of
6:55systems, even today, support OAuth. So,
6:57here again, there is no on behalf of
6:59principal that is working. So, you don't
7:01even know if there is a program that is
7:04acting on behalf of the user, or the
7:06user acting by themselves. Right? That's
7:08a fundamental problem. The second
7:10problem is even more dangerous, which is
7:13right now, the program is not written by
7:15human. There is no determinism baked in
7:18to say what the agent will do or won't
7:21do. Right? Just because an agent does
7:23certain things today, you can't be 100%
7:25certain that the agent can't do the same
7:28or the exact same thing tomorrow, day
7:30after, or even if it's the next
7:31immediate run.
7:33Right? Because of this non-deterministic
7:36nature of agent,
7:39we usually pick one of the two lanes,
7:41right? You give a specific identity to
7:43the agent, which is what we call as
7:45client ID in the context of OAuth, and
7:47then say, you act on behalf of this
7:49particular user, or we go back to the
7:52agent
7:53acts as the user, which is even worse.
7:57The fundamental reason why I'm harping
7:59on the same thing is because
8:02when an agent is acting on behalf of
8:04user one versus user two or user three,
8:07the agent needs specific permissions
8:11based on the user's context. Now, the
8:13OAuth solved this perfectly fine by
8:16saying, hey, the user will grant
8:19specific scopes or permissions to the
8:22agent. So, when the agent is acting on
8:24behalf of the user, it can't do
8:26everything, but
8:28the agent will only do certain things.
8:30Now, in the kind of the world that we're
8:32living in, most of the MCP servers that
8:34we've worked with
8:35don't actually limit the tool context
8:38access to the agent based on which user
8:42authorized the agent. They typically
8:44surface all the tools that
8:47the user has access to, or all the tools
8:50that the application can even support,
8:52and then let the agent determine what
8:54they are can or cannot do. And now the
8:58agent ends up picking the wrong tool,
8:59doing wrong things. Maybe there is some
9:01runtime check in the application that
9:03prevents some of these things, but the
9:04agent is still seeing the same surface
9:06regardless
9:08whom it is acting for. Now, two things
9:11that we need to solve for. One is the
9:13actor,
9:14in this case, an agent, has to be bound
9:16to the principal at all times. And the
9:19agent should have its own identity.
9:21Agent should have extremely fine-grained
9:23credentials, not the OAuth scopes that
9:25we are seeing today. If you inspect the
9:27scopes for some of these applications,
9:29like even very popular applications like
9:31Gmail, it'll say, can this client send
9:34emails on your behalf? There's no
9:36extremely fine-grained scoping to say
9:38can this agent act at this hour? Can
9:41this agent read emails only from these
9:44senders? Can this agent send emails to
9:47only this recipients? The reason why
9:49that is important is because again, we
9:51spoke about this earlier in the context
9:53of non-deterministic agent workflows,
9:57it's extremely important that the agent
9:59should have permissions for limited
10:02amount of time that they're operating in
10:03number one. Every agent has a goal,
10:05every agent has a job. So, you should be
10:07able to deterministically say that this
10:09agent should have access only to those
10:12tools or only to those jobs it has
10:14access to. So, gone are the days when
10:17the broad scoped auth scopes that we
10:20defined is okay because in that case
10:22developer was writing a deterministic
10:25application and you can review the code
10:27to make sure that he's not doing
10:28anything sinister. But in the case of
10:30agents is extremely non-deterministic,
10:32it is probabilistic. Agents are bound to
10:35do things whatever they can get a hold
10:37of. So, in the context of when you're
10:40giving access to the agents,
10:42you should be in a position to give
10:44extremely fine-grained scopes. It should
10:46be at an attribute level scoping, it
10:48should be context level scoping, it
10:50should be principal level scoping. So,
10:52all of that is extremely important.
10:54Again, I think everyone agrees that
10:55agents should be least
10:58privileged by default and they should be
11:00able to ask for just-in-time
11:02authorization if they want elevated
11:04scopes. Now, the reason why we're
11:06talking about this is because it's not
11:07some futuristic thing. It is happening
11:09today. We have seen enough incidents
11:10where agents end up doing rogue things.
11:13They end up deleting production
11:14databases and stuff like that. So, how
11:16do you put deterministic guardrails in
11:19place
11:20is an important problem to be solved
11:22right now. Again, one of our customers,
11:24ref.tools, they don't even have humans
11:26as actors. Their predominant product is
11:29about how to give context to coding
11:31agents so that they can do their job
11:33effectively. So, they built the entire
11:35OAuth scoping. How do you do things the
11:36right way and things like that. So, the
11:38reason why I give this example is not to
11:40say that this is a warning shot, but
11:42this is a problem of today and not for
11:43tomorrow. Before I go, you have to have
11:46absolute visibility into what your agent
11:49can do, every action that's taken in
11:51your system, who took it, on behalf of
11:54whom, and who authorized it, when was
11:56the authorization given, what
11:59authorization was given, how long is it
12:02given for.
12:03If you don't have visibility into all
12:05these actions at every single time, and
12:09if you can't deterministically control
12:10what your agent can or cannot do, then
12:12you're just praying that agent doesn't
12:14end up doing what it's not supposed to
12:16do. And praying is not a strategy, as we
12:18all know. One last thing to take away.
12:20If you architected so far with humans
12:23and APIs in mind,
12:25you need to start rethinking about how
12:28you need to give deterministic
12:30guardrails and deterministic
12:31authorization controls to the agent.
12:33And
12:35OAuth is a good place to start, but you
12:37need something beyond OAuth to make sure
12:39that the agents have extremely
12:41fine-grained access controls, and agents
12:42are always acting by themselves on
12:44behalf of certain users.
12:46Thank you so much for your time.