Full transcript
0:07[music]
0:15>> And even though I know some people are
0:16still coming in, I've also been informed
0:18to start directly on time. So much
0:21apologies to anyone who was hustling up
0:23here from one of the talks downstairs.
0:25You should have known this was the best
0:26track and hung out here from hour zero,
0:29but uh by all means come in, have fun.
0:32Hi there. I am Rachel Lee, Rachel Lee
0:34Neighbors, Neighbors like Nearest
0:36Neighbors, but not spelled that way. I
0:38have worked on the standards that powers
0:40today's web today's web with Mozilla on
0:43Firefox DevTools, the W3C on web
0:45standards like the Web Animations API.
0:47I've even been a PM on Microsoft's Edge
0:49browser, and you may know me from my
0:51pandemic babies on the React team,
0:53react.dev and reactnative.dev.
0:56All right, so I've spent the last 3
0:58years consulting with AI startups and
1:00LLM / big browser companies, some of
1:03whose logos you may recognize, most of
1:04whom you will not,
1:06on all things web, AI, and UI. Recently,
1:10I've just become newly minted principal
1:12developer experience engineer at a rise.
1:15Uh have you ever had your CTO ruin your
1:17agentic flow with a slight change of
1:19prompt or an LLM migration? Have you
1:22ever been that CTO?
1:24Anyway, if that is you, you need to
1:27check out a rise. Evals are your
1:28friends, check them out.
1:31Now, before my big tech career, I made
1:33web comics for teenage girls on a well,
1:36it was a it was a part of the iVillage
1:38Network website. I don't think anyone
1:40remembers iVillage, but it was kind of a
1:42big deal back in the day. I had 400,000
1:45weekly readers. It was powered by ads,
1:48and I learned how to build for the web
1:50so I could reach those people where they
1:52were. It was 2006, and it was a mood,
1:56let me tell you. Everyone online was
1:59edgy. We were on MySpace where we had
2:0114-year-olds talking about music with
2:0342-year-olds, and as you can see, I was
2:05really into the goth and emo scene.
2:08I was the only goth in my village.
2:11>> [clears throat]
2:12>> When I got into the web, I believed that
2:14the browser is not just a document
2:16reader, despite what the CSS wonks will
2:18try to get you to believe. It is an
2:20infinite canvas that can render
2:22anything, not just documents, but video,
2:24audio, whatever you need, there's an API
2:26for it, and I have been working to that
2:28end on the web ever since. I worked on
2:30that Web Animation API with the W3C
2:32creating such insane demos as the
2:35interactive Alice in Wonderland, uh and
2:38generally made the web do things that it
2:40was not supposed to do.
2:43Uh
2:45And now I write about how agents and AI
2:47are changing how we browse and consume
2:49content on my little newsletter, uh The
2:52Agentic Web.
2:54And I do a fair bit of move shenanigans
2:56with agents myself. This is uh ASA, it
3:00stands for Anti-Social Social Agent.
3:02It's an MCP app that lets you use
3:04Twitter and BlueSky next to each other
3:06without actually having to visit the
3:07sites or be subjected to other people's
3:09algorithms. Anyway, check it out if you
3:11like it. Uh I'm here today to show you
3:14how, with the power of the browser's
3:16primitives that already exist and have
3:18already been specked out right now,
3:20you can turn your agent into an infinite
3:23canvas, or at least get your website
3:25into agents,
3:26one way or the other.
3:28So, I have this habit. Every year when
3:31I'm thinking, "What do I want to make
3:32talks about? What do I want to do?" I
3:33look at the one thing on my laundry list
3:35that I keep putting off.
3:37In this case, my web comics site, the
3:39archive site that I put together at like
3:412010 said, "Okay, you're good there. I
3:43got to go now." And it's starting to get
3:44404s, starting to get some broken
3:47images. Maybe I shut down some clusters
3:50here and there, and it's like, "Oh dear,
3:52the CDN has failed." Anyway, it needed
3:55some repair, and I was like, "Well,
3:56don't just repair it, make it
3:57future-proof."
3:59If everyone is working in agencies these
4:01days, let's meet them where they are.
4:03I'm always meeting my audience where
4:04they are, and I still get fan mail from
4:06the women who grew up with my comics, so
4:08maybe I can get them to try out agents,
4:10too.
4:11All right, so just as a cringe warning
4:14before you go to rachelagreat.com,
4:172000s were a vibe.
4:19Old comics, younger Rachel Lee,
4:22just a little bit cringe, okay?
4:24I want to honor the fans,
4:27and also I'm working a lot with
4:30copyrighted images here, and I don't
4:32want to get sued by Marvel, so I'm going
4:33to use my own stuff. The nice thing
4:34about having a former career as an
4:35artist is I can do whatever I want with
4:38my art, no questions asked. Of course, I
4:41did have to hose them down with glazing
4:42to make sure nobody else would, but
4:44that's another story. Check out the repo
4:46at the end of this if you want to know
4:47more about protecting your content from
4:49being used for training purposes. Now,
4:52I wanted to make my comics available and
4:54navigable, one server, but three
4:57different kinds of clients, humans in
4:59browsers, humans in agents, and agents
5:02in browsers using the web on behalf of
5:03their humans. I know, it's complicated.
5:05They've got like this polycule thing
5:06going on. So,
5:08what we're going [clears throat] to
5:09cover today.
5:11MCP tools and transports, how to host
5:13them on the web just like you would a
5:14website.
5:16MCP apps, how do they work? You've
5:18already been told a lot about this. I'm
5:20just going to show you a really cool
5:22example and point out the foot guns.
5:24And Web MCP, coming to a browser near
5:27you. Before we go, how many of you
5:30currently host an MCP server?
5:34Like on the web, not locally.
5:36Okay, well, I love you guys. Everyone
5:38else, you're going to learn. And how
5:40many of you have made an MCP app?
5:44All right, how many of you want to make
5:45one?
5:46Uh yeah, this is going to be the next
5:48web pages. And lastly, has anyone here
5:50played with the
5:51with Web MCP?
5:53Okay, well, we'll see if we can change
5:55that.
5:56So,
5:57you're here. I assume you know what an
5:59MCP server is. I'm I'm not going to go
6:01into them too much. Um
6:04So, we're going to get started with
6:05transports. Now, if you've built for the
6:08web, I guess I should ask, how many of
6:09you have built websites and hosted them
6:12in your lifetimes?
6:14All right. So, you're familiar you're
6:17probably familiar with HTTP, but you
6:18might not be familiar with the concept
6:20of transports. It turns out there's more
6:22than one protocol you can use to
6:23communicate from one surface to another.
6:26Transports are how MCP servers
6:28communicate with agents. And there's a
6:31important difference between when you
6:32should use
6:33either of these. So, STDIO stands God,
6:37it sounds terrible when you read it out.
6:39Standard input output, you can call it
6:43>> [laughter]
6:45>> studio. Like the Phil Collins song, cuz
6:48we're in England, okay? Studio. We're
6:51going to call it that for the rest of
6:52this talk.
6:53Unlike my German proofreader who was
6:55like, "I don't think you call it
6:56studio." I was like, "Dude, you want me
6:58to call it anything else? I think we're
6:59going to have problems." All right, so
7:02the server, it runs as a local process,
7:04it's spawned by the client. This is why
7:05when you're wiring up studio-based
7:08MCP app, it's always like a string of
7:10JSON with a bunch of command-line inputs
7:12that most of your users don't want to
7:14use, don't want to configure.
7:16Uh communication, it happens via as I
7:19just said, standard input and output,
7:22and the server stays alive throughout
7:23the session. HTTP is a little different
7:25cuz we're running over the web. We are
7:27not touching with a 10-ft pole, by the
7:30way, any of the security or privacy
7:31concerns with either of these layers.
7:33That is beyond the scope of today's
7:35conversation. But HTTP, the server runs
7:38as a web service, it listens at an HTTP
7:40endpoint which you may be familiar with
7:41from API development, and communication
7:44happens via our best friend HTTP post
7:47requests,
7:48and it works well with serverless
7:49setups. You know, Vercel keeps and and
7:52Cloudflare keep trying to sell you those
7:54edge functions. Well, they got something
7:56to put in them now.
7:57All right.
7:58>> [clears throat]
7:59>> With studio, you have to update a config
8:01file with a command to spin up a local
8:03server, but with HTTP, your user's expe-
8:06Oh my god. Your user experience is as
8:09simple as going over to your own
8:11settings. Okay, it is buried a little
8:13bit, but trust me, when you get Oh my
8:15god.
8:17Please, if anyone from Anthropic is
8:18watching this, help. Give it a name, and
8:21put in a URL,
8:23and it is now
8:25Come on, hurry. Come on, come on, come
8:26on, come on. Past Rachel Lee, but you
8:28can see I can actually type the damn
8:30thing out. It's very It That's not how
8:32you spell it. There you go.
8:35It's [clears throat] very There you go,
8:36I put it at /mcp. You hit add, and now
8:40there it is. It's a connector, and I go
8:43ahead and go in, and I I give it all the
8:45permissions to always allow because I
8:46know this one.
8:47But it's not easy
8:49if you're using an HTTP server for your
8:51MCP tools for the user to get them into
8:54their
8:55in this case, Claude.
8:57All right, so these are the tools I
8:58wanted to make available.
9:00They look a lot like what's available
9:02when they visit the website. They're
9:04very similar to that navigation, and
9:05they all return structured data in the
9:07form of JSON, except for get transcripts
9:10here, and we'll talk a little bit more
9:11about this guy in a little bit. But
9:13point is, list the comics, list the
9:15storylines because the comics include
9:17different story arcs and endings and
9:19beginnings. List the characters, search
9:21said comics, which is actually searching
9:23the transcripts, that's a different
9:24talk,
9:26and search [clears throat] by character
9:27because I love the cat, I want to see
9:29all the cat. Uh and the transcript is
9:31different, it returns markdown.
9:34Boom. All right, let's take a look at
9:36these babies in action.
9:38These are just the tools. We haven't
9:39gotten to the interactive part, bear
9:41with me. I have a narrative arc.
9:46All right.
9:48It is a bit wordy. Everyone who's like,
9:50"Chat's the future." I'm like, "It is
9:52not the future."
9:54This is like, "Thank you for having a
9:57long discussion with me, Claude. Yeah,
9:59you did find that I make the comics
10:01good, Claude. Good baby.
10:04I would like to make a
10:07I would like to make the full
10:09transcripts and text available to the
10:10context for meta-analysis of themes and
10:13character arcs, etc. instead of asking
10:15Claude to like get transcript from 533
10:18different comics.
10:20It does It is not an efficient use of
10:22context to do that. I would really love
10:24to use MCP resources, which are the
10:27perfect vehicle for this and skills and
10:29so many other things, but the spec is
10:31loosely defined, it is loosely
10:32implemented, and I cannot find a client
10:34where I can actively get to these
10:35resources. They're on the server. They
10:38just ain't visible in your UI. So,
10:41please if anyone here is listening who
10:43builds an agent harness, go and even
10:47bare-bones, I'll take it. Give me
10:48something to work with here. All right.
10:50So,
10:51I would love for one or more clients to
10:53implement resources. Agents really don't
10:56like using MCP tools to go add markdown
10:59to their context. If you've ever
11:00maintained a documentation site and you
11:03thought, I will add the MCP tools to the
11:05documentation site, that just solves the
11:07problems with
11:09the coding agent knowing how to No. No,
11:12I would rather die in a fire than go
11:14call your documentation by MCP tools.
11:17So, let That is not how they were
11:19intended to be used by God.
11:21>> [clears throat]
11:22>> So, resources could also allow harnesses
11:23to decide when to pre-prime that context
11:25with docs. Like you're switching over to
11:27react mode, good. Go get all the react
11:29resources from react's MCP server. Thank
11:31you. Goodbye. All right, little tirade
11:33over. That was your public service
11:35announcement. Uh tools,
11:38they can return plain data, JSON, text,
11:40markdown, but they can also return an
11:42interactive app app app app app
11:44>> [laughter]
11:45>> So, I wanted to put a comic reader in
11:47your agent. So, while you're reading,
11:48you can read some things with some
11:49pictures instead of a long wall of text.
11:51It's like, Claude,
11:53you are not my my best friend in crisis.
11:57Why are you wall of texting me right
11:58now?
11:59>> [clears throat]
12:00>> It's been said that chat is the lowest
12:02common denominator of the user
12:03experience. That it is to the future of
12:06agentic experiences what the CLI was to
12:09software. My mother programmed in COBOL,
12:12and when at school they were making us
12:14click double-click on icons to access
12:17video games, she was like, we do not do
12:19that in this household. We open DOS and
12:21we do CD blah blah blah.exe. That is how
12:25programmers do things.
12:27But we all know that you don't do that
12:30on an iPhone, you tap and you point and
12:32grunt, right? So, this is probably a
12:34phase for us developers. There's a
12:36current what I call the chatbox landing
12:39page or starfish design because it sits
12:41there and it lets the user do all of the
12:43work. And you have to have a lot of
12:45context in your head when you land. You
12:47have to kind of know what this thing can
12:49do. Like if you're on linear, you're
12:51like, I know this is a ticketing system.
12:54I'm pretty sure I want to see XY or Z.
12:56It's great if you're already familiar
12:57with the system, but otherwise it can be
12:59a bit challenging not to have any visual
13:01cues or guidance. It's sort of like
13:03these text games were really popular
13:05back in the day, but I wouldn't want to
13:07have to do my taxes by being like,
13:09check mailbox. Is there mail? Whose
13:12house is this? That's not great.
13:14It's asking
13:16asking your poor customer to do a little
13:18bit too much discovery. So, fortunately,
13:21we have MCP apps
13:23inline in agent rich media experiences
13:26created by bundling of HTML, CSS, and
13:28JavaScript into a single file. It sounds
13:31so familiar. I'm having flashbacks to
13:33like DHTML.
13:36Huh. So, this is get page. Get page is
13:39my MCP tool baby, and I love it very
13:42much. This is it in action. It returns
13:44an MCP app. I say, "Yo, I want to read
13:47Crow Princess the comic to the starfish
13:50um
13:50chatbox of everything."
13:53And of course, Claude's getting
13:54chatting. It's like, "I'll help you
13:56read. Oh, let me just check all of my
13:58MCP things. Wow, I found it in the other
14:00comics collection. Let me go get page
14:02that." Here we go. Okay, thank you.
14:05And check it out. I can show the
14:08commentary. I can show the comments.
14:10That's pretty cool. Those are available
14:11on the website. It's everything the
14:13website can do. It looks just as good as
14:14the website. You can navigate forward
14:16and backward. But the fun part is you go
14:19up here, you can click on text mode, and
14:21boom, it shows you the transcript. So,
14:23you could just read it if you wanted to.
14:25That'll come back later. So, yeah, it's
14:27like a little mini website.
14:29Comic reader, but it's using all the
14:31resources from the website.
14:33Okay.
14:34Now,
14:36ha,
14:37>> [sighs]
14:37>> how do you do that? Well, I define the
14:39tool as get page, give it the
14:41description. There's an input schema. Of
14:43course, I'm going to have to take the
14:45comics ID, the storyline ID, the page
14:47number, so I can really hone in on where
14:49to get this. To be honest, it just needs
14:51a storyline ID and the page number. I'm
14:52going to refactor that.
14:54And of course, all of this stuff is
14:55required.
14:57But most importantly, down here where
14:59the magic happens is you pass this
15:01little meta attribute, give it UI, and
15:04you point it you point it at a universal
15:06resource locator. Boom, right there.
15:09Love it. Okay.
15:11Important thing to keep in mind is that
15:13MCP apps are a little special. They're
15:15not like websites. They're They're an
15:16island.
15:18They're single HTML files. So,
15:19everything that you need there has to be
15:21embedded base64. If you're going to link
15:23to something external like a cached CDN
15:26font file or something, you got to add
15:28that to essentially a course policy.
15:31So, this means you're not going to be
15:32dynamically pulling in stuff from other
15:33people's servers unless you really are
15:35certain about that and enabling it in
15:36your app. It's sandboxed in an iframe.
15:40You don't get local storage with this.
15:41So, you're not going to be tracking
15:42state across iframes. You're stuck with
15:44it. Um async tool calling.
15:48Okay.
15:49And no network access. Ask You have to
15:52ask server to do things for you with the
15:54call server tool. This gets into a real
15:57mother may I situation, which I'll cover
15:59in the gotchas. So, what does this mean?
16:02It means that
16:04becomes super useful to build your MCP
16:06app with your design system. In this
16:08case, I'm sharing fonts, CSS,
16:12and everything from the same server. Got
16:14the course policy set up and everything.
16:16And the HTML and JavaScript components
16:18can be bundled. I'm using V single file
16:20for this. So, if you already have a
16:22design system, put your paw up if you
16:24do.
16:26Okay, good. All the rest of you, God,
16:28I'm sorry for you.
16:29But a design system is a thing of
16:31beauty.
16:32All right, bonus round.
16:34Ha.
16:34>> [sighs]
16:35>> Links require host permission. Remember
16:37I said mother may I? You can't just
16:39window open or, you know, href. No, no.
16:43No, no. You got to be like, appref
16:46current open link pretty pretty please.
16:49Okay.
16:51Remember I said external resources
16:52needed course? Well, they got this thing
16:54called the CSP or content security
16:56policy. Fonts, images, other external
16:58resources, they're blocked by default.
17:00So, without that CSP config, the browser
17:02would just not load those things. So, if
17:04you're getting a blank look and you're
17:05wondering what's jacked up, check your
17:06CSP.
17:07And unlike MCP UI, which you may be
17:10familiar with, MCP apps can call tools.
17:13So, I could do that forward and backward
17:14navigation with a tool call.
17:16But if you're doing that, remember some
17:18tools were not meant to be viewed as a
17:20big fat pile of JSON. You want to set
17:24visibility app there just to make sure
17:26that the model's not going to try to use
17:28that and return text. It's not going to
17:30go, "Oh, look at this lovely forward and
17:32backward. I will use that." No, sucker
17:35arm. Don't do that. All right. So,
17:38people are still using browsers, right?
17:40There's smarter, more accessible
17:43browsers coming out all the time.
17:45Big and small browsers are shipping with
17:47agents built right into them. Sadly,
17:50they still struggle with navigating
17:51websites because they rely on taking
17:53screenshots and guessing how to use the
17:55navigation or traversing the DOM, which
17:58burns tokens chewing up XML, which is
18:02really not a great use. Both of these
18:05are compute-intensive cuz you're using
18:06visual models or you're just eating a
18:08bunch of HTML to find out where the next
18:10URL is. Wouldn't it be great if they
18:12just had a way to call the same
18:13functions and links you already have in
18:15your JavaScript directly from the page
18:17that they're visiting?
18:19Well, with Web MCP, they can do just
18:20that. It makes every HTML page a mini
18:23MCP tools server. However, I just want
18:26to point out that Web MCP is to MCP as a
18:30JavaScript is to Java. Inspired by
18:33riding the hype wave of But if you
18:36actually look under the hood, it's
18:37taking inspiration. It is not a
18:39one-to-one compliance with the MCP spec.
18:42It's really just like
18:43tools for agents in your browser, yo.
18:46All right. So, the specs might diverge.
18:48Keep that in mind. It comes in two
18:50flavors. Since my comics are not forms
18:52except for search, I use mostly the
18:53imperative model.
18:55But with declarative, you can just add
18:57tool name and tool description
18:58attributes to a form. Boom. If you're
19:00using JavaScript or functions to process
19:02that form, you just basically told you
19:05exposed that form as a tool to the
19:07agent. Great for sites with lots of
19:09forms already. If you're one of those
19:10people whose site is a dashboard, using
19:12the imperative model, it pass
19:14navigator.modelContext.registerTools
19:17for with parameters. It's great for API
19:20calls, workflows,
19:21data transformation.
19:23I used a lot of those. All right. So,
19:27>> [sighs]
19:27>> Now, if you if it's available, if you
19:30find model context in navigator, you can
19:33say, "I'm going to register a tool." You
19:35give it a name, give it a description,
19:37give it an input schema. My god, it's so
19:39similar to MCP. Amazing. How can this
19:41not be the same thing? But you also pass
19:43it an execution
19:45a callback. And in this case, this is a
19:49What do we do? Next page? This one
19:51basically grabs
19:52the link rail next from the page and
19:55passes it as the thing that you're
19:57supposed to navigate to next. So, you
19:59could call this to navigate forward and
20:00backward on the page with a browser
20:02without having to click around.
20:04Woo.
20:05This is new, so I can only help you
20:07imagine how it works because the
20:09standards bodies are still fighting back
20:11and forth on it.
20:12So, I'm going to show you how to use it
20:14using the MCP B extension. This is kind
20:17of like a debugging extension. You can
20:18see all of the little uh tools are
20:21exposed over there. I got a little chat
20:23window. This kind of imitates what
20:25people might see in Gemini or some
20:27future in-browser agent. In this case, I
20:30believe I am saying uh read this to me.
20:33It returns the transcript. And then I
20:35might say I might type in I uh thank
20:37you. Love that transcript. I might type
20:39in like go to the next page.
20:42Come on, go to the next page.
20:45Oh, I was speaking into the microphone
20:47at this point. That's fine.
20:48Anyway,
20:49>> [clears throat]
20:50>> and when I hit when I hit go, it takes
20:53me to the next page. That's just the
20:55model calling that function on its own.
20:57Yay! That's the wow. Uh don't applaud
21:00yet. So, best place to get started to
21:02check this out is actually with this
21:04interview I did with the creator of Web
21:06MCP. It actually started at Amazon. This
21:09was something he made to get around a
21:10bunch of auth issues and I love that I
21:12screen captured a
21:13window.navigator.registerTool.
21:15Uh
21:16there's standards community group, all
21:18very lovely.
21:19Um it's not fully supported yet, but you
21:22find all the resources in the commentary
21:24on that article.
21:25So, I started my journey with the
21:27agentic web with this talk like 2 years
21:30ago when I was like, "The browser's
21:32going to die." But what people didn't
21:33take away from the clickbait was that
21:35the web isn't dead, y'all. Like, it's
21:38just changing. It's evolving. CSS and
21:40JavaScript aren't just the language of
21:42the web. They're the language of
21:43interactive experiences on agents.
21:47All right. Here it is in action. I want
21:49to show you that
21:51there is a version of this
21:53that talks. Of course. Brenda looks
21:55skeptical while Rachel continues
21:57explaining. Rachel, "Ha, Gore left out
22:00some important tidbits." Rachel, "What
22:02about how plant life will respond to the
22:04extra carbon dioxide?" Comma panel two
22:07illustration shows tall plants and
22:09vegetation growing vigorously.
22:11I could keep going, but that's just the
22:12Web Speech API there. It sounds
22:14terrible. I'm sure you can do something
22:15great with 11 Labs. It's a browser's
22:17built-in text-to-speech capability.
22:19Zero-dependency solution. No no
22:21inference required for that. It's right
22:23there waiting for you to use it. So, are
22:26all the other amazing
22:28uh APIs available on the web today. So,
22:31all these APIs, Web Speech, Animation,
22:33Audio, Canvas, WASM, CSS, and more.
22:35They're already there waiting for you to
22:37make something with them. And I just
22:39can't wait to see what you do with this
22:41infinite canvas.
22:44>> [applause]
22:57[music]