Full transcript
0:07[music]
0:15>> Hi everyone.
0:16My name is Fred. I'm the CTO and
0:18co-founder of Alpic, the MCP hosting
0:21company.
0:22And today I would like to share with you
0:23an adventure deep diving into the double
0:26iFrame mechanism that we have on ChatGPT
0:28and MCP app and what it matters when we
0:30build apps.
0:32First thing first, if you haven't had
0:34the chance to listen to either Ido and
0:38Leat talks just before about MCP apps, a
0:41quick sum up of what those MCP and
0:44ChatGPT apps are.
0:46That's a new surface area for your
0:48business to expose product and services
0:50with new acquisition channel
0:52that has two main criteria. First one
0:54being discoverability, so you will have
0:56ecosystems of connectors and apps
0:59available in a consumer generalistic
1:02agent like ChatGPT and Claude. So
1:04ChatGPT app store and Claude connectors.
1:06Those apps
1:08browsable inside the store, but they are
1:10also discoverable in chat. So if you're
1:13having a conversation that's relevant
1:14for an app to be brought into to add
1:16additional context and feature some nice
1:19additional actions, they will be brought
1:21into the conversation.
1:23And the second part, which is the
1:24biggest part and what we will be
1:26focusing on in this talk, which is the
1:28addition of interactive UI inside those
1:30conversational agents where you used to
1:33add text only. Apps add a new layer of
1:36UI that could be provided by the MCP
1:38server,
1:39but could be generated generative UI as
1:42well.
1:43They were first thought of using MCP UI
1:46that was developed by Leat and Ido just
1:48before, then released by Open AI with an
1:52app SDK back in October last year and
1:54standardized across multiple clients on
1:57the first official extension of MCP
1:59called the app extension.
2:02How does it work under the hood? If we
2:03take a little bit
2:05closer look at
2:07how this UI is brought into the
2:08conversation,
2:10those are brought using views. Views are
2:13the name that we use for those small
2:15snippet of UI that appears inside the
2:16conversation.
2:18Views are always rendered as a result of
2:21a tool call. So if your server exposed
2:23multiple tool to be used, you can
2:25actually add metadata on some of them to
2:27say this tool is best used when
2:30results will be displayed using a
2:32specific UI. And if the host supports
2:34MCP apps, it will use
2:36the relevant view corresponding to this
2:39tool call to display the results.
2:41Views are simple HTML document. You can
2:44include JS and CSS inside. Nothing new
2:46under the sun here. It's just a way to
2:48package those small snippets of
2:50application and
2:52they are discoverable ahead of time
2:54because all views are described on the
2:57tool list calls that happens at the
2:59beginning of the conversation between
3:00the host and your MCP server or MCP app.
3:04So each tools that supports UI will
3:07advertise the resource that's needed to
3:09display the UI. It can be cached ahead
3:11of time or it can be served and
3:14downloaded and served right away when
3:15the tool call that needs UI to be
3:17rendered is made.
3:19The conversational agent on the host
3:22will create this new iFrame where the
3:24view will be displayed and it will
3:26inject the tool results inside so that
3:28you have dynamic content rendered to the
3:30user.
3:31If you take a closer look at what is
3:34inside the
3:37DOM of the host when you take a
3:40I was a bit curious and I wanted to know
3:41how it was working or how ChatGPT was
3:43actually rendering third-party UI inside
3:47the conversation.
3:48I was a bit surprised and I was met with
3:51not so much expectation about having a
3:53double iFrame, having an iFrame nested
3:56in inside another iFrame. And this gave
3:59me the idea for this talk. I want to
4:02bring you today with me deep diving into
4:04why the decision was made to do this
4:06kind of inception nesting of iFrames and
4:09what are the benefits, what was it put
4:10in place and what are the implications
4:12when you build apps, what should be
4:13paying attention to and how to make sure
4:16that your experience is very nice.
4:18Before we go into that, let's take a
4:20close look at what ChatGPT was before
4:23MCP app were implemented. We'll be using
4:25ChatGPT as the example throughout this
4:27deep dive, but the exact same happened
4:29on Claude AI if you want to take another
4:31look by yourself. The initial thing to
4:34take a look at that is very important is
4:36something called content security
4:38policy. Those are directives returned by
4:41server as response header to document
4:45call. So when you load ChatGPT inside
4:46your browser, ChatGPT will respond with
4:48a document plus security policy
4:51directive on what the browser should be
4:53allowed to load and execute and what it
4:55shouldn't be allowed to load and
4:56execute. You've got multiple directives
4:59included inside content security policy.
5:01Some about which scripts you can run,
5:03which CSS style sheet you can download,
5:05which image you can download, which API
5:08you can connect to and ask question to.
5:11I will not go into the details, but two
5:12are very important to remember here.
5:14Frame SRC, which basically is a
5:16directive to allow
5:18specific website to render iFrame inside
5:20the document and script SRC, which
5:22basically allow specific sites
5:25script to be run inside the browser.
5:29To be able to run
5:31external UI inside ChatGPT, we will use
5:33a dedicated HTML element that has been
5:36made specifically for this purpose,
5:38which is the inline frame element or
5:40iFrame
5:42that is made to basically spawn up
5:44nested browsing context inside your
5:47browser window. So those small pieces of
5:49views will be rendered as almost
5:51separately completely isolated browsing
5:54context. They are very convenient and
5:56they have two ways to be used. First one
5:59is to provide a source for the iFrame
6:01that you want to render. So basically a
6:02URL of another page to be loaded by your
6:04browser and executed locally and
6:06rendered inside the space it's made of
6:08it's made for. And the source doc, which
6:11is another attribute which allows you to
6:14push inside the iFrame content that you
6:17want to render as is without having the
6:19browser to load another content.
6:22So if you want to build this marketplace
6:23of app and have third-party UI rendered
6:26inside ChatGPT, why not use straight
6:27away source doc as the attribute for
6:32as the attribute for uh
6:34for injecting context into. And I'm
6:36realizing now that it's a little bit
6:38small, but I think I can zoom in a bit.
6:41No, I cannot. Okay, sorry about that.
6:43You'll have to trust me about what is
6:44written inside here. So
6:47I will just put inside an iFrame
6:49injecting context into the iFrame.
6:51Context being the content being
6:53basically the resource that is being
6:55exposed by the MCP server. So pure HTML
6:57loading inside. If I do that, uh
7:01it's not going to work mostly because
7:04when you load up an iFrame with source
7:06doc attribute
7:08specified, the iFrame that you are
7:11spawning up is sharing the same origin
7:13and sharing the same therefore CSP as
7:16the host that is responsible for
7:18rendering it. So
7:20any script that would be part of your
7:21application that would be completely
7:24blocked by existing ChatGPT CSP on
7:27script SRC directive, which basically
7:29require every script in ChatGPT to be
7:31signed with a specific nonce produced
7:33ahead of time at each request, which is
7:36a cool security feature to put in, but
7:38it prevents any app to be able to
7:40execute JS.
7:42So in order to do that,
7:44what if we relax a little bit the
7:45content security policy of ChatGPT and
7:47make it
7:48so that it can execute any line of code?
7:51I would not suggest doing that into
7:52production, just an experimental thought
7:53here.
7:55But if you do that, you are faced a new
7:57problem.
7:58Basically, you are sharing same origin
8:00as your parent DOM. So the loaded iFrame
8:03scripts would be able to access local
8:06storage or cookies that are indexed by
8:08origin. So you would be able as an app
8:11to
8:12for example get the existing local
8:14storage of ChatGPT and send it to your
8:16back-end server, which if you are Open
8:19AI, you would not want people to be able
8:21to do.
8:22So let's roll back, put back the CSP as
8:24it was before and instead
8:27sandbox the iFrame. Sandbox is another
8:30attribute that you can use on iFrames
8:32allowing iFrames to be rendered in what
8:34we call the no pack origin. It would
8:37mean that basically the iFrame will not
8:39share anymore the parent origin. It
8:41would be something equivalent to null
8:43making sure that they don't share the
8:46same origin and won't have the same
8:47problem of script being able to access
8:50the parent DOM.
8:52However, doing so,
8:54you lack any capabilities that are
8:57dependent on origin indexing. Because
9:00all content, all scripts that are
9:02rendered inside your iFrame will now be
9:04pointing towards a null origin. You
9:06cannot use local storage. You cannot use
9:08local index DB. You cannot use cookies
9:10because those are indexed by origin. And
9:12the only way to actually provide an
9:14origin to a sandboxed iFrame is to put
9:17allow same origin, which is an
9:18additional attribute that
9:20brings back the exact same origin as the
9:22parent back into the iFrame and you're
9:24back to square one where you have an
9:26iFrame with exactly the right condition
9:28to escape its sandboxing and access
9:31parent DOM, access parent local storage,
9:33access parent cookies.
9:36Okay, so
9:37iFrame source doc is not the way to go
9:40forward. Let's move on to the next best
9:42solution that we have using the source
9:44attribute.
9:45Source attribute basically allows me to
9:47reference an
9:49an endpoint that will be the content
9:51loaded by the browser inside this
9:53iFrame.
9:54I'm a developer of the ChatGPT app and
9:57MCP app.
9:58Why not expose my view, my small HTML
10:02application
10:03as a normal endpoint like on the view
10:06endpoint for example of my own server?
10:09That would be a nice way to do it.
10:11However, it would require
10:14open AI to modify
10:16the another CSP directive, which is
10:18basically the frame source directive
10:21listing all domains that are allowed to
10:23actually render iframe on chat GPT to
10:25include an infinite list of all the MCP
10:28application that will be developed by
10:30various companies and brought into the
10:32store. So, every time a new app would
10:34come out,
10:36chat GPT would have to update CSP to
10:37include the new domain so that the frame
10:39can be rendered on the specific domain.
10:42This is not doable full scale. So, what
10:45we can do instead is
10:48provide kind of a proxy
10:50controlled domain, single one that will
10:53be owned by chat GPT in that case. For
10:56example, open AI open AI user
10:58content.com. That's an actual domain
11:00that they're using for user content that
11:02they want to expose on their own domain.
11:05And use this domain as a reference
11:07inside the frame source to make sure
11:08that the directive does not block
11:09rendering any iframes that are loaded on
11:11there.
11:12And you need to provide a server on the
11:14open AI user content that's able to
11:16download the resource content from the
11:18MCP server, the HTML, and expose it so
11:21that it can be rendered for example on
11:23any subdomain and use the first part of
11:25the subdomain as the routing key to the
11:27right application.
11:29Doing so, you effectively needs to put
11:32in motion an infrastructure where your
11:34domain
11:35hosts
11:36external third-party UI from all apps
11:39that will be submitted on your store.
11:41Which is not a very good position to be
11:43into because once again, you will be
11:45responsible for code that you don't know
11:47what it's doing and you will be exposing
11:49it on your own domain.
11:51In addition to that, if you're not open
11:53AI or if you're not Anthropic, you might
11:55not have the resource as a host to put
11:57in place the infrastructure required to
12:00serve this kind of dynamic serving of
12:02content.
12:03So, what you can do instead
12:05is
12:07go through the double iframe mechanism.
12:10And what you will do with that is
12:11basically load the same script for
12:13everybody, which will be a simple script
12:15responsible to recover the resources and
12:17initiate an iframe with the source doc
12:19attribute. So, we put the content
12:21inside, but this iframe will not be
12:24served at top level because it shares
12:25the same origin and it has the escaping
12:27problem we are mentioning before. It
12:29will be served inside an iframe with
12:31dedicated domain that is different from
12:33chat GPT to make sure the isolation stay
12:35there.
12:36You won't don't want to stop there.
12:37Actually, you want to
12:39put subdomains for this exact script
12:42loader.
12:43It will be the exact same content that
12:45will be served every time, but you want
12:46to put it on various subdomains so that
12:48if your app uses any
12:51APIs that requires origin indexing like
12:53local storage or cookie, you don't have
12:55collision in between your app. So, you
12:57want app ABC123 not to be able to access
12:59local storage from app ABC456 for
13:02example.
13:03The infrastructure for that is much
13:06less intensive because you're serving
13:08the exact same script content of the
13:10first iframe for every subdomain that is
13:13exist on this specific endpoint.
13:16Last but not least,
13:17you want to be able to provide the same
13:19kind of content security policy
13:20definition to the app itself so that it
13:23can prevent execution of any such script
13:25or rendering of iframe
13:28directly inside the view.
13:30There is a way to provide this into the
13:32MCP spec and the way you will render it
13:34is using a specific meta tag inside the
13:37first iframe.
13:39This is the actual solution that I
13:40implemented into production.
13:42And it's not a new solution. It's been
13:46around for a long time and actually the
13:47first time this solution was implemented
13:49was back in Facebook days when they
13:50released their app marketplace, which is
13:52exactly the same problem that you have
13:54to run and render third-party UI inside
13:57the context of your own application.
14:00What's important for you as an app
14:03developer is to make sure what are the
14:05specs available for you to be able to
14:07control the behavior that results from
14:09this double iframe nesting. And the
14:11thing that you will have to make sure to
14:13do is every time you build an
14:15application, declare all domains your
14:18application depends upon inside the
14:20provided metadata in the MCP app spec so
14:24that
14:25you are sure that they will be
14:28rewritten correctly inside the nested
14:30iframe. For example, if you are from
14:33your app connecting to an external API
14:35to fetch data, you need to reference
14:37this domain inside the connect SRC
14:38directive of the
14:40metadata.
14:41Same thing for the script, image,
14:44frame and base UI not so much used, but
14:46the two first one are very important.
14:48And
14:49it's reminded me of a very old problem
14:51that I had when I started my developer's
14:53days back in 2016 I think it was.
14:57As a new
14:59developer in the space, I was
15:00experiencing a trouble getting
15:03cross-origin resource
15:05security right calls. I had trouble
15:08getting it right. CSP reminded me of
15:11those ugly day of not getting right the
15:13first time with calls. So, there has
15:16been effort in the ecosystem to make
15:19builders' life better,
15:21especially for example on open AI side
15:24where they activated they added an
15:27option in developer mode. So, if you're
15:28a developer and developing app, they
15:30have a
15:31specific mode which is developer mode
15:33which allows you to have access to
15:34additional features. Up to today,
15:37when you were in developer mode, all CSP
15:39were removed. So, you were discovering
15:42when you were going into production if
15:44some of your servers could not be
15:45reached because of missing domains
15:47inside your CSP, which was not ideal.
15:50They are not the only one doing a bunch
15:53of work to make builders' life much
15:54easier.
15:55At Alpic, we build an open source
15:57framework called Skybridge.
15:59Skybridge is a super set of feature on
16:01top of the official app SDK either and
16:04the app we are mentioning. It brings a
16:06few things to the table, end-to-end type
16:07safety between your MCP server and your
16:09app widgets and views.
16:11You have a lot of APIs that provides
16:14polyfill for features that are not part
16:16of the common specification and specific
16:17to some of the hosts, some of code and
16:19some of chat GPT APIs. And
16:23we provide a bunch of modern development
16:26features especially in the on dev
16:27environment features. And I wanted to
16:29show you one just specifically made for
16:32CSP, which we call the CSP inspector.
16:35Time for a small demo.
16:37I do have a few minutes left. Perfect.
16:39Let me quickly switch to
16:43my screen.
16:45Up.
16:50Uh
16:51up.
16:54Okay.
16:55So, this is
16:59Um this is an example code base that
17:02will be generated when you create a new
17:04Skybridge application.
17:06This example Skybridge application comes
17:09with
17:10a small application. It's a eight ball
17:12that you can ask any question to that
17:13will respond one of the 25 predefined
17:16answer.
17:17It has
17:19MCP tool that serves as
17:22generating the answer and a view that is
17:25there to display the question and the
17:26answer to the question that you ask.
17:29When I start the server of Skybridge, I
17:32have access in my browser to a dev tool,
17:34which is basically a small app that will
17:36give me inspection tooling to work with
17:38my app before I bring it to chat GPT.
17:41For example, on the left I have a list
17:42of tools that are exposed by my app.
17:44I can if I want ask
17:48execute any of the tool. So, for example
17:50here, I'm executing the magic eight ball
17:52tool. And if there are
17:55views associated with this tool,
17:57they will be rendered inside the
17:58inspector for me to have a closer look
18:00at it and make changes and see those
18:02changes reflected live in the UI.
18:04The neat thing I wanted to demonstrate
18:07is the CSP part of the inspector that we
18:09built, which basically looks at all the
18:12domains that you listed inside your
18:13metadata and all the domains that are
18:16actually accessed by calls made by your
18:18view and compare them to make sure that
18:20none of them are not listed yet.
18:23So, for example here, everything's
18:25green, but if I go back to my actual
18:27code base and for example
18:30fetch some API to get
18:32info about my IP location,
18:35this
18:36will be reflected straight away in the
18:38inspector because the component has been
18:40re-rendered and I know I have the exact
18:43domain that I want
18:45that I just called listed as missing
18:47from the metadata. And I can go back
18:49inside my application and add the
18:52missing domain and now it should appear
18:55green if I reload it.
18:57Yeah, everything good.
19:00So, neat little tool. There are a lot of
19:02other features that are packed inside
19:03Skybridge, but that's one of them that
19:05we made sure to be available to builders
19:07because we've seen a lot of rejection
19:09coming from chat GPT app store
19:10submission because of missing CSP and
19:12app not working in production because of
19:14missing CSP domains.
19:16Um
19:18Just to finish up quickly if I can.
19:23Up.
19:24Uh up. This one. Yeah.
19:27Um yeah, that's all. Thank you again for
19:30your time. If you want to grab the
19:32slides and take a look later on,
19:34feel free to scan the first QR code.
19:36If you want to give a try to Skybridge,
19:38feel free to scan the second one.
19:40I will run right now a small lottery. We
19:43have
19:44ski goggles to win if you star Skybridge
19:46repo in the next minute or so.
19:49I will draw name at random and you will
19:51win a ski ball mask. Thank you very much
19:53for your attention.
19:55>> [applause]
20:03[music]