Full transcript
Introduction: The evolution from 'poor man's bit coding' to high-fidelity UI generation
0:07[music]
0:16>> Hello everybody. So, I know I am the
0:17person standing between you and your
0:19lunch. And but this is going to be a
0:22very interesting talk that combines the
0:24the previous two talks into the future.
0:27And that's what I want to talk about
0:28today.
0:29So, back in November 2022, what we used
0:32to do was we used to go to ChatGPT and
0:34ask ChatGPT to
0:36create a component. And we would just
0:39copy
0:40paste. You have to ask reply in code
0:42blocks. Then, you know, again, fix it,
0:45repeat. And this is what I call the poor
0:48man's by coding.
0:50And we have come a long way.
0:53It kind of worked. It was very exciting.
0:57You could get models to could actually
0:59build some UI for you.
1:02And I'm sure that was not going to
1:05write better code than me, right?
1:07And then things improved very very
1:09rapidly, very very fast.
1:12What happened last year, and if you are
1:14aware of what happened in the the the
1:16last months of 2025 was this
1:18acceleration, an incredible inflection
1:21point where things changed. And it will
1:25go down in the history books as
1:27things changed very fast all at once.
1:30And this is in part because of the
1:32release of two very important models,
1:35which were
1:365.2 ChatGPT, sorry. It was yeah, a GPT
1:395.2 and Opus 4.5.
1:42And they were not just very good at uh
1:45most of the task, long horizon tasks.
1:48They were also very good at high
1:50fidelity UI generation.
1:53And they were producing very good
1:56working UI.
1:57Sometimes thoughtful, sometimes really
2:00really good. And also very fast.
2:05Now, I experienced this when I tried one
2:08of these models, tried to rewrite my my
2:10blog. I know people have used this in a
2:12more creative ways, but I just tried,
2:13you know, a single prompt, rewrite my
2:15blog. And then it did this, which I
2:18didn't ask for.
2:19It created a nice nice search box
2:22with a blur animation, with
2:25accessibility out of the box.
2:27And then that's when I realized that in
2:30the space of of 3 years from when
2:32ChatGPT was released to today,
2:35we went from
2:36you know, few lines of code is great.
2:40It can it runs. Oh.
2:43And now it can write
2:45better front-end code than me.
2:49And you know, I I don't mind. No no ego.
2:52Uh
2:53It's just reality. So, here's the
Why are we still stuck with static UI?
2:56question. If these models are so good at
2:58writing UI code,
3:01why are we still stuck in this mainly
3:04old
3:05paradigm of mostly static UI?
3:08And where is where is that Jarvis moment
3:11that we've been talking about earlier?
3:13Where are my floating UI windows that
3:15appear and disappear? And why we not
3:17there yet?
3:19So, my name is Ruben Casas. I am a staff
3:21engineer at Postman. And I've been
3:23looking at UI and generative UI for the
3:25past year, and I've been working with
3:26MCP apps as well.
3:28And today I want to show you what we're
3:30doing today and where we're going in the
3:31future.
The new computer: Searching for the interface of the future
3:33So, the news are we have a new computer.
3:36And as Andrej Karpathy put it,
3:39interacting with this new computer is
3:40like talking to the terminal. You have
3:43direct access to this operating system.
3:46And the GUI has not been invented yet.
3:48It's like we are in the '70s, where
3:50everything was just text.
3:52And we have a super intelligence, but we
3:54don't have a mature interface language.
3:57And today we are still trying to figure
3:59out what is this new interface for for
4:03this computer.
4:05And people ask, is it chat?
4:07I'll show you what we're what we're
4:08doing today. And actually this was a
4:09very recent tweet last week, where
4:12people were complaining that most SaaS
4:14companies have been adding
4:16chat to their their homepages, and
4:19everybody's just putting chat
4:20everywhere.
4:21And and that's that's fine. I don't have
4:23a problem with chat. It's not the final
4:26UI. It's okay for now.
4:29But the question is, if it's not chat,
4:31then then what is the interface
4:33for this computer?
4:35On the other hand, as we have seen with
4:39MCP apps,
4:40is there [clears throat] is another
4:42thought, which is we will have one app
4:44or super app to rule them all. And this
The role of MCP apps and 'Super Apps'
4:47is where MCP apps comes in, where
4:50instead of putting all of these chat
4:51windows into your homepages and and to
4:54every single app that you use, we will
4:56have a super app like ChatGPT or Claude
4:58or Gemini, where you will be interfacing
5:00with most of the the UI and the websites
5:03that we have today.
5:05And this is good. This is the way we
5:07using MCP apps today to to render
5:09third-party UI inside one agent
5:11environment.
5:13Now, these two options both could be
5:16valid. And and I believe these are part
5:19of the evolution towards finding out
5:22what is that new interface for that
5:25computer. And to be honest, I don't know
5:26which one is going to be the the final
5:29one. Consumers will tell us.
5:31But one thing is that these are two
5:33different questions.
5:35The question is where does the
5:37UI runs? In this case, is it third-party
5:40UI, a super app, or in this case, chat
5:42everywhere?
5:43But most interesting is what is the
5:45model generating. And this is what I
5:47want to talk about today in terms of how
5:49are we generating this UI? And we have
5:51seen this. We have mostly static,
Three levels of UI generation: Static, Declarative, and Generative
5:53declarative, and generative UI.
5:57And I'm going to describe briefly this
6:00one. So, we have um
6:02to start with the the static components
Understanding Static UI components (e.g., AG UI, Goose)
6:04way of running running UI, which is what
6:06most agents do today.
6:09The agent is just an orchestrator. The
6:12agent makes a tool call via MCP apps or
6:15direct agent tool call. Then we will
6:18have some parameters
6:20and data passed to predefined static
6:23components that
6:25have been created by developers. And
6:27this is very similar to what what we
6:28have been doing
6:30for the past 20 years with with UI. And
6:33then the client renders the component.
6:34And if you see here, it's very similar
6:36to just getting a server to send some
6:38data, and then the UI will be rendered
6:40by the client. But in this case, the
6:42agent will be generating that data and
6:44the props to to do this. And some
6:47examples that we have today are the AGUI
6:50protocol. They have an SDK when you can
6:52register a client tool that maps to a
6:56React component. The tool call will
6:59receive some props. Those props will be
7:01mapped to a static component that then
7:04will be rendered to to the user.
7:06Another example is Goose.
7:09Goose is a
7:10an MCP client where you can try most of
7:12the MCP features. And Goose has this
7:14really interesting feature called Goose
7:16Auto Visualizer, where you can just pass
7:18any type of data to Goose, and Goose
7:22will try to match that data, organize
7:24it, and then pass it to a set of
7:26predefined components that the Goose
7:28team have created. In this case, we have
7:30a few interesting components that you
7:31can
7:32use to visualize your data.
7:35So, that's the static way. That's the
7:38most common way of generating UI today.
7:41But I have seen an evolution recently
The benefits of Declarative UI (e.g., JSON/YAML renderers)
7:43where we call now declarative UI.
7:46And declarative UI takes it to the next
7:50level. So, we will still have some
7:52predefined static components that
7:54developers build, and it contains your
7:56design system and all these
7:58components that you have.
8:00But instead of the agent just passing
8:02the props and the data, the agent uses
8:05a descriptor that could be either JSON
8:07or YAML. Or I've seen Python as well
8:10with with fast MCPs, where they have a a
8:12descriptor in Python that maps to these
8:15predefined static components.
8:17And then you have this translation
8:20rendering engine that takes those
8:22descriptors and converts them into the
8:24final UI.
8:26How is this different? Well, in this
8:27case,
8:29it is more dynamic. They are still
8:32static components, but it's more
8:34personalized.
8:35And you if you look at this and you
8:37think that this might look familiar as
8:38well, it's because it is not new.
8:41Netflix has been doing this for a long
8:43time since the the personalization and
8:46server during UI era, where when you go
8:48to the Netflix homepage, you will get a
8:51UI that is completely personalized to
8:53you. But that's still mapped to the
8:56Netflix components and UI elements.
8:59Another very
9:01good um
9:02tool that I've seen recently is JSON
9:04Render. JSON Render is being built by
9:07Vercel. And it is a way to map your
9:11components using JSON and also YAML.
9:13They released the YAML support recently.
9:15And create all of these very dynamic,
9:18very good UI interactions that you can
9:21use today.
9:22But now JSON Render still, they say, is
9:27constrained to your static components.
9:32And yes, they're still static
9:34components. The LLM is not generating
9:37these components. The LLM is generating
9:39the JSON.
9:40However, I think in at this point in
9:42time, declarative generative UI is
9:45probably the perfect balance today in
9:47terms of flexibility and consistency.
9:51Because you would still want your design
9:53system. You still want to have uh,
9:55predictability of what the UI is going
9:57to be generated, also faster and also
9:59potentially cheaper at this point. Uh,
10:01so you don't create and use a lot of
10:03tokens to create the UI.
Moving to the next level: Generative UI components
10:06But as I mentioned at the beginning, why
10:07why are we still stuck here?
10:09And what's the next level? I think the
10:11next level will be uh, generative
10:13components.
10:14And generative components
10:16uh,
10:17goes into like the the premise I I put
10:19at the beginning where the models are
10:20good at writing front-end code. They are
10:22good at writing React. They're good at
10:23React creating in JavaScript, CSS.
10:27And the question is why we don't let
10:29them just write that
10:32on demand at runtime.
10:34What could possibly go wrong with that,
10:36right?
10:37This model um, of generating the UI uses
10:41the agent capabilities. And in this
10:43case, you can also use a tool call, but
10:45instead of calling this uh, layout
10:48rendering engine, you can call the same
10:50model with reverse sampling or you can
10:51call another model that will generate
10:54the HTML, CSS, JavaScript
10:57on demand and then it will be passed to
10:59the client.
11:00I did this experiment um, I work at
11:02Postman on this experiment where I
11:03created uh, this weather agent that goes
11:06to the API, the weather API. It creates
11:08a joke. It creates the HTML, CSS,
11:11JavaScript, all in one tool call.
11:13And you get presented with this random
11:16but very uh, imaginative UI where
11:20everything is created by the agent.
11:22There is no component. There is no
11:23translation.
The challenge of trust: The need for sandboxing and containment
11:25So there is of course a problem with
11:27this approach. Uh, and the problem with
11:28this approach is uh, if we
11:31don't trust third-party code,
11:34well, we should not trust um,
11:36code that has been generated by LLMs and
11:38then just present it to the user.
11:41Uh, generative UI and and this level of
11:44generative UI needs a distribution
11:46model.
11:47And this distribution model requires a
11:49boundary, requires containment and
11:51requires a sandbox, which is what we
11:53were talking about earlier.
11:55This is what I think MCP apps matter a
11:58lot because MCP apps are the best uh,
12:01delivery mechanism uh, for
12:04generative UI.
12:06We have the features provided by MCP,
12:09including authentication and tool
12:10calling and message passing between the
12:12UI and the agent. Uh, it's sandboxed by
12:14default with that double iFrame. Is the
12:17default for third-party UI delivery
12:19today. Does this become the standard?
Why MCP apps are the ideal delivery mechanism for Generative UI
12:22And one interesting thing is it's not
12:24just for
12:25for third-party UI. It can also be used
12:28for first-party UI.
12:31And this is why I think what Anthropic
12:33is doing with the the visualizer feature
12:36is very interesting strategically
12:38speaking because
12:40they could have just created their own
12:43rendering
12:44um, and an architecture mechanism for
12:46delivering this in interaction in in
12:49cloud,
12:50but they decided to go with MCP apps
12:52because MCP apps provide most of those
12:55um, features that I mentioned earlier
12:57uh, out of the box.
12:59So if Anthropic decided to use MCP apps
13:01for their first-party UI, uh, you can
13:03ask yourselves why cannot we do the
13:05same? It is
13:07um, a very very strong protocol. And
13:09especially when the UI is being
13:10generated on the fly by the agents, by
13:13the the code uh, coding models,
13:15then is the best um,
13:17mechanism for delivery.
The 'TV/Radio' analogy: Imagining the future of agent interaction
13:21Now,
13:22today is probably not the final form.
13:26And people keep saying
13:28is chat the final form? Is MCP apps the
13:31final form?
13:33We're still we're still trying to figure
13:34this out.
13:36And the obvious future
13:39is probably too obvious.
13:41And we said about where is my Jarvis?
13:44Where is my
13:45floating windows?
13:47And if you think about it, that's the
13:50obvious things thing that people would
13:51think how we would look like if we were
13:53generating or creating a new uh, user
13:56interaction.
13:57But what I think is
14:00we don't have enough imagination yet.
14:02And and this analogy I heard recently is
14:05very interesting when when uh, radio
14:08came out in the 30s,
14:10um, the the the first um, sorry, the the
14:13TV came out, the first
14:15uh, TV shows were radio shows
14:18with cameras
14:20because they could not imagine what you
14:22could do with this new technology. So
14:24this new technology that we have today
14:26is very similar when television came out
14:28and we are still in the radio era where
14:32we don't know all the amazing things
14:34that we will do in the future with this
14:36new media, with this new power that we
14:38have with the with the new computer.
14:40And we can see that we cannot even
14:43imagine what it's going to look like.
Beyond components: Towards true human-agent collaboration
14:46Uh, of course this is a speculative, but
14:47what do I think is actually going to
14:49happen
14:50is we are going to be moving uh, beyond
14:53components and more towards a
14:55collaboration uh, through human agent
14:58collaboration.
14:59If you haven't heard about the
15:00Excalidraw MCP app, uh, definitely check
15:03it out because
15:04the Excalidraw MCP app
15:06is not just for um, output and
15:09visualization of diagrams.
15:11The Excalidraw MCP app does something
15:13very interesting, which it creates a a
15:16shared artifact.
15:18It creates a canvas where a human and an
15:22agent can collaborate together into a
15:25shared space where you can go back and
15:27forth with the agent and ask, you know,
15:29change this, but you can also click
15:31around, modify the UI the way that you
15:33are used to. And that becomes the new
15:36way of interacting, the new way of
15:40experiencing the the agent um,
15:42powers.
15:44And and at the moment again, we are very
15:46constrained to our imagination. And and
15:48I believe these agents are very very
15:50powerful for just to just use them as a
15:54orchestrator and a delivery mechanism to
15:56show me some visualizations.
15:58So I believe beyond components,
16:01it will be the future of generative UI
16:04will be more a collaborative experience
16:06where yes, we will have some generative
16:09UI, but that UI is going to be super
16:12personalized and it's going to be
16:14collaborative.
Conclusion: Shaping the future of user interfaces
16:16So we we are still early.
16:19Um,
16:20we don't have the answer. People say,
16:22you know, what is what is the future of
16:25the user interaction, the user
16:26interfaces? We don't know yet.
16:28But we can shape that future uh, and
16:31create this uh, new computer.
16:34And that's me. Thank you so much uh, for
16:36listening to this.
16:37>> [applause]
16:37>> You can find me
16:39and ask any questions.
16:42Thank you.