Inside the World's Smartest Robot Brain [VLA]

0:00This may be the most significant moment

0:02in modern robotics.

0:04In 2023, a researcher at Google set up a

0:07table with a Coke can, pictures of Tom

0:09Cruise, Snoop Dogg, and Taylor Swift,

0:11and asked Google's newest robot brain,

0:13RT2, to move the Coke can to Taylor

0:16Swift.

0:17RT2 was too large to run on the robot

0:19itself.

0:21The robot sent one image at a time from

0:22its onboard camera to a TPU cluster,

0:25which sent back control signals.

0:28The robot controlled by RT2 slowly

0:31picked up the Coke can and awkwardly

0:33placed it on the edge of the picture of

0:34Taylor Swift.

0:36A couple years later, in 2025, one of

0:39the researchers on the team, Carol

0:40Hausman,

0:41would describe this scene as the moment

0:43it became clear to him that this was

0:45going to work.

0:47Within a year of the 2023 Coke can demo,

0:50Hausman and many of the key members of

0:52the RT2 team had left Google,

0:54and reassembled to form a startup called

0:56Physical Intelligence.

0:58And their robots have gotten better, a

1:00lot better.

1:02The latest robot brains from Physical

1:03Intelligence can open padlocks, fold

1:06your laundry, peel an orange, make a

1:08grilled cheese sandwich, make coffee,

1:11and clean up bedrooms and kitchens that

1:13it's never seen before.

1:15Why was this unimpressive Coke can demo

1:17such a breakthrough?

1:19And how did it enable Physical

1:20Intelligence to improve their robots so

1:22rapidly?

1:24In this video, we'll first explore the

1:26fascinating build-up to RT2 at Google.

1:29From here, we'll take a deep dive into

1:31the Physical Intelligence Robotics

1:33Foundation Models,

1:35and see what makes these incredibly

1:36impressive robot brains tick.

1:41In 2022, the year ChatGPT was released,

1:44researchers at Google began exploring

1:46what role large language models might

1:48play in robotics.

1:50Their first notable result, a system

1:52known as SayCan, used a large language

1:54model as a planning system to break down

1:57complex tasks into subtasks.

2:00In this demo, SayCan breaks down

2:01cleaning up a spill into the subtasks of

2:04finding a sponge, picking up the sponge,

2:06going to the spill, and so on.

2:09From here, the team expanded on this

2:11work, creating a more capable iteration

2:13of the idea called inner monologue, and

2:16another interesting variant where the

2:18team used an LLM to write code to

2:20control the robot on the fly.

2:23However, these early efforts were

2:24effectively bottlenecked by the

2:26available robot controls algorithms.

2:30Once the LLM in SayCan decided to pick

2:32up a sponge, a completely separate

2:34neural network that had been trained to

2:36imitate humans controlling robots to

2:38perform various small tasks, was used to

2:40compute the actual robot control

2:42signals.

2:44This meant that SayCan was effectively

2:45limited to a menu of actions the LLM

2:48could choose from.

2:50To get SayCan to place a Coke can on an

2:51image of Taylor Swift, behaviors

2:54involving Coke cans and Taylor Swift

2:55would have to be explicitly trained.

2:59At the end of 2022, the team made a

3:01significant improvement to their control

3:03layer, introducing Robot Transformer 1,

3:06or RT1.

3:08Like the team's previous control

3:10algorithms, RT1 was trained to imitate

3:12humans, but used a significantly larger

3:15dataset, with over 130,000 human

3:18demonstrations, and used a larger

3:20transformer-based architecture.

3:23RT1 was able to perform a significantly

3:25broader range of actions than its

3:27predecessors.

3:28This effectively gave the planning layer

3:30a much larger menu of actions to choose

3:33from.

3:34The RT1 team showed that using the

3:36planning LLM from SayCan, coupled with

3:38RT1 to control the robot, significantly

3:41improved performance on long-horizon

3:43tasks, like finding certain items in

3:46kitchens that the robot hadn't seen

3:47before.

3:50As Google incrementally improved their

3:51robot brains, large language models were

3:54also rapidly advancing.

3:57The LLM used for planning in the SayCan

3:59and RT1 systems was the text-only Palm

4:02540B model, trained in early 2022.

4:06This meant that the robot's planning

4:08layer couldn't actually see the world.

4:11After breaking down a task like helping

4:13clean up the kitchen into text subtasks,

4:15Google's robots relied on the RT1

4:17control layer to take in images from the

4:19robot's camera, and iteratively send

4:22control signals to the robot's

4:23actuators, until each subtask was

4:26complete.

4:27This approach worked fine for some

4:29tasks, but having a planning layer that

4:31was effectively blind was clearly not

4:33ideal.

4:35On March 6th, 2023, about a week before

4:38the release of GPT-4,

4:40Google researchers demonstrated Palm-E,

4:43a variant of the Palm large language

4:45model that directly incorporated images

4:47and other data sources.

4:49Using the multimodal Palm-E instead of

4:52the purely text-based Palm LLM as a

4:54planner, with RT1 as the control layer,

4:58the team demonstrated a significant

4:59expansion in capabilities.

5:02Now that the planning layer had access

5:03to vision information, the robot could

5:06perform more complex tasks that require

5:08adaptive planning,

5:10like moving objects out of the way to

5:12reach a desired object, and fully

5:15autonomously recovering from setbacks.

5:17Here, a robot using Palm-E as its

5:19planning layer and RT1 as its control

5:21layer, is asked to retrieve a bag of

5:23chips.

5:25And when a researcher repeatedly puts

5:26the chips back in the drawer, Palm-E is

5:29remarkably able to recognize that

5:30something has changed and adapt its

5:32plan.

5:34Now, let's zoom out a little and

5:36consider the full Palm-E plus RT1 robot

5:39brain.

5:40Although Palm-E and RT1 were designed to

5:42work at different levels of the stack,

5:44they have some really interesting

5:46similarities.

5:48Both models take in images from the

5:49robot's camera, and use a vision encoder

5:52neural network to process the images.

5:55From here, in both models, these encoded

5:58image representations are passed into a

6:00transformer.

6:01This is the same type of compute block

6:03used fairly universally in large

6:05language models.

6:07The big difference here is what these

6:09transformers are trained to do.

6:11The RT1 transformer was trained to

6:13directly output robot control signals by

6:16imitating humans controlling robots to

6:18solve various tasks.

6:20While the Palm-E transformer is trained

6:22to output text across a wide variety of

6:25tasks,

6:26including simple next token prediction

6:28on internet text, as we see in standard

6:30LLM pretraining,

6:32but also language-vision tasks, like

6:35image captioning. And importantly,

6:37Palm-E was also trained to break apart

6:39robotics tasks into smaller subtasks.

6:43The similarities between Palm-E and RT1,

6:46and the fact that the team was able to

6:47expand the Palm language-only model to

6:50effectively make use of other types of

6:52data,

6:53all beg the question,

6:55do we really need two separate models

6:57here?

6:58Why not just continue expanding the Palm

7:01language model to not only take in image

7:03data, but also to directly output robot

7:06control data,

7:08effectively absorbing RT1 into a single

7:10powerful end-to-end model?

7:14Said differently, can large language

7:16models, by far the most powerful AI

7:18systems we trained so far,

7:20be trained to become robot brains?

7:25This brings us to Taylor Swift and the

7:27Coke can.

7:28In July 2023, a few months after the

7:31Palm-E paper came out,

7:33the Google Robotics team demonstrated

7:34RT2.

7:36Taking Palm-E and another multimodal LLM

7:39known as PaLI-X as starting points,

7:42the Google team trained these LLMs to

7:44directly output robot control signals,

7:47training on the same human control

7:49demonstration data they had used to

7:51train RT1 6 months earlier.

7:54And incredibly, it worked.

7:57RT2 was able to generalize shockingly

7:59well to objects, environments, and tasks

8:01that were not in the human demonstration

8:03data.

8:05This is what makes the Taylor Swift demo

8:07so impressive.

8:09The robot control training data

8:11definitely did not include Taylor Swift.

8:14So, for RT2 to solve this task, it had

8:17to learn how to bring together abstract

8:19concepts it had learned in its

8:20internet-scale pretraining with the

8:23robot control episodes.

8:25This means that these models can learn

8:27to connect the vast amounts of image,

8:29video, and text data on the internet

8:31with real-world actions,

8:34potentially harnessing the full

8:35knowledge of the internet into robot

8:37brains.

8:39This is why this demo is such a big

8:41deal.

8:42It answers the question, can large

8:44language models be trained to be robot

8:46brains? With a shaky, but definitive,

8:49yes.

8:51The RT2 team coined a new name for this

8:54type of model, vision-language-action,

8:56or VLA,

8:58linking together vision, language, and

9:00action into a single unified model.

9:03This video is about to get technical.

9:06To see how language models can learn to

9:08become robot brains, we're going to

9:10reference transformers, embedding

9:12vectors, diffusion models, attention

9:14heads, softmax, and more.

9:16The required context for all these

9:18concepts is way more than we can fit

9:20into a single video, which is why I

9:23wrote this book.

9:24The Welch Labs Illustrated Guide to AI

9:27breaks down all of these concepts using

9:29hundreds of figures, detailed

9:30descriptions, and exercises.

9:33You can pick up a copy at welchlabs.com.

9:35And we're very excited to announce that

9:37we're beginning to offer international

9:39shipping.

9:40Stay tuned to the end of the video for

9:42more updates on the book, and to see the

9:44poster that goes along with this video

9:46that nicely breaks down the

9:48vision-language-action model

9:49architecture.

9:52By early 2024, a number of key members

9:55of the RT2 team had left Google and

9:58reassembled to form the startup physical

10:00intelligence.

10:01In October of that year, the team demoed

10:04their first robot brain, Pi Zero.

10:07Compared to the RT2 Coke can Taylor

10:09Swift demo 15 months before at Google,

10:12Pi Zero is remarkable.

10:15It starts to really feel like a robot

10:17that could help you around the house,

10:19performing tasks like getting laundry

10:20out of the dryer, folding the laundry,

10:22and cleaning up tables.

10:24How was the physical intelligence team

10:26able to improve on RT2 so significantly

10:29and so quickly?

10:31Like RT2, Pi Zero is a vision language

10:34action model built on top of a

10:36pre-trained multimodal LLM.

10:40Based on Pi Zero's strong performance,

10:43you might guess that the physical

10:44intelligence team increased the model

10:46size relative to RT2, but Pi Zero is

10:49actually smaller.

10:51The RT2 model family ranged from 5 to 55

10:54billion parameters, and Pi Zero

10:57remarkably only uses 3.3 billion

10:59parameters,

11:01allowing the model to run on the robot

11:02itself using a consumer-grade Nvidia RTX

11:054090 GPU at a very respectable

11:0873-millisecond inference time.

11:11Here's what Pi Zero looks like hooked up

11:13to a two-arm robot platform called Aloha

11:16and tasked with uncapping a pen.

11:19Pi Zero takes images from an overhead

11:21camera and from one camera on the wrist

11:23of each robot arm and a text prompt.

11:27At each time step, Pi Zero returns 14

11:30numbers. One number for the position of

11:32each of the seven actuators on each arm.

11:35Here we're plotting these outputs as a

11:37time series.

11:38This movement in our pink curve here

11:41shows us where Pi is telling the left

11:42gripper to grab onto the pen cap.

11:48Pi Zero is built on top of PaliGemma,

11:51an open-weight multimodal LLM from

11:53Google.

11:55PaliGemma is built from two other

11:56open-weight models, the SigLip image

11:59encoder and the Gemma large language

12:01model,

12:02that are trained together to solve

12:04vision language tasks like image

12:05captioning.

12:07Now, following the RT2 approach, the

12:10underlying language model, in this case

12:12PaliGemma,

12:14would be trained to directly output

12:15control values.

12:17However, the physical intelligence team

12:19made a clever improvement here that

12:21makes Pi Zero significantly better at

12:23dexterous manipulation. Instead of

12:25having the underlying language model

12:27directly output control values,

12:29Pi Zero introduces a second neural

12:31network the team calls an action expert.

12:35Interestingly, the Pi Zero action expert

12:37uses the same architecture as Gemma.

12:40In fact, in the Pi Zero code base, the

12:42action expert is instantiated as a Gemma

12:44model.

12:46The only differences are that the action

12:48expert is randomly initialized instead

12:50of pre-trained,

12:52and the action expert is not as wide as

12:54Gemma,

12:55using fewer parameters within each

12:57layer.

12:58Now, this may sound like we're going

12:59back to the earlier SayCan system,

13:02where a high-level LLM performed

13:04planning

13:05and a lower-level network handled robot

13:07control.

13:09The key distinction here is that in the

13:10SayCan system, the interface between

13:13models was natural language.

13:16The planning LLM told the control

13:17network what to do using predetermined

13:19text instructions.

13:22Pi Zero, in contrast, uses a much richer

13:25interface between the two models.

13:27Since the Gemma LLM and action expert

13:29effectively share the same architecture,

13:32it's possible for these models to almost

13:34think as one

13:36while retaining some really nice

13:37benefits of modularity.

13:39Let's have a closer look at how our

13:41Gemma LLM learns to act as a robot

13:43brain.

13:44Then we'll have a closer look at how the

13:46interface between these two models

13:47works.

13:49The Gemma LLM processes both the images

13:51and text prompts that come into Pi Zero.

13:54Each image is broken into a grid of

13:56patches, resulting in 256 image patches

13:59per image and 768 total patches.

14:03The patches from each image are passed

14:05into an image encoder model,

14:07resulting in 768 embedding vectors, each

14:11of length 2048.

14:14These vectors are sometimes referred to

14:15as soft tokens.

14:18Here we're coloring each embedding

14:19vector to approximately match its

14:21corresponding image patch.

14:23This will help us keep track of our data

14:25as it flows through our model.

14:27These embedding vectors live in a

14:29semantically rich embedding space,

14:32meaning they should contain lots of

14:33easily accessible information about our

14:35images,

14:37like whether a given image patch

14:38contains a pen.

14:41For more on embedding spaces and image

14:42encoders, check out the Welch Labs video

14:45on AlexNet, the AI image generation

14:48video we did with Three Blue One Brown,

14:50or the Welch Labs illustrated guide to

14:51AI.

14:53The text prompt we give Pi Zero, in this

14:56case uncap the pen,

14:58is broken into four tokens, and each

15:00token is mapped to an embedding vector

15:02of the same length as our image patch

15:04embedding vectors.

15:07So, we now have 772 total embedding

15:10vectors,

15:12768 from our images and four from our

15:15text prompt.

15:16From here, these embedding vectors are

15:18passed into our Gemma LLM.

15:21Gemma is composed of 18 transformer

15:23blocks, each containing an attention and

15:25multi-layer perceptron compute block.

15:29Each attention block contains eight

15:31attention heads.

15:33These attention heads are arguably the

15:34most critical part of the transformer

15:36architecture

15:37and are the key to the tight integration

15:40between Pi Zero's underlying LLM Gemma

15:42and Pi Zero's action expert.

15:45In a given attention head, the incoming

15:47embedding vectors are multiplied by

15:50three separate matrices of learnable

15:51weights,

15:53producing three new matrices known as

15:55queries, keys, and values.

15:59Each of these matrices has 772 rows, one

16:03for each input to our transformer.

16:06We don't have enough space to visualize

16:08all 772 rows of our matrix.

16:11Here we're showing the first row, which

16:13corresponds to the upper left patch of

16:15our overhead image.

16:17Next, we're showing rows 373 to 376,

16:22which correspond to these four patches

16:24of our left wrist image.

16:27This will be important shortly as we see

16:29how Gemma figures out how to connect the

16:30word for pen to the parts of the images

16:33that contain the pen.

16:35As we did with our embedding vectors,

16:37we'll color each row of our matrix with

16:39the approximately average color from its

16:41corresponding image patch.

16:44Our two patches that contain the orange

16:45pen get colored orange.

16:48And finally, the light and dark parts of

16:50each vector correspond to the actual

16:51numerical values of the vector.

16:54Dark regions are lower numbers and light

16:56regions are higher numbers.

16:58Finishing out our matrix, these last

17:00four rows come from our input text with

17:03one row for each token,

17:05and we'll color all our text rows blue.

17:08From here, Gemma's attention head

17:10searches for similar query and key

17:12matrix rows.

17:14This attention head may have learned,

17:15for example, to specialize in searching

17:18the incoming images for objects that

17:20match words that appear in the prompt.

17:23After all, if our robot brain is going

17:24to uncap the pen, it needs to know where

17:27the pen is in our images.

17:30The word pen shows up at our very last

17:32token input position,

17:34and its query vector looks like this.

17:37The attention head computes the dot

17:39product between this row and every row

17:41in our key matrix,

17:43and larger dot products indicate closer

17:45matches between queries and keys.

17:49Interestingly, our highest dot products

17:51in this sample by far occurred at two

17:54image patches that contain the pen.

17:57From here, our attention head normalizes

17:59these dot product values using a softmax

18:01operation.

18:02We can take our visualization one step

18:04further here and show these attention

18:06values as a heat map on top of our

18:09images,

18:10where brighter shades of magenta

18:12correspond to larger attention values.

18:15So, the two orange rows of our key

18:17matrix with high attention values that

18:20correspond to these two image patches

18:22get colored bright magenta,

18:24and their neighboring patches with low

18:25attention scores do not.

18:28So, the idea here is that our heat map

18:29visualization shows us the strongest

18:32matches in our images to our query

18:34vector for the word pen in our prompt.

18:38And remarkably, our best matches occur

18:40at the patches in all three images that

18:42show the pen.

18:44Playing our video and running this

18:46analysis at each frame, we see

18:48impressive pen tracking results.

18:52Our model is clearly using this

18:53attention head to connect the word pen

18:55in our prompt to the parts of our images

18:57that contain the pen.

19:01Now, our attention head doesn't just

19:03search for matches to our pen query.

19:06All 772 query vectors, corresponding to

19:10all input images and prompt tokens, are

19:12compared to all 772 key vectors.

19:16The resulting attention values from all

19:18these comparisons are collected in a 772

19:21by 772 attention pattern matrix.

19:25Each row of the attention pattern

19:27corresponds to a single query.

19:30The final query row corresponds to the

19:32pen token in the prompt that we've been

19:33visualizing.

19:35So, our heat map values end up in the

19:37bottom row of our attention pattern.

19:40At the beginning of our attention head,

19:42we computed three matrices, our queries,

19:45keys, and values.

19:47We've used our queries and keys to

19:49create our attention pattern,

19:51and now our attention pattern is

19:52multiplied by our value matrix,

19:55creating this attention heads output, a

19:57new 772 by 256 matrix.

20:01Multiplying our value matrix by our

20:03attention pattern effectively moves

20:04information between token positions.

20:08The large attention values we see

20:10between our pen query and pen image

20:12patches

20:13mean that these image patch rows are

20:15copied and added to the pen position in

20:17our final output.

20:19One way to think about this operation is

20:21that our attention head is forming a

20:23unified representation of the text for

20:26pen and the parts of our images that

20:28contain pens.

20:30Now, this is just a single head in a

20:32single layer of our 18-layer Gemma LLM.

20:36And we expect different heads to learn

20:37to pick up on different types of

20:38patterns. And remember that our Gemma

20:41LLM is just one part of the Pi Zero

20:43system.

20:45Let's now turn to Pi Zero's action

20:47expert model and see how the physical

20:49intelligence team was able to get these

20:51models to work together so seamlessly.

20:54While the PaliGemma portion of Pi Zero

20:56takes in our 772 image and text prompt

20:59tokens,

21:00the action expert takes in information

21:02about our robot state.

21:04That is the position of all of its

21:06joints.

21:08On the Aloha platform we've been

21:09experimenting with, each arm has a

21:12movable waist, shoulder, elbow, forearm

21:15rotation, wrist, wrist rotation, and

21:17gripper.

21:19This makes for seven joints per arm or

21:2214 total numerical values that we need

21:24to control our robot.

21:27Just as our text prompt and input images

21:29are mapped to embedding vectors, our

21:31vector of 14 joint positions is also

21:33mapped to an embedding vector.

21:36This mapping is done by multiplying our

21:37joint vector by a 14 by 1024 matrix of

21:41learned weights.

21:43Note that while the Gemma LLM in Pi Zero

21:45uses an embedding vector of length 2048,

21:48the action expert uses embedding vectors

21:50of length 1024.

21:52This reduces the compute requirements

21:54and inference time of the action expert

21:56model.

21:57So the robot's current state fits into a

21:59single embedding vector or soft token.

22:02This is one of the inputs to our action

22:04expert.

22:06The action expert has one more set of

22:07inputs,

22:09the joint positions of the robot over

22:10the next 50 time steps,

22:13generally referred to as actions.

22:16Now, this might seem backwards. The

22:18whole point of the action expert is to

22:20predict the future robot actions.

22:23How could the model take predicted

22:25actions as an input?

22:27In a fascinating transfer of ideas from

22:30AI video and image generation,

22:32Pi Zero's action expert uses a method

22:34called flow matching.

22:36The idea is that instead of outputting

22:38robot actions in one go, the model

22:41iteratively shapes completely random

22:43actions into a final trajectory.

22:46The comparison to AI image generation is

22:48really interesting here.

22:50A final set of actions produced by our

22:52action expert will be of dimension 14 by

22:5450,

22:56with one row to control each robot joint

22:59and one column for each of the next 50

23:00time steps.

23:02We can visualize this matrix as an

23:04image, as we have with other matrices in

23:06our model.

23:08In this set of actions, we see an

23:09increase in the values in our ninth row.

23:13We can plot these values as a time

23:14series.

23:16This set of actions is telling our robot

23:18to move its right shoulder,

23:20reaching its right gripper towards the

23:21pen.

23:23In AI image generation, we can create an

23:25image of a cat by iteratively refining a

23:27pure noise image into a detailed cat

23:30image.

23:32Pi Zero's action expert does the same

23:34thing,

23:35refining a 14 by 50 random image of

23:37joint trajectories into a detailed plan

23:40for how to move each robot joint.

23:44One reason this flow matching or

23:45diffusion process works so well for

23:47generating natural images

23:49is that the distribution of natural

23:51images is multimodal.

23:53There are many ways to create an image

23:55of a cat.

23:56Analogously, there are many ways we can

23:58move our 14 robot joints to uncap a pen.

24:03So to generate a set of actions, the

24:05action expert starts with completely

24:07random actions

24:08and predicts how these actions should be

24:10updated to produce a slightly more

24:12realistic and accurate set of

24:13trajectories.

24:15These trajectories are added to the

24:17input actions and then passed back into

24:19the model, which then computes a new set

24:22of updates.

24:23This process is repeated 10 times in Pi

24:25Zero until we have a nice set of

24:28trajectories.

24:30The fact that we can use the same exact

24:31flow matching process to generate images

24:34and videos and control robots is so

24:37interesting to me.

24:38It's such a surprisingly effective

24:40abstraction on top of what feel like

24:42very different applications of AI.

24:46So our action expert model can

24:47iteratively shape pure noise into robot

24:50trajectories.

24:52But how does it know what trajectory is

24:53to generate?

24:54The action expert needs to know what the

24:56goal is, in the case of our example,

24:58uncapping the pen.

25:00And of course, it needs lots of

25:01information about the scene, like where

25:03the pen is in space.

25:06As we saw earlier, this is exactly the

25:08type of information our Gemma LLM is

25:10already processing in its attention

25:12heads.

25:14The question from here is, how do we

25:16best give our action expert access to

25:18this information?

25:20As we saw earlier, the action expert

25:22uses the same architecture as our Gemma

25:24LLM.

25:26This means that like Gemma, our action

25:28expert has 18 attention blocks with

25:30eight attention heads each.

25:33As we saw earlier, each Gemma attention

25:35head computes a separate query, key, and

25:37value matrix.

25:40Our action expert attention heads

25:41perform the same operations, but with

25:44different inputs.

25:46Our action expert has 51 inputs, one for

25:49the robot's current state, and 50 for

25:52the robot's predicted actions over the

25:53next 50 time steps.

25:56So within each attention head, our

25:58action expert's query matrix will have

26:0051 rows,

26:02one for each model input.

26:04Now, using the standard attention

26:06mechanism, each query is able to search

26:08for matches in the keys.

26:10This could allow, for example, our

26:12second action step to use information

26:14from our first action step,

26:17which would help our model create a nice

26:18smooth trajectory from time step to time

26:21step.

26:22Of course, to figure out where these

26:23trajectories should go at all, our

26:25action expert's queries ideally need

26:28access to the prompt and image

26:29information from our Gemma LLM.

26:32This is where the team's decision to use

26:34the same architecture for the LLM and

26:36action expert really pays off.

26:39All we have to do at this stage is take

26:41the keys and values from the

26:42corresponding attention head of our

26:44Gemma LLM and append them to the keys

26:47and values from our action expert.

26:50So we now have 51 plus 772,

26:53making for a total of 823 keys that our

26:56action expert can query.

26:58These keys contain all the information

27:00the action expert needs, the text

27:03prompt, the encoded images, the robot

27:06state, and other time steps in the

27:08planning process.

27:10This gives the attention heads in our

27:12action expert an immediately available,

27:14incredibly rich information source.

27:17It's a really clever design.

27:20This modular design allows for some

27:22impressive efficiency gains.

27:25After the images and prompt are passed

27:26into PaliGemma, the computed keys and

27:29values in each attention head are

27:30cached.

27:32This is a common step in LLM inference,

27:35preventing redundant computation as new

27:36tokens come along.

27:39However, in this case, the physical

27:41intelligence team uses PaliGemma's KV

27:43cache to feed into each action expert's

27:46attention head.

27:49Since the action expert uses a flow

27:51matching process,

27:52it needs to run multiple times to

27:54produce final smooth trajectories, but

27:57is able to use the same KV cache each

27:59time because the input images don't

28:01change until the next time step.

28:04The fact that all these components can

28:06be trained to work together so well is

28:08absolutely incredible.

28:10At each step, Pi Zero takes in its

28:12prompt and images, runs them through

28:14PaliGemma, caches all the key and value

28:17matrices,

28:18and then runs the action expert to

28:20iteratively denoise random trajectories

28:22into final paths.

28:25The robot then follows these paths for a

28:27few steps, and the process is repeated,

28:30controlling the robot to achieve the

28:31task at hand.

28:34Since Pi Zero was first demoed in

28:35October of 2024,

28:37the physical intelligence team has made

28:39various improvements to their models and

28:40training approach,

28:43but their core VLA architecture, using a

28:45tightly coupled multimodal LLM with a

28:47flow matching action expert, has

28:49remained unchanged.

28:52Looking back on the RT-2 Taylor Swift

28:54Coke can demo in 2023,

28:56it's incredible to see how far VLA

28:59models have come.

29:00And what's perhaps even more impressive

29:02to me

29:03is that the physical intelligence team

29:05had the foresight

29:06to realize what this unimpressive demo

29:08really meant,

29:10that large language models could be

29:11trained to be robots,

29:14potentially leveraging the full

29:15knowledge of the internet into robot

29:17brains.

29:20Now, as impressive as these demos are,

29:22they're still demos.

29:24In 1995, a team from Carnegie Mellon

29:26demonstrated a self-driving system,

29:28Ralph, that drove across the US at 98.2%

29:32autonomously.

29:34This clearly did not mean that

29:35self-driving cars were around the

29:36corner.

29:37>> [music]

29:38>> And the generation of self-driving cars

29:39we have today works very differently.

29:42And interestingly, there's a different

29:44paradigm emerging for building robot

29:46brains, broadly known as world models,

29:50that actually do not use large language

29:52models as a backbone.

29:54Yann LeCun, AI pioneer and long-time

29:56chief AI scientist at Meta, recently

29:59left his role at Meta to start a new

30:01venture focused on world models.

30:04Yann was kind enough to chat with us

30:06about it and wasn't shy about giving his

30:08opinion on VLA models.

30:11What's your expectation here? Do you

30:12think JEPA-based approaches will

30:13eventually overtake VLA approaches? Oh,

30:16absolutely. Yeah, VLA are doomed. I

30:18mean, they they basically don't work

30:20really well. Okay. I mean,

30:23Next time, we'll dig into Yann's

30:24approach.

30:27If you enjoyed this video, check out the

30:29companion poster.

30:31The poster walks through the full Pi

30:32Zero architecture with helpful

30:34descriptions along the way.

30:36Fitting everything on screen was a huge

30:38challenge when animating this video.

30:40And the large format of the poster is

30:42perfect for getting everything into one

30:44place.

30:45The poster is printed on high-quality

30:47large format photo paper with genuine

30:49Canon inks for excellent colors and

30:51details.

30:53For a limited time, you can get a

30:54discount on the poster when bundled with

30:56the Welch Labs' Illustrated Guide to AI

30:58using code VLA.

31:01Speaking of the Welch Labs' Illustrated

31:02Guide to AI, I'm very excited to

31:04announce that international shipping is

31:06now available in these nine countries.

31:08And we're planning to expand to these

31:10countries next.

31:12I know this has taken a really long

31:14time. Thank you for your patience. A ton

31:16of you have emailed us and joined our

31:18international shipping waitlist.

31:20Today, all of our books are printed in

31:22the US. We have a great relationship

31:24with our printer and the quality is

31:26outstanding.

31:27We've received a bunch of nice feedback

31:29about this. This viewer told us that the

31:31construction quality is the best they've

31:33ever seen.

31:35However, our print costs are fairly high

31:37and being self-published makes

31:38international logistics a real

31:40challenge.

31:41Until very recently, my family and I

31:43packed all the books ourselves.

31:46Here's 7,000 lbs of books getting

31:48dropped off on the street in front of my

31:49house late last year. We use

31:51high-quality boxes and corner protectors

31:53to make sure your book arrives in

31:54pristine condition.

31:56The other packaging options we've tried

31:58it just don't protect this heavy book

32:00very well.

32:01Here's some outgoing shipments and a van

32:03we rent sometimes for post office runs.

32:06Here's part of another shipment that was

32:07delivered during a snowstorm this year.

32:10And here's some more books heading to

32:11the post office in my family's SUV.

32:14It's definitely been an adventure.

32:16Early this year, we started looking at

32:17ways to improve and scale our process.

32:20We've had some interesting calls with

32:22publishers, but the deals we've seen so

32:24far either significantly reduce print

32:26quality or cut too deeply into our

32:28margins, effectively introducing one or

32:30two layers of middlemen to our supply

32:32chain.

32:34So, we've decided to stay self-published

32:35for now.

32:37We did find a great local packing

32:38logistics partner who now, thankfully,

32:41is handling fulfillment.

32:43They also ship enough volume to get some

32:44nice international rate discounts.

32:47This is what has allowed us to start to

32:48tackle international shipping.

32:51Starting today, we're offering flat rate

32:52shipping to Canada, Mexico, the UK,

32:55Ireland, Germany, France, the

32:57Netherlands, Italy, and Belgium. And the

32:59flat rate includes all relevant VAT,

33:01GST, and duties.

33:04We chose these countries by

33:05cross-referencing the countries with the

33:06highest demand on our waitlist and where

33:08we're able to ship without exorbitant

33:10shipping costs.

33:12Next, we're looking at expanding to

33:14India, Australia, New Zealand,

33:16Singapore, Japan, South Korea, Hong

33:18Kong, Thailand, and Malaysia.

33:21Although higher shipping costs due to a

33:22global increase in fuel prices is making

33:25this a bit more challenging than we

33:26expected.

33:28I'm really happy that we now have some

33:29international options, but the price we

33:32need to charge to cover printing and

33:33shipping is still higher than I would

33:35like.

33:36If we continue to see strong demand,

33:38this will allow us to invest in larger

33:40print runs, bringing down printing cost.

33:43And we're even looking at doing some of

33:45our printing regionally, starting in

33:46Europe.

33:47This would significantly bring down our

33:49European shipping costs and allow for

33:51lower prices.

33:53Sometimes, I really question if this is

33:54all just crazy and really a distraction

33:57from making videos,

33:58especially on days when 7,000 lbs of

34:01books show up at my house.

34:03However, at the end of the day, the

34:04mission of Welch Labs is to make these

34:06complex topics as understandable as

34:08possible, and books are a big part of

34:10that mission.

34:12One supporter on Patreon, Lauren Steely,

34:14put this really nicely when talking

34:16about the book.

34:17It's not just a condensed version of the

34:19videos. The book actually adds so much

34:22more detail that the videos couldn't

34:24possibly contain.

34:26Finally, a big thank you to all the

34:28readers who have helped find errors and

34:29made suggestions for improvements.

34:32These readers are listed on the credits

34:33page of the latest version of the book.

34:36And we published an errata at

34:37welchlabs.com/ai-book.

34:41I especially want to thank Robert

34:43Blumoff.

34:44He's been incredibly meticulous at

34:46rooting out little issues in the book

34:48and has even made his own perceptron

34:49machine and build guide. I'll include a

34:52link in the description below.

34:54Thank you so much for your patience and

34:55to everyone who's bought a book. It

34:57really helps the business work and means

34:59a lot to us. Thank you.