Full transcript
Intro
0:00In early 2015, the relatively new field
0:03of deep learning got stuck. The field's
0:05namesake and main driver of progress,
0:08making models deeper, stopped working.
0:11Once researchers reached models between
0:1320 and 30 layers deep, performance gains
0:16stalled out [music] and then actually
0:19reversed. Here's a 74 layer model being
0:22outperformed by an eight layer model.
0:25The solution to this problem turned out
0:27to be a shockingly simple idea.
0:29Originally published in this 12-page
0:31paper in December 2015 that has gone on
0:34to become the most cited paper of the
0:3621st century.
0:38The impact of this simple idea is
0:40difficult to overstate.
0:42The discovery forced the field to
0:44completely reconceptualize how these
0:47models work. coming to the realization
0:49that this simple idea actually formed a
0:51new critical backbone for neural
0:53networks, a kind of working memory that
0:57ultimately enabled these models to reach
0:58unprecedented levels of performance.
When Deep Learning Stopped Working
1:04Giansoon's research team at Microsoft
1:06Research Asia was stuck. A few years
1:09earlier in 2012, the AlexNet paper
1:11kicked off the deep learning revolution,
1:13demonstrating an eight- layer
1:15convolutional neural network that
1:16significantly outperformed all previous
1:18methods on the imageet image
1:20classification challenge. In 2014, teams
1:24at Oxford and Google successfully
1:26trained deeper models with 19 and 22
1:29layers respectively, significantly
1:31improving on the original Alexet
1:33results. And in early 2015, Sunn's
1:36research group had made a real
1:37breakthrough careful parameter
1:40initialization. They reached an
1:42unprecedented depth of 30 layers. Here's
1:45a training plot from their paper,
1:47showing that using a then standard
1:49Xavier initialization approach. Their 30
1:52layer model completely failed to learn.
1:54The model's error rate just stays fixed
1:56at 100%. But when the team switched to
1:59their newly proposed initialization
2:01approach, now known as HU
2:03initialization,
2:04the model was actually able to learn,
2:07bringing down its error rate. However,
2:10while the team's new deeper models were
2:11able to learn, the final performance of
2:14these models was surprisingly poor.
2:17This 30 layer model only reached a
2:1916.59%
2:21error rate while the team's 14 layer
2:24model using a similar architecture
2:26achieved a superior error rate of
2:2813.34%.
2:30This result is especially confounding
2:33because as the team would later note
2:35there's an essentially trivial solution
2:37that we can obtain by construction where
2:40the 30 layer model would achieve at
2:41least the same performance as the 14
2:43layer model. If we take the first 14
2:46layers of our smaller trained model and
2:49add 16 pass through layers where each
2:52layer simply passes its input to its
2:54output unchanged, the 30 layer model
2:57would produce exactly the same final
2:59outputs as our 14 layer model. The
3:02layers in the team's models were
3:03perfectly capable of performing this
3:05identity mapping. So if the 30 layer
3:08model was capable of achieving at least
3:09the same performance as the 14 layer
3:12model, then why couldn't the team's
3:14optimizers find these solutions?
Tracing Data Forward
3:17Let's trace the flow of information
3:19through one of these shallower networks
3:21and see if we can figure out exactly
3:23what breaks down as we add more layers.
3:26In mid 2015, as the team puzzled over
3:29these results, the state-of-the-art deep
3:31learning architecture was the
3:32convolutional neural network. In
3:35convolutional networks, images are
3:38processed by sliding a window of
3:39learnable weights across the image and
3:42at each position computing the
3:43dotproduct between the image pixel
3:45intensity values and the learned weight
3:47kernel. This dotproduct operation will
3:50return large values when our kernels and
3:53image patches are similar. Here's a
3:55learned vertical edge detector kernel
3:58returning large activation values when
4:00passing over vertical edges in the
4:02image. The first layer of our network
4:05uses 64 different learned kernels,
4:08resulting in 64 new activation maps,
4:11each responding to various features in
4:13our image, such as different edge
4:15orientations or colors.
4:18From here, these 64 activation maps are
4:20stacked into a 64x 112x 112 tensor of
4:24activations.
4:26These activations form a new sort of
4:28image. But where our input image has
4:30red, green, and blue color channels, our
4:33activation tensor has 64 channels, each
4:36corresponding to a different type of
4:38image feature.
4:40Our activations are scaled and passed
4:42into a RLU activation function, which
4:44sets all values less than zero to zero.
4:48Note that here we're not showing values
4:50below a certain threshold to make our
4:52activations easier to see. These three
4:55steps, the sliding kernel convolution,
4:58scaling, and the RLU activation function
5:00form a single layer of our model.
5:04Our activations are then passed into a
5:05second convolutional layer where we
5:08slide a new set of learned kernels over
5:10our activation tensor, producing a new
5:13activation tensor of dimension 64x 56 x
5:1656. Just as in our first layer, this
5:19tensor is again passed through a scaling
5:21step and a RLU activation function.
5:25These sliding kernel scaling and
5:26activation steps are repeated again and
5:28again with a down sampling step every
5:31few layers until we're left with an
5:33activation tensor of dimension 256x
5:3614x4.
5:38This tensor is downsampled one final
5:41time to a 256x1
5:43vector of activations.
5:46These 256 numbers are then multiplied by
5:48a final matrix of learned weights of
5:51dimension 256 by a th00and resulting in
5:54a thousand numbers where each number
5:57corresponds to the model's confidence in
5:59each of the a thousand classes in the
6:00imageet data set. If we plot these final
6:04a thousand numbers, we see that index
6:07951 has the largest value. This index
6:10corresponds to lemon in the imageet
6:12labeling scheme, meaning that our model
6:15has correctly classified this image.
6:19Our model currently has eight total
6:21layers and achieves an accuracy of 44.1%
6:24on the imageet validation data set. As
6:28we've seen, it correctly classifies this
6:29lemon, but it mclassifies this rocking
6:32chair, safe, and screwdrivers.
6:35Now, just as the Oxford and Google teams
6:37did in 2014,
6:39let's add more layers to our model and
6:41see if we can improve performance.
6:44Adding layers to our convolutional
6:45architecture is a surprisingly simple
6:47affair. We just need to choose where to
6:50place them within our model. Let's add
6:52six new layers. Two in this first block
6:55of layers and four after our final
6:57convolutional block.
7:00These new layers perform the same exact
7:02sliding dot product, scaling, and RLU
7:04operations as their neighbors, just with
7:07different learned weights.
7:09Now, let's visualize the accuracy of our
7:12new deeper 14 layer model as it learns
7:15and compare these results to our
7:16shallower eight layer model. After
7:1940,000 steps, our deeper model performs
7:22significantly better, reaching an
7:24accuracy of 56.7% on the imageet
7:27validation set.
7:29Now, as Giann's team did in early 2015,
7:33let's add even more layers and see if we
7:35can continue improving performance.
7:39At 20 layers, we get this training
7:41curve, only reaching an accuracy of
7:4353.1% after the same 40,000 training
7:46steps.
7:48Now, note that our accuracy curve hasn't
7:50really leveled off yet, and deeper
7:52models with more parameters may require
7:54more training steps.
7:57Training our 20 layer model for 20,000
7:59more steps, we're able to reach a higher
8:02image net accuracy of 62.5%.
8:06Pushing to 26 layers and training a bit
8:08longer, we see a modest improvement to
8:1063.6%.
8:13At 34 layers, though, our luck starts to
8:15run out. Even after 80,000 training
The Degradation Problem
8:19steps, we only reach an accuracy
8:21slightly below our 26 layer model at
8:2462.8%.
8:26If we add even more layers, the trend
8:29amplifies with this 56 layer model only
8:32reaching an accuracy of 56.6%.
8:36And this 74 layer model getting stuck at
8:39an accuracy of just 38.9%.
8:42This is worse performance than our
8:44dramatically smaller eight layer model.
8:47This is the same performance degradation
8:49problem that Soon's team found.
8:52Why does adding more layers no longer
8:54help? and actually start making things
8:57worse.
8:58Soon's team speculated in early 2015
9:01that this could just be a fundamental
9:03limitation of neural networks, writing,
9:06this is perhaps because the method of
9:08increasing depth is not appropriate.
Probing the Model
9:11Let's take a closer look at our
9:13shallower eight layer model and see if
9:15we can figure out exactly where things
9:17go wrong. This model has 1.45 million
9:20learnable parameters spread across its
9:23eight layers. Let's consider one of the
9:25images our model got wrong earlier. When
9:28we pass this image of these long
9:30screwdrivers into our shallow eight
9:32layer model, our model only returns a
9:34probability of 0.056
9:37for the correct label of screwdriver.
9:40The model's maximum output probability
9:41is 0.09
9:43at index 623
9:46corresponding to the class for letter
9:47opener.
9:49Note that here we're plotting our
9:50model's outputs after a final softmax
9:52operation. which scales our final
9:55activations into nice probabilities that
9:57all add up to one. Now, the whole point
10:00of training is to adjust our model's
10:03millions of parameters to increase the
10:05model's predicted probability of the
10:06correct answer. Let's focus for a moment
10:09on just one parameter in our model's
10:12final layer. Our parameter's current
10:14value is 0.215.
10:18Let's explore a range of values for this
10:20single parameter. Starting by setting
10:22our parameter to a large negative value
10:24of minus2.5.
10:27This change pushes down the model's
10:29probability of the correct answer to
10:31essentially zero. Increasing our
10:33parameter's value, our model slowly
10:35becomes more confident in the correct
10:37answer of screwdriver.
10:39And by the time our parameter reaches
10:41positive 2.5, our model's probability of
10:44the correct label is almost 100%. We can
10:47plot our model's confidence in the
10:49correct label as a function of the value
10:51of the single parameter and see that
10:53large values of our parameter cleanly
10:55lead to higher probabilities of the
10:57correct answer in this case. Now, we of
11:00course can't train our model just by
11:02tuning one parameter. And this parameter
11:04happens to be connected directly to the
11:06model's output for the screwdriver
11:08class. How does this picture change as
11:11we move to earlier layers in our model?
11:13Here are the curves for a few parameters
11:15in the middle of our model. Here, the
11:18relationship between our parameters and
11:19the model's outputs becomes more
11:21complex. We can see that as we sweep
11:24through these parameters, the model's
11:25output probabilities change in less
11:27predictable ways.
11:30Now, note that these curves are
11:31intimately connected to how our model
11:34actually learns. While it's
11:36computationally infeasible to compute
11:38curves like this for all 1.45 45 million
11:40parameters. Remarkably, we can
11:42efficiently compute the slopes of all
11:451.45 million curves using back
11:47propagation. This collection of slopes
11:49is known as the gradient and it guides
11:51the entire learning process.
11:55Let's look at the curve for one more
11:56parameter. This time in the very first
11:59layer of our model. Here, the mapping
12:01between our parameter and the model's
12:03probability of the correct label becomes
12:05even more complex. The output of this
12:07first layer is modified by the millions
12:10of parameters in our later layers before
12:12becoming the final output, resulting in
12:15increasingly complex mappings. And we're
12:18still only looking at our eight layer
12:20model. How might this complexity build
12:22as we move to deeper models?
12:26Let's make an improvement to our
12:27visualization process and then explore
12:29how things change as we move to these
12:31deeper models.
12:33Thus far, we focused on maximizing the
12:35model's probability of the correct
12:37answer. But in practice, we typically
12:39use a slightly different objective. For
12:41classification problems like this, we
12:43would typically use the cross entropy
12:45loss. To compute the cross entropy loss,
12:48we take the negative natural logarithm
12:51of the model's probability of the
12:52correct answer.
12:55So if our model's probability of the
12:56correct answer is one, our loss is zero.
13:00Plotting our cross entropy loss on top
13:02of our probability curves, we see that
13:05our loss is essentially a flipped and
13:07stretched version of the model's
13:08probability of the correct answer.
13:12The critical conceptual change here is
13:14that we're now trying to minimize our
13:16loss instead of trying to maximize our
13:18probability.
Loss Landscapes
13:21Now, when we train our model, we of
13:23course aren't updating single parameters
13:25at a time. Instead, all 1.45 four or
13:28five million parameters are updated
13:30simultaneously at each learning step.
13:32Visualizing this full process is
13:34perilous because our model is
13:37effectively stepping through 1.45
13:39million dimensional parameter space as
13:41it learns and we're limited to
13:43visualizing just a few dimensions at
13:45once.
13:47One approach we can take here is to
13:49choose a direction randomly in this
13:51parameter space. iteratively taking
13:54steps in this direction [music] and
13:56recomputing our loss at each step along
13:58the way.
14:00Functionally, this means sampling a
14:01vector with the same number of entries
14:03as the number of parameters we're
14:04exploring and adding scaled versions of
14:07this vector to our initial parameters.
14:10This gives us a new 1D curve for each
14:12randomly chosen direction.
14:15Things get more interesting when we pair
14:17two randomly chosen directions together,
14:20computing the loss for each combination
14:22of random directions in a grid and
14:24visualizing the results as a landscape.
14:27For more on this process, see chapter 2
14:29of the Welch Labs illustrated guide to
14:31AI. Here's what the lost landscape looks
14:34like for a modern LLM. Let's apply this
14:37lost landscape approach to our shallower
14:39eight layer model. When we vary the
14:42parameters in just the last few layers
14:44of this model, we see a very smooth and
14:46convex landscape, suggesting that the
14:49gradient descent learning process should
14:51proceed smoothly for these layers.
14:54Switching to the first few layers of the
14:56model, our surface becomes more complex,
14:59just as we saw with our single weight
15:01probes. Now that we've developed a more
15:03robust visualization approach, let's use
15:06our approach to explore our deeper
15:08models.
15:10As we added layers to our model earlier,
15:13we saw accuracy saturate around 26
15:16layers. The loss landscape of the first
15:18few layers of this model is
15:20significantly more complex than our
15:22eight layer model. This makes sense
15:25given that we've added 18 additional
15:27layers between our initial layers and
15:29our final output, increasing the
15:31complexity of this mapping. Here's the
15:34loss landscape for the first few layers
15:36of our deep 74 layer model. This
15:39landscape is especially chaotic with
15:42many local minima, especially compared
15:44to the smooth landscape we see for the
15:46last few layers of this model.
Shattered Gradients
15:49One way to think about the bearing of
15:51this landscape on the learning process
15:53is to consider the downhill direction,
15:56the gradient at each point on our
15:58surface.
15:59This is the signal that actually drives
16:01the learning process. Visualizing our
16:04gradient as a vector field, we see that
16:06our gradient direction varies wildly
16:08across our landscape.
16:11This general problem was given the
16:12memorable name the shattered gradient
16:15problem in this 2017 paper where the
16:18authors observed that as depth
16:20increases, gradients in standard feed
16:23forward networks increasingly resemble
16:25white noise.
16:27So the critical signal that guides our
16:29entire learning process, the gradient,
16:32appears to become less and less reliable
16:35as the depth of our model increases.
16:38This brings us back to Giansoon's
Residual Networks
16:40research team at Microsoft Research Asia
16:42in 2015.
16:44Remarkably, by the end of that year, the
16:47team was able to find an incredibly
16:49effective solution that [music] would
16:51completely address the shattered
16:52gradients problem, allow them to train
16:55models up to,2 layers deep, and
16:58completely crush every relevant computer
17:00vision benchmark.
17:02Earlier we saw that the output of each
17:04layer in our model is a tensor of
17:06activation values and that the
17:08compounding complexity of each
17:09additional layer as our activations pass
17:11through our model makes our gradients a
17:14potentially unreliable guide for the
17:16learning process.
17:18The solution GNS's team found is almost
17:21comically simple. Between each pair of
17:24layers in the model, simply take the
17:26input activation tensor and add it to
17:29the output of the layers.
17:32This operation is now often referred to
17:34as a skip connection. We're effectively
17:37adding a direct path around each pair of
17:39compute layers for activations to flow
17:41forward through and gradients to flow
17:44backwards through. Note that since we're
17:46adding full tensors together, when our
17:49skip connection goes around a down
17:51sampling step in our network, we need to
17:53change the dimension of the tensor
17:55moving through our skip connection. And
17:57soon's team found a few simple and
17:59effective ways to handle this.
18:01Let's temporarily switch to showing our
18:03network layers as discrete blocks
18:05instead of just kernels connecting our
18:06intermediate activations.
18:09Another way to think about the skip
18:10connection is that we're adding an
18:12identity pass through for our data
18:14around our compute layers and now
18:17relying on our compute layers to learn
18:19additional or residual behavior on top
18:21of this pass through.
18:24The team called their modified network a
18:26residual neural network or ResNet for
18:28short. ResNets work shockingly well,
18:32sweeping the 2015 imageet
18:34classification, detection, and
18:35localization competitions [music] and
18:37the KCO detection and segmentation
18:40competitions. Here's the ResNet papers
18:42lead author, Kaiming Hu, presenting the
18:44team's results at the CVPR conference.
18:47So, here is how deep learning looked
18:49like uh three or four years ago. And
18:51this is the Alex net that has eight a
18:54layers. So, here is how deep learning
18:56looked like uh two years ago. And these
18:58are the V net and Google net that has
19:00about 20 layer. And here is the deep
19:02residual network that have over 150
19:05layer. And here is another view of this
19:07zero network. And it is not too special.
19:10It is just a lot of layers. Let's add
19:13skip connections to our 74 layer
19:15network. As we saw earlier, this network
19:19struggled to learn, only achieving an
19:22accuracy of 38.9%.
19:25significantly lower than our much
19:26shallower eight layer network.
19:29We also observed the shattered gradient
19:31problem and a chaotic loss landscape
19:34when probing early layers of the model.
19:37Converting our network into a residual
19:39network by adding skip connections and
19:41retraining. Our model learned
19:44significantly more quickly, achieving a
19:46very strong accuracy of 72.6%.
19:50Outperforming all of the other models
19:52that we tested. And when we recomputee
19:54the loss landscape for our early layers,
19:57we see a dramatically smoother and more
19:59convex surface
20:02with similar geometry to the loss
20:03landscape from our later layers.
20:06Our skip connections provide a more
20:08direct path from our early layers to our
20:11model's output, effectively eliminating
20:14the shattered gradients problem. Few
20:17ideas in the history of machine learning
20:19have had such a profound and rapid
20:20impact on the field, ultimately earning
20:23the ResNet paper more citations than any
20:26other paper in the 21st century.
20:29But as the dust settled, some profound
20:31new questions came into focus. It's
20:34interesting to consider why such a
20:35simple idea had not caught on before.
20:38One likely reason is that skip
20:40connections violated the mainline
20:42understanding of how deep neural
20:44networks operated at the time.
20:46Within a few months of the ResNet
20:48publication, a team at Cornell would
20:50demonstrate some shocking [music] facts
20:51about ResNets. It turns out that we can
20:54just delete or even shuffle the layers
20:56and residual networks with only small
20:58impacts on performance.
21:01These realizations and subsequent
21:02developments would force the field to
21:04completely reconceptualize how deep
21:07neural networks operate. As we'll see,
21:09the ResNet team had inadvertently
21:11invented a new critical backbone for
21:13neural networks. a kind of working
21:16memory that would allow these models to
21:18reach unprecedented levels of
21:20performance.
Jane Street is a Fascinating Place to Work
21:22One of the things that I love about
21:24ResNet is that we have this relatively
21:27simple but incredibly powerful
21:29architecture change just waiting for
21:31[music] someone to discover. I recently
21:33had a chance to chat with Aloque
21:34Perinic. Alok works as a researcher at
21:37this video sponsor Jane Street. As Alok
21:40told me, he recently found himself
21:41digging into some fundamental
21:42architecture questions himself. And
21:45remarkably, it all started with Alok's
21:47own personal exploration into quantum
21:49mechanics. So, actually, the first
21:52reason I started thinking about this is
21:54because like I was studying kind of
21:56quantum mechanics for like totally
21:58irrelevant reasons. Um, and a thing that
22:01happens or that comes up a lot in
22:03quantum mechanics is like representation
22:05theory where you have certain groups.
22:07This got a look thinking about the
22:09connection to positional encodings in
22:11modern transformers.
22:13Unlike the convolutional neural networks
22:15that we've been considering, the
22:16attention mechanism in transformers does
22:18not naturally have a sense of where
22:20input tokens are relative to each other.
22:24So this capability is generally added to
22:26the architecture using a technique known
22:28as a positional encoding. Aloque has a
22:31great write up on his approach on the
22:32Jane Street blog. I won't spoil the
22:35whole thing here, but Aloque found that
22:37the space of possible positional
22:38encodings is actually surprisingly
22:40constrained and that the most sensible
22:43strategies are already being used in
22:44practice. But he also found that there's
22:47a completely unexplored set of
22:49approaches. Alok started his career at
22:51Jane Street as an intern. Here's one of
22:53the many great things he had to say
22:55about his experience. the fact that all
22:58the all my co-workers and like kind of
23:00collaborators were really like
23:03intellectually invested in the decisions
23:06like I I could just like walk up to
23:07anyone with an interesting topic for
23:10discussion and they like listen and have
23:13something insightful to say um was
23:15really really incredible. I feel like I
23:16was learning so much and I still feel
23:18like I'm learning a lot. If this sounds
23:19interesting to you, now is the perfect
23:21time to apply for the Jane Street
23:23internship program or for a full-time
23:25role. Jane Street offers internships in
23:27machine learning, [music] quantitative
23:29trading, and many other areas. Interns
23:32have the opportunity to work on real
23:34problems alongside some incredibly
23:36bright collaborators like ALOK. You
23:38don't need a finance background, and you
23:40can apply today at the link in the
23:42description below.
ResNets Force a New Understanding
23:45A few months after the ResNet
23:47publication, a team at Cornell showed
23:49that these new models demonstrated some
23:50highly counterintuitive behavior. In
23:542015, the predominant understanding of
23:56how deep neural networks operated was
23:58through learning hierarchical
24:00representations.
24:02This viewpoint was supported by strong
24:04empirical evidence.
24:06Here's the activation patterns from the
24:08first layer of AlexNet responding to
24:10simple features like edges and color
24:12blobs.
24:14Here's the second layer of the model,
24:16bringing these simple edges together
24:17into features like corner detectors.
24:20And here's the fifth layer of the model
24:22responding to fully formed abstract
24:24concepts like faces. So, as the field
24:27understood it at the time, each layer of
24:29the model builds on top of the learned
24:31representation of the layer before. In
24:34this mental model of neural networks,
24:37each layer plays a critical role in the
24:39chain.
24:40Deleting or transposing layers would be
24:42catastrophic.
24:44However, in 2016, the Cornell team
24:47showed that this was not true for
24:48ResNets.
24:50While removing or transposing layers
24:52would destroy the performance in
24:53non-residual networks like Alexnet,
24:56completely removing a layer from a 56
24:58layer ResNet barely impacted performance
25:02and performance degraded smoothly as the
25:04team removed more layers. Clearly, deep
25:07residual networks were not just building
25:09deeper versions of the hierarchical
25:12representations learned by models like
25:14AlexNet.
The Residual Stream
25:16Let's have a closer look at the skip
25:18connections introduced by the ResNet
25:19authors.
25:21Every two model layers, we take the full
25:24input tensor, move it around the layers,
25:27and add it to the layers output. Let's
25:29redraw our network in a slightly
25:31different way. We'll straighten out all
25:33of our skip connections into a single
25:35line from the input to the output of our
25:38network and draw the inputs and outputs
25:40to our layers as branching off of our
25:42continuous skip connection.
25:45Now, these drawings represent the same
25:47exact network. Just as before, every two
25:51layers, we take the full input tensor,
25:53move it around the layers, and add it to
25:56the layers output. However, this way of
25:58drawing our network shows that there's
26:00an unbroken flow of data from the input
26:03to the output of our network that our
26:05layers add to as we move through our
26:08model. In a residual network with six
26:11layers, if we label the outputs of each
26:13pair of layers F, G, and H, then our
26:17model's final output is equal to our
26:19input X plus F + G plus H.
26:23This flow of information from the input
26:26to the output of our residual network
26:28iteratively refined by each layer was
26:30later given the name the residual stream
26:33and has become one of the defining
26:35features of modern AI. This shift in
26:38understanding from the network's layers
26:40forming hierarchical representations to
26:43iteratively refining the residual stream
26:45can help us make sense of the Cornell
26:47team's results. If each layer in a 56
26:50layer ResNet is just incrementally
26:52refining the residual stream, then
26:55removing or transposing layers should
26:57only have an incremental effect.
27:00Now, we should note here that both of
27:02these things can be true. A separate
27:04research team would later show strong
27:06evidence that residual networks were
27:08both learning hierarchal representations
27:10in subsets of layers while iteratively
27:13refining the residual stream.
27:16In 2017, a team at Google published what
27:19would become the breakthrough
27:20transformer architecture, which is, by
27:23the way, now the seventh most cited
27:25paper of the 21st century. In their
27:27architecture, the team included skip
27:29connections between each of their
27:30alternating attention and multi-layer
27:32perceptron layers, placing the residual
27:35stream at the heart of their design.
27:38The transformer proved remarkably
27:40effective at language modeling, becoming
27:42the workhorse of large language models
27:45and ultimately working its way into
27:47vision applications, setting new
27:49state-of-the-art results on the imageet
27:51data set. But in 2023, a research team
Vision Transformers Need Registers
27:54at Meta noticed something strange about
27:57the residual stream in vision
27:59transformers.
28:01Here's a look at the residual stream
28:03from one of the models the team
28:04investigated, Dino V2. This version of
28:07the model has 40 layers and the residual
28:10stream is of dimension 37x 37 x 1536 for
28:14the full depth of the network. Let's
28:17simplify our visualization. After each
28:20layer, we'll collapse our residual
28:21stream into a single 37x 37 2D array by
28:26taking the maximum of our 1536
28:28activation values at each position. This
28:31gives us 40 2D arrays, each showing the
28:34maximum activation values in the
28:36residual stream after each of our models
28:3940 layers.
28:41In the earlier stages of the model, we
28:43can see high activations around visually
28:45important regions of the image such as
28:47the face, the hands, and the lights on
28:49the bookshelf. But as we move deeper
28:51into the model, these activations become
28:54completely dominated by some very large
28:56values at a small number of positions.
29:00If we overlay these high activation
29:02positions on our original image, we see
29:04that they generally appear in
29:06unimportant regions of the image, the
29:08wall, the door, and the drawers. Are
29:11these high activating positions just a
29:12random artifact of the architecture, or
29:15could they be serving some purpose?
29:18The meta team developed an interesting
29:20hypothesis.
29:21We proposed the following interpretation
29:23of these elements. The model learns to
29:26recognize patches containing little
29:28useful information and recycles the
29:30corresponding tokens to aggregate global
29:33image information while discarding
29:35spatial information. So under this
29:37hypothesis, the residual stream acts as
29:40a working memory where the model can
29:42store, edit, and retrieve information.
29:45The meta team tested their hypothesis in
29:47a few ways. First, they probed the
29:50residual stream at these high activating
29:52positions. training image classifiers
29:54using the embedding vectors from these
29:57positions as inputs.
29:59Remarkably, the high activating
30:01embeddings prove dramatically better at
30:03image classification relative to
30:05normally activating embeddings,
30:07especially on fine grain data sets. Non-
30:11highly activating embedding vectors
30:12reached only a 10.8% accuracy on the
30:15challenging cars data set, while the
30:18highly activating embedding vectors
30:19achieved an 85.2% accuracy. So these
30:23highly activating positions appear to be
30:26storing information about the image as a
30:28whole rather than being limited to
30:30information about the specific patch of
30:32image that they fall on.
30:34The meta team's next experiment was
30:36brilliant. They simply gave the model
30:38another place to put this information.
30:41The team added new positions in the
30:43residual stream alongside the 37 by 37
30:46grid of patch embeddings from our input
30:48images.
30:50These are embedding vectors of length
30:521536
30:53just like the embedding vector for each
30:55image patch. However, unlike the image
30:58patch embedding vectors, these new
31:00vectors are randomly initialized before
31:02training and their values at the
31:05beginning of the residual stream are
31:06learned like any other parameter.
31:09At the end of the model, these vectors
31:11are simply discarded. Their final values
31:14are not connected to any learning
31:16objective. The team called these new
31:18positions register tokens, borrowing the
31:21term from the registers that perform
31:23short-term data storage and computer
31:25CPUs.
31:27Let's visualize our maximum activation
31:29values as 2D grids as we did earlier,
31:33but now for a version of the model
31:34trained with register tokens in place.
31:38Remarkably, the strange high activation
31:40values that the team saw in DOV2 and
31:43other models completely disappear.
31:46Here's a view of the residual stream
31:48with and without the added register
31:50tokens. The model has shifted from
31:52storing global information in
31:54unimportant patch positions to actually
31:57using the register positions.
32:00These results strongly support the view
32:02of the residual stream as a working
32:04memory for the model. And it's so
32:07remarkable to me that in the absence of
32:09registered tokens, these models
32:11effectively learn to make their own by
32:14repurposing unimportant parts of the
32:16image.
AI & Scientific Progress
32:18A century before Alex Kreseky, Ilascover
32:21and Jeff Hinton published the AlexNet
32:23paper, Max Plac published on the law of
32:26the distribution of energy in the normal
32:28spectrum.
32:29Plac showed that if he assumed that
32:31energy was emitted and absorbed in
32:33discrete quanta, he could solve one of
32:35the most pressing physics problems of
32:37the day known as the ultraviolet
32:39catastrophe.
32:41Plunk considered his work more of a
32:43mathematical trick than a true
32:45discovery. But the effectiveness of his
32:47approach could not be ignored. A few
32:50years later, Einstein applied Plonc's
32:52idea successfully to the study of the
32:54photoelectric effect. And a few years
32:57after that, Neil's Boore extended
32:58Plonc's idea further into a quantized
33:01model of electron energy states within
33:03the atom. From here, the dominoes
33:06continued to fall in rapid succession,
33:09culminating in the late 1920s with the
33:11work of Schroinger, Heisenberg, and
33:12Durac, resulting in a complete
33:15reconceptualization of matter and
33:17energy.
33:19The physicist George Gamoff would later
33:21call this buildup to quantum mechanics
33:2330 years that shook physics.
33:27Today, the hype, noise, and
33:29commercialization around modern
33:30artificial intelligence can make it easy
33:33to forget that real science, real
33:35discoveries are happening and that we
33:37have a front row seat.
33:40Alexet definitively showed that given
33:43sufficient training data, deep neural
33:45networks could perform remarkably well.
33:48The discovery of residual networks a few
33:50years later was one of these critical
33:51early dominoes to fall, enabling deeper
33:55networks with unprecedented levels of
33:57performance and ultimately forcing the
33:59field to completely reconceptualize how
34:02these models work. Residual networks
34:05were critical in the subsequent
34:06development of the transformer, large
34:08language models, diffusion models, and
34:10virtually all modern AI systems we use
34:12today. We'll have to wait and see if
34:15most of the dominoes in this wave of AI
34:17have already fallen or if we're just
34:20getting started.
Support Welch Labs on Patreon!
34:27Now is the perfect time to join the
34:29Welch Labs Patreon. If you join at the
34:32$5 per month or higher level, we'll send
34:34you a real paper cutout from a video.
34:37Making and shooting all these loss
34:39curves was a ton of fun. This is the
34:41next batch of cutouts that will ship. We
34:43also do a book raffle every quarter for
34:46patrons. This quarter, we'll have two
34:48winners who will each receive a copy of
34:50the Welch Labs Illustrated Guide to AI
34:53and a book that I'm currently reading.
34:55I've just decided on the book for this
34:57raffle, and I really think you'll like
34:59it. Finally, the first batch of Welch
35:02[music] Labs t-shirts ever is coming out
35:04this fall, and patrons will get early
35:06and discounted access. Thank you so much
35:09for your continued support and stay
35:12tuned for some exciting new things
35:13coming out this fall.