Full transcript
At some point you had an electrician
0:00At some point, you had an electrician in
0:02your house. He was at your breaker panel
0:04with a voltage tester, and he asked the
0:06question electricians ask, which is,
0:09"What's it for?" And you had to decide,
0:11in your own hallway, how honest to be
0:14with a professional about why you needed
0:16a dedicated 20-amp circuit run to a
0:18spare room. You said it was basically a
0:21computer. He looked past you at four
0:24graphics cards bolted to an open frame
0:26on a shelf, humming with no case around
0:29them. He didn't say anything. That was
0:31its own kind of review. Because here's
0:34what nobody warns you about. The ceiling
0:36on this hobby isn't money, and it isn't
0:39skill. It's the copper in your walls.
0:42There's a hard limit on how much
0:44artificial intelligence you can install
0:46in a home, and the wiring sets it, not
0:48you. And all of it, the electrician, the
0:51frame, the 20 amps, started with a
0:54switch. You turned off the Wi-Fi just to
0:57check. And a thing sitting on your own
1:00hard drive, connected to nothing,
1:02answered you. Hold on to that flick,
1:05because the gesture never changes. Only
1:08the price does. Level zero. I ran it on
Level zero — I ran it on my laptop
1:11my laptop, 10 minutes. That's the trap.
1:15You install Ollama, pull a 4-GB file,
1:18and there it is, producing words at
1:20about the speed of someone typing with
1:22one finger while thinking about
1:23something else. You showed one person.
1:26You were quite excited. They watched it
1:29crawl out a paragraph and said, kindly,
1:31"Isn't that just a worse ChatGPT?"
1:34And it is. On that laptop, in that
1:37month, it is a worse ChatGPT that also
1:40heats your legs and makes the fan sound
1:42like it's trying to leave the building.
1:44You had no answer. You changed the
1:47subject. Four out of five people stop
1:49here. They delete the folder and go back
1:52to the browser tab, and they're right
1:54to. You didn't. And the reason is the
1:57actual beginning. The question in your
1:59head changed shape. It stopped being, is
2:03this any good? And became, what would it
2:06take to make this good? The first is a
2:09review. The second is a project. And
2:12projects have budgets.
Level one — I have a gaming GPU anyway
2:15>> Level one. I have a gaming GPU anyway.
2:18>> The card in your desktop plays games. It
2:21turns out it has a second job. And the
2:24second job pays in status. This is the
2:27largest doorway into the whole thing.
2:30Not researchers, not privacy obsessives,
2:33but gamers who found their toy was
2:35secretly industrial equipment. And this
2:38is where your mental model of computers
2:40gets rearranged. Because you arrive
2:42assuming the fast [music] card wins.
2:45Wrong axis. The card isn't an engine.
2:48It's a bucket. How much memory it holds
2:51decides whether a model can exist on it
2:54at all. How fast that memory moves
2:56decides how quickly it talks. Speed was
3:00never the constraint. Room was.
3:03You learn this the way everyone does.
3:05CUDA out of memory in a terminal at
3:0711:00 at night. It isn't really an
3:09error. It's a tape measure. Then the
3:13crueler lesson underneath. You abandon
3:16Ollama for llama.cpp,
3:18find a page of launch flags, and
3:20discover the same 8 GB card doing 10
3:24tokens a second will do 30 if you just
3:27tell it where to put things. You spent
3:30two weeks blaming silicon. It was a text
3:33file. And then you say it. Everyone says
3:36it. At every level, forever. I'm only 8
3:40gigs short. Nobody in the history of
3:43this hobby has ever been only 8 gigs
Level two — the 24gb threshold
3:46short. Level two. The 24 GB threshold.
3:51You buy a graphics card from 2020 from a
3:54person you will never meet again.
3:56Sometimes that's a shipping label with
3:58no return address. Sometimes it's cash
4:01in a parking lot at a time you agreed
4:03over text and neither of you can work
4:05out why this feels illegal. $1,200
4:09and 2 years ago it was half that. A
4:126-year-old consumer product is
4:14appreciating, not because gamers want
4:16it, but because several hundred thousand
4:19people are trying to fit a mind into a
4:21closet and a used 3090 is the cheapest
4:24honest way to buy 24 GB. It arrives and
4:28you immediately take it apart because
4:30there's a decent chance it spent 2021 in
4:33a warehouse mining cryptocurrency and
4:35its thermal pads are now the consistency
4:37of old chewing gum. You replace them
4:40yourself on a $1,200 card from a
4:43stranger. This feels normal to you now.
4:46But the real shift isn't technical.
4:48Everything before was repurposing a
4:51laptop you had, a card you already
4:53owned. This is the first object that
4:56entered your home because of this and
4:58when somebody asks what it's for, you're
5:01going to have to say a full sentence.
5:03You tell yourself it holds its value,
5:06that you'll sell it on. Technically
5:08true. You won't. And then almost
5:11immediately the ceiling. Llama 70B at
5:144-bit needs about 42 GB. You have 24. So
5:19you settle for something smaller and
5:21it's genuinely good and you're genuinely
5:23happy and there is now a number on a
5:26screen telling you precisely what you
5:28can't have. Level three, two cards, the
Level three — two cards, the rubicon
5:32Rubicon.
5:33The second card arrives months later and
5:35was obviously always going to. 48 GB and
5:40for the first time the thing on your
5:41desk isn't a demonstration, it's a
5:44collie. Slower than the cloud, worse at
5:47hard problems, but holding a real
5:49conversation without anyone's
5:50permission. The case stops closing
5:53around now.
5:54>> [music]
5:54>> There's a riser cable. There may be a
5:56second power supply resting on a book.
5:59It has stopped being a computer in the
6:01way a thing with a lid is a computer and
6:03become equipment. Then, on a Tuesday in
6:06April, somebody you live with comes in
6:08looking for a phone charger. They stop
6:10in the doorway. They don't say anything
6:12for a second. Then, "Why hot in here?"
6:16And you say the fans are fine, which is
6:18true, and they say it's like a car in
6:20here, which is also true, and they take
6:22the charger and leave. No argument.
6:25Nothing happens. But you sit there
6:27afterwards understanding that a file has
6:29been opened on you, and every decision
6:32you make about this now has a second
6:34person in it. So you find the trick, and
6:36you will tell people about the trick for
6:38years.
6:39Dial the cards from 350 W down to 220,
6:43and you lose almost nothing. 39 tokens a
6:46second instead of 44. 90% of the
6:49performance for a third less heat.
6:52Everyone who finds this believes they
6:53discovered it. Every one of them is
6:55doing it because of the doorway. There's
6:58a sideways move here, and some people
7:00take it. Instead of a third card, you
7:02buy the quiet box, a Mac Studio, or that
7:05$4,000 Nvidia brick that draws less than
7:08a hairdryer and sits on a desk looking
7:10like a router. Silent, small, and
7:12requiring no conversation with anyone
7:14you live with. It has one flaw, the
7:17exact inverse of level one. Room is not
7:20the same as speed. Hand one an
7:238,000-word prompt, and it can take 14
7:26minutes and 48 seconds to produce a
7:28single character. It works. It
7:30absolutely works. It works at the speed
7:34of mail. So the advice from everyone
7:36further up is to stop here or step
7:38sideways. Two cards is easy to cool,
7:42easy to power, and does most of what
7:44you'd realistically ask. Nobody stops
Level four — 4×3090, the summit
7:47here. Level four, four times 3090, the
7:51summit. Four cards, 96 GB, $3,500
7:56in graphics alone. Realistically, 8 to
7:5910,000 once you've bought a server board
8:01with enough lanes to feed them. The
8:03first surprise is geometric. Four modern
8:06cards do not fit inside anything sold as
8:09a computer case. So, you buy an open
8:11frame built for mining, and your
8:13artificial intelligence now lives on a
8:15shelf in the open, like a set of
8:17weights. The second surprise is that
8:19cooling kills more of these builds than
8:21power does. Everyone plans the
8:23electricity. Nobody plans the air. Two
8:27cards made a room warm. Four cards make
8:30it a different room. You have installed,
8:33permanently, roughly the heat of a space
8:35heater you are not allowed to switch off
8:37because it's thinking. In January, this
8:40is a feature, and you will tell people
8:42it's a feature. By June, it's a room you
8:44enter briefly and at speed, like a sauna
8:47you're annoyed with. And then the
8:49arithmetic that put a man in your
8:51hallway. Four cards at stock pull
8:54roughly 2,000 W. One builder in an
8:57apartment ran those numbers, looked at
8:59his wiring, and stopped at two because
9:022,000 W is well past what his flat could
9:05tolerate while somebody was also running
9:07the dishwasher and the kettle. Read that
9:10again. The frontier of consumer
9:12artificial intelligence in this decade,
9:15in a wealthy country, is capped by a
9:18kettle. At sustained draw, that's
9:20something like $300 a month in
9:23electricity, a number you will mention
9:25to nobody, and which you have absolutely
9:27calculated. Then there's what you were
9:30promised and don't get. 96 GB is not one
9:34pool. How much you can use depends on
9:36the software, the wiring between cards,
9:39and the model's shape. And VLLM, the
9:43serving engine half this hobby runs on,
9:45quietly recommends in its own docs that
9:48you use a single GPU whenever the model
9:50fits on one. Somewhere in here, a used
9:53server board will decline to turn on.
9:56Not fail, decline. So will its
9:59replacement. One builder lost three
10:01weekends and two boards before changing
10:03brands. There's a detail in all this
10:06that says everything. Builders choose
10:08parts so the thing reads as a slightly
10:11oversized gaming PC rather than what it
10:14is, because it has to pass as normal to
10:16visitors. You are camouflaging your
10:18hobby inside a different hobby. That
10:21camouflage is the whole problem,
10:23actually. There's no uniform for this
10:25one. Nothing you can wear that explains
10:27to a stranger what's humming behind a
10:29door in your house. So we made a hoodie
10:32about it at shop.devpsych.com
10:35on the theory that it's the closest
10:36thing this has to a membership card and
10:39dramatically cheaper than another
10:40graphics card. Around now the machine
10:43begins to move. Desk, under desk,
10:46closet, garage. Every level up pushes it
10:50one room further from where people live.
10:52You tell people it's for fine-tuning.
10:54You have fine-tuned one thing. It was in
Level five — you have left the hobby
10:57March. Level five. You have left the
11:00hobby. $10,000,
11:03one card, 96 GB of error-correcting
11:06memory, 600 W.
11:09There's no build story here because
11:11there's no build. You bought a solution.
11:14The project is opening a box. And
11:17something quietly ends. A friend's
11:19company started routing work through
11:21your machine because it was there and it
11:23was free and you offered. There is now a
11:25Slack channel with four people in it
11:27where somebody occasionally types, "Is
11:29it up?" So you stop taking the side
11:32panel
11:33You stop reading release threads at
11:34midnight. You stop trying the
11:36interesting new model because it might
11:38break the thing four people are typing
11:40into.
11:41The tell is linguistic. The day you say
11:44the cluster instead of [music] my rig,
11:47something has finished. Fun's over.
11:50Uptime has begun.
11:52Here's the part nobody prepares you for.
11:55The problem was never the model. The
11:58problem was having a problem.
12:00And the moment you can bypass every
12:02constraint at once, the part you enjoyed
12:05evaporates. People at this level talk
12:08about their old two-card setup the way
12:10other men talk about a car they sold.
Level six — the way back down
12:13Level six, the way back down.
12:16So, you sell it. The listing goes up on
12:18a Sunday. Four times RTX 3090, open
12:21frame, all tested, buyer collects.
12:25You light and photograph it more
12:26carefully than you have ever
12:28photographed a member of your own
12:30family. A man drives three hours to
12:32collect it. He's at level three. He's
12:35thrilled. He asks questions in the
12:37driveway that you answered for yourself
12:39two years ago, gets them answered in 40
12:41seconds, and hears none of it because
12:44nobody does.
12:46Watching him load it into a hatchback is
12:48like watching yourself leave.
12:50Then the room goes cold. The bill drops
12:53by 2/3.
12:54The frame is gone, and the shelf is just
12:57a shelf.
12:58And here's the part that stings. You use
13:01it more now because while you were
13:03climbing, the models came down to meet
13:05you. What needed 96 GB in 2024 fits in
13:0924 today and runs faster. You didn't
13:13downgrade. You waited expensively in a
13:16hot room for the software to catch up.
13:19You ask it normal questions now in a
13:21room at a normal temperature with nobody
13:24standing in the doorway, and you write
13:26it up and post it, and it does numbers
13:29because it gets read at 1:00 in the
13:30morning by people sitting next to a
13:32machine they've become slightly afraid
The uncomfortable truth
13:34of. The uncomfortable truth.
13:37Here's the confession nobody posts.
13:39Almost everyone at every level of this
13:42is hybrid. Local for the bulk work, the
13:45private documents, the overnight jobs,
13:48the things you'd rather not upload. The
13:50genuinely hard thinking still quietly
13:53goes to the cloud. Almost nobody leads
13:55with that because the identity of the
13:57thing is independence, and the honest
14:00version is mostly independence. So, why
14:03do 800,000 people keep going?
14:07In January 2024, a man named Tom
14:10Jobbins, who everyone knew as The Bloke,
14:13disappeared. For a year he had taken
14:15nearly every significant new model,
14:18converted it into a file ordinary people
14:20could actually run, and put it on
14:22Hugging Face for nothing.
14:24He is the reason this hobby has
14:26civilians in it. Then, he stopped. No
14:29announcement, no reason, no goodbye. And
14:33nobody really knows why. The community
14:36didn't collapse. Within weeks, other
14:38volunteers, Bartowski, Em Rademaker, a
14:41dozen more, had filled the gap. And
14:44they're still there tonight converting
14:46files for strangers, unpaid.
14:49That's what you joined when you handed
14:50cash to a stranger in a parking lot. Not
14:53a product, a relay.
14:55Because everyone else is renting their
14:57thinking from about five companies on
14:59terms that can be rewritten in a blog
15:01post with a model that can be retired on
15:04a Tuesday. And you never wanted a better
15:07model than theirs. You wanted to be the
15:10only person who could switch yours off.
15:13So, go on. Turn off the Wi-Fi.