Better than Jev - because you can build with it

0:00There's a free LLM that works like Jev,

0:02so it will make no mistakes. It's small

0:04enough to incorporate into your

0:06software. It can even be put into small

0:08hardware that you make. It doesn't need

0:10an internet connection to work, so it's

0:12private. And did I mention it's free?

0:14Here's how you can use it. Presented by

0:16Zapier, the AI automation company. Okay,

0:19Henry, we are now going to ask your LLM

0:22to do something. What does it say in the

0:23top right over there?

0:25>> Um turn off all the lights.

0:27>> Okay, and then you run it.

0:31>> So, it turned off all the lights.

0:33>> And all the lights in the top left

0:35turned off. Turn them on. Yeah, let's

0:36make sure people see in the left. There

0:38you go.

0:39Okay.

0:40What you're replicating here is a human

0:42being speaking, and obviously you're in

0:45this demo it's typed out, but a human

0:46being saying turn on all the lights, the

0:49LLM takes understands it, and then

0:51what's underneath? What it's underneath

0:52is the magic of what it's doing to

0:54ensure that it's accurate. What is that?

0:57>> Okay, so Needle actually generates like

1:00structured instructions,

1:02and you look for some kind of execution

1:04system, automation system to kind of

1:06carry out.

1:07We kind of just call them tool calls,

1:09right? So, this is what it looks like,

1:12and a lot of the output quality is

1:13dependent on how you scope out its

1:16available tools. So, what tools are you

1:19you know giving to the model? For

1:20instance, it has the ability to set

1:22lights. It can define ability to you

1:24know like um

1:25also like let's say set temperature and

1:27you know like other functionalities. So,

1:29I think a lot of the quality gets here

1:32is dependent on how you scope out its

1:35characteristics.

1:38>> Okay, I think I need to see another

1:40example to fully understand it. Let's

1:42look at something that's just code.

1:44Maybe go to the Yeah, go to that

1:46drop-down menu from your site, and then

1:48let's select search and matching, the

1:50very bottom one. Okay, this is more

1:51typical, right? I might write software.

1:54I'm not doing any lighting automation or

1:56home automation at all. and now what I

1:58have on the left is all the different

2:00things that my software can do. It can,

2:02on the top left, you can see set alarm,

2:04it could set a timer, it could play a

2:05podcast. I'm just reading randomly here.

2:08Um it could stop a timer and calculate.

2:10These are standard things that my

2:11software might do. The right is the

2:13query. You want to type in a query, and

2:15let's just see how it does it.

2:18>> Okay. So, we can use like uh

2:20presets. I I set up like a couple

2:22presets. So, users can literally just

2:24see uh examples to get started with. So,

2:27this is for example, remind me to renew

2:28the passport next month. And when you

2:30run it, it's kind of like you know, like

2:31sets

2:32you know, emits an instruction, which

2:34the software, the automation system has

2:36to consume and actually execute. So, in

2:38this case, it needs to call the set you

2:40know, reminder, and it needs to like

2:42renew the passport. And it needs to, you

2:44know, like for next month as instructed.

2:47There are other things you can also do.

2:48You can also like let's say search.

2:51You know, like we also do like a

2:53structured extraction for instance.

2:55>> Mhm.

2:55>> Um so, giving like a schema, let's say

2:57what you want to extract from a body of

2:59text. In this case, dentist.

3:02Right? So, it says find my notes about

3:04the dentist, and it was able to kind of

3:05like

3:06find the notes that has the word

3:08dentist. It like you know, that's quite

3:09similar to dentist easily.

3:11>> Okay. So, I want to get more practical

3:14before we see more examples. When I

3:16write software, and I'm I'm just using

3:18Claude code, I tell it, "Here's what I

3:20need you to do. I want a specialized

3:22note-taking app that also takes some my

3:23notes and turns them into do's, and then

3:25I want the do's to buzz me at this

3:27time." One of things that my software

3:29will say is, "Do you want to connect to

3:31the Claude API?" And I'm like, "No, I

3:32don't need the Anthropic API. It's going

3:34to be too expensive." So, then it says,

3:36"Do you want to use these free things?

3:37They're free for a limited time or free

3:39for a limited set of things." What

3:41you're saying is, "I don't need to go to

3:42a third party like Anthropic or some

3:44other some other. I just take Needle

3:48because it's small and it's dependable,

3:50and the dependable part is maybe even

3:52more valuable than the small in this

3:53case. I can know exactly what it's going

3:55to do. It's not guessing, but it also

3:57can't do a million different things. It

3:58can't also act as my therapist. Because

4:00of that, I just give it to Claude. I

4:02say, "Incorporate this." And now I don't

4:04have to make any external calls, and I

4:06get a dependable result when I'm coding.

4:08That's the thing, right?

4:10>> Yes, correct. So, um Cactus is quite

4:13open source. So, on the GitHub you have

4:15access to everything.

4:16Um most times if you want Claude to

4:18build an app for you, you just use this

4:20lm.txt and literally just copy this and

4:23>> [laughter]

4:24>> give it to Claude, and Claude gets uh

4:26the context for how to use like you know

4:29>> Got it. I just give it to Claude. I say,

4:30"Boom. Now start using it." Everything

4:32is now contained, and I have my

4:33note-taking app without without even

4:35necessarily even being online. It could

4:36be directly on my computer, right?

4:39Okay.

4:39>> Correct.

4:40>> So, this is the brilliance of it. But,

4:43the reason that we started talking about

4:44you on the channel was you guys were

4:46trending like crazy on GitHub. And what

4:48people are excited about was not just

4:50these use cases, it was going beyond the

4:53computer to actually put it on devices.

4:55And so, you have other examples here

4:56that show what's possible. Give me one

4:58more. What's another example?

5:01Maybe the vacuum?

5:01>> Okay. So, for example, yes, for example,

5:04you have like

5:05like vacuum systems where you just leave

5:08them laying around, but sometimes they

5:10just do things we wish you could

5:12actually tell them what exactly we want

5:14them to do. So, with Noodle installed on

5:17this powering this vacuum system, you

5:18can actually get them to do stuff. They

5:20can be like you can explain to it to

5:22clean the kitchen and not the bedroom,

5:24for instance, and it just clean the

5:26kitchen

5:28not the bedroom, right? And not the

5:30living room. And then you can follow up

5:32with like

5:33uh

5:36uh go to the dock, and it goes back to

5:38charge itself, right? So, um you can

5:41have voice control, you know, like a

5:42you're able to command, you know, like

5:44your home system the way you want with

5:46natural voice.

5:48>> So, if I'm making a robot vacuum,

5:51I get to take this, put it on the vacuum

5:53itself. It doesn't have to call into the

5:55internet to get the to get the LLM. And

5:57now human beings can interact with it

5:59the way that we naturally talk with it.

6:02I get it. Um you know what? What I want

6:04to know from you before we go into more

6:06examples is

6:07how does it not make mistakes or

6:09virtually not make mistakes? And you

6:11explained it to me before, but I didn't

6:12fully get it. Let's get it here. But

6:14first, I want to say my sponsor is

6:16another company that's a Y Combinator

6:17company like you. You're both backed by

6:19YC. It's Zapier. And if anyone out there

6:22is listening to me and building software

6:24themselves, yes, you should absolutely

6:26put Needle in if what you want is this

6:28LLM that's self-contained and free and

6:30part of your software. And we'll see why

6:33it won't make make many mistakes. But if

6:34you also want to connect it to Notion,

6:36um Salesforce, HubSpot, and thousands of

6:39other apps, you can build those

6:41integrations yourself so your user has

6:42it or you just go to zapier.com/sdk.

6:45They've had over a decade of connecting

6:48apps together. They'll bring that

6:50intelligence into your software and

6:51you'll be able to give that speed and

6:54usability and access to your users. Go

6:56to zapier.com/sdk.

6:58And tell them Andrew or The Next New

6:59Thing or any one of them. Uh because

7:02they don't even want to use you like

7:03URLs with me. They just want people to

7:05say we heard it from Andrew. Okay,

7:07Henry,

7:08what why you explained to me earlier why

7:10it doesn't make as many mistakes or

7:11hardly any. I don't understand. Tell me.

7:14Tell me again.

7:15>> So yeah, a lot of times people um I I I

7:17we see a lot of people testing the

7:20random queries without having to define

7:22that activity here that capacity here.

7:25So Needle is kind of like designed to be

7:27customized. You have to, you know, like

7:30carefully craft like its behavior. Like

7:33in the description you give it here like

7:35of what each tool can do is so strict uh

7:38because automation is kind of like a you

7:40know, like a very like careful you know,

7:42like setup. You don't want it to kind of

7:44like turning on like the fridge or

7:45turning off the fridge over night and

7:47having your food spoil. So, um we kind

7:49of like really want to prevent the model

7:52doing stuff it wasn't asked to. And so,

7:55you have to really pay a lot of

7:56attention. For instance, you can even

7:59like

8:00enumerate like options, right? Similar

8:02to Jeff, right? It can only select one

8:04of these four options. Forward,

8:05backward, left, for instance.

8:07It shouldn't like go out of that scope.

8:10So,

8:11we we have like a blog post on like how

8:13to design tools for Nido. It's kind of

8:16like we kind of use a lot of visuals to

8:18explain in like the best way to kind of

8:20like set up tools and how to kind of

8:22like ensure you like it just acts the

8:24way you want it to.

8:27>> So, what you're saying to me is it has a

8:30limited number of options that the

8:32developer articulates ahead of time. And

8:34still, can't it make a mistake where I

8:37say turn on the lights and it would turn

8:39off the lights? Off and on are both

8:40options.

8:42>> Exactly. So, it can if if this is poorly

8:45designed, right? But most times, it

8:48would much rather refuse to act.

8:51And also, we have the concept of like

8:53confidence.

8:54Nido kind of like returns its confidence

8:56score. Like let's say if we set this up,

9:00um

9:01you're going to see it's a

9:04uh confidence 0.97 confidence. And so,

9:07you know, like it gives you like

9:08its own self-reflection, you know, what

9:11it's done. And if it's confident, then

9:13you got to trust the numbers. But

9:15sometimes, when it's it has low

9:17confidence, sometimes Nido refuses to do

9:19anything. And sometimes, it still gives

9:22you the results, but it gives you its

9:24confidence also. It's like so, you

9:25decide. I don't think, you know, like if

9:27it's below like this threshold, just

9:29don't execute it.

9:31>> Okay, so if the creator, the developer

9:33gives it clear boundaries, clear sets of

9:37things it can do, and then

9:39before taking action based on what

9:41needle spits back out, it looks for the

9:43high enough confidence. At that point,

9:46it is not going to make mistakes.

9:49Got it. And And part of the power is

9:51also the limited capabilities. It is

9:54also not going to do what ChatGPT does,

9:56which is suddenly go from being your

9:57therapist to being the thing that

9:59controls your computer, to being the

10:00thing that then writes code for you. It

10:02is limited, limited, limited. Okay. So,

10:04now with that in mind, let's take a look

10:05at a couple of other examples from that

10:07drop-down menu on your site. What else

10:08can it do?

10:10>> Okay. So, a phone.

10:11>> Okay.

10:12>> Let's say a phone cuz that's like easy

10:14easy to do. Okay, let's say for

10:17instance, let's say

10:20this create an album called weekend and

10:23move the photos from

10:25you know, like from last weekend into

10:26it. And you can see, you know, like this

10:29you know, like these are like it

10:30selected the photos from last weekend.

10:32And we just kind of like in this demo,

10:34you just kind of like uh

10:35extract the tags you know, like uh from

10:37last weekend. And it's ready to move

10:39them into you know, like uh

10:41Sorry, into like uh favorites. And then

10:43uh

10:45uh we also have like options, let's say

10:47on your laptop.

10:48>> Okay.

10:49>> Uh for instance, you want to like uh

10:50start a timer.

10:52Uh or you want to like uh it it can you

10:54know, like start a timer with the voice.

10:56You know, like uh and then you can also

10:58like I don't know, like copy like your

11:00clipboard. You can listen listen to your

11:01laptop. Hey, could you copy what's on

11:03the screen to clipboard?

11:04Instead of having to manually kind of

11:05scroll it. So, technically, it should be

11:08able to automate anything, actually. As

11:11long as you you scope it quite well.

11:13So, it could be able it should be able

11:15to, you know, like control your whole

11:17laptop as long as you give it the

11:19available enough tools to do so.

11:22>> All right.

11:22>> Okay. I Now I'm seeing the

11:24possibilities. I'm seeing why someone

11:26would, for example, want to incorporate

11:27this into a desktop app or iPhone app so

11:30that it doesn't make any

11:32It doesn't have to go online to get its

11:34intelligence.

11:35This does seem a lot like Jev and you've

11:37now used Jev a few times in my private

11:39conversation with you in this. How is

11:41this like Jev?

11:43>> Okay, yeah. So, Jev we kind of like have

11:46similar uses around like 50.

11:49We have to we found a lot of times when

11:51it comes to practical tasks people don't

11:54necessarily build automation systems

11:56with generalist LLMs. You don't need the

11:58LLM running. And you know, like those

12:01are for like chat-based applications

12:03like interactions like writing code and

12:05stuff like that. But for general

12:07automation you don't need all that

12:08feature. You simply need like safe

12:11actions that

12:13and that's kind of like the

12:15the ethos. And so, the whole concept is

12:18to kind of restrict

12:20the model's output so it's forced to

12:22kind of like

12:23select you know, like within like a

12:25limited space. In this case for example,

12:28a finite amount of possible tools. It

12:30shouldn't have access to I don't know

12:32delete pictures if you don't give it the

12:34tools. It shouldn't have access to

12:36anything else.

12:37Also, within that each tool it also has

12:41a finite and like capacity you know,

12:43like you can limit its options also.

12:46And anything beyond that it doesn't do

12:47anything.

12:48>> And is that how Jev works also? Limited

12:50options and limited actions?

12:54>> Yes, so Jev is

12:56kind of like a subset but you can still

12:58solve a problem with Jev if you use it

13:01so Jev is kind of like strictly based on

13:03enums. It it it literally

13:05just model has to select you know, one

13:07of multiple choices.

13:09>> And that's the way it works with Jev

13:10too, you're saying? And with Needle?

13:12>> Yes.

13:12>> I didn't realize that.

13:14Okay. So, are they competition now that

13:16they've launched? You seem pretty happy

13:17to talk about them. I thought then it

13:19would be an issue for you.

13:22>> No, actually. So, we're on device. Jev

13:23is cloud.

13:25And we focus on much smaller devices

13:27like this is I think laptops are kind of

13:30like the biggest we can of go for.

13:32Our models are kind of 8 to 29

13:34megabytes.

13:35We kind of design them to run on like

13:37your smart TV remote, your your smart

13:40watch, your you know your airpods and

13:41you know like devices like that.

13:44You know where you need like ultimate

13:46privacy. You wouldn't want like a real

13:48time AI let's say home speaker you put

13:50in your kitchen to listen to your

13:52conversations 24 hours and do stuff. You

13:54don't want it to kind of like stream

13:56into a server. You wouldn't trust a big

13:57corporation to kind of like

13:59have access to your you know your live

14:01activities, right? So it's best

14:04designing you know for stuff like that

14:06we call AI. So that's kind of our

14:07market.

14:09>> Okay. Let me say this.

14:11You and I discovered each other because

14:13you were trending on GitHub. You were

14:15one of the top 10 of the week.

14:18Anyone who's watching right now should

14:19see the other nine and how we covered

14:21Henry's needle on that and there's a

14:24link right here where you can go and

14:25watch the top 10 GitHub repos which

14:27needle was a part of. See you there.