Why Your AI Agent Sucks (It's Not the Model)

0:00In this video, I'm going to show you

0:01exactly how you can take your Hermes or

0:04open claw or claw code agent from

0:07producing AI slot and mid-tier content

0:10through to being something that is

0:12incredibly good and incredibly

0:15personalized. And so all about building

0:17the correct infrastructure around the

0:19context layer.

0:21So this isn't going to be a fancy demo

0:24where I showcase the actual agent

0:26running in action. Today we're going to

0:28dive into a little bit more around the

0:30thesis and how it looks under the hood.

0:34So I believe in 2026 the era is not

0:40around which LLM is the smartest. To to

0:43be honest, you can pick any LLM out

0:45there and it's going to operate

0:47incredibly well if we build the

0:49infrastructure sensibly around that

0:52agent. And the key kind of main two

0:55things are the context engine that we

0:58build and the the uh memory uh system

1:02that we then kind of scaffold around

1:04that context essentially.

1:07So this is how it looks for me.

1:11>> [music]

1:11>> I break the context layer down into two

1:15kind of key categories: internal context

1:18and external context.

1:21>> [music]

1:21>> So internal context is thinking about

1:24how can you plug your agent into your

1:28business, into your life essentially. So

1:31these are things like sales calls, um

1:34uh

1:35internal meetings. So I use like

1:37something like Grainer for that.

1:38>> [music]

1:39>> It's Notion documents, it's CRM

1:41documents, it's it's it's business

1:43performance, churn stats, new clients

1:45coming on board. What's the conversion

1:48ratios look like? What's your website

1:50performance looking like? It's all

1:52internally owned data essentially. And

1:55there's lots of different tools here.

1:56The ones that I'm commonly using is

1:58HubSpot, Notion, Slack,

2:00>> [music]

2:00>> Gmail, Fireflies, Granola,

2:03and FeatureBase. Those are the common

2:05ones that I have connected right now

2:07into my Hermes agent.

2:11Then we use external contacts. So, this

2:14is providing

2:16data points to our agent that are

2:19externally facing from the business. So,

2:22very, very popular is social media

2:25platforms and also maybe like news

2:27publications, things that are happening

2:30that we don't control or own. To do

2:32that, I use Triggerfly as the tool which

2:35allows me to connect through unified

2:37APIs to all of the socials and news data

2:41as well.

2:43And the way that we are using or

2:46basically

2:47creating the memory system, which I've

2:49talked a lot about on previous videos,

2:52is using the kind of Cap'n C Obsidian

2:55style memory [music] system here. So, in

2:58layman's terms, if we boil this down

3:00into a very simple

3:03analogy of describing it, what happens

3:06is it takes all of the data points from

3:09both the internal external point. It

3:11feeds it into this wiki style memory

3:15system as raw data. So, we're just

3:18dumping the raw data into a database,

3:21essentially, which in this case is just

3:23an Obsidian note.

3:25Then what happens once a day, twice a

3:27day, is we have an automation, aka cron,

3:30set up that works across the raw data

3:33files [music] to synthesis any of that

3:36data down into an actual tangible memory

3:41that the agent then can reference going

3:43forward. So, think if you're using a

3:45working on a new project or you've said

3:48that you love this style or it's found

3:51that there's a new piece of content

3:52that's blowing up or the business that

3:55is now at 10 million ARR or whatever it

3:58is. That is key pieces of information

4:01which we'll take from the raw data

4:03then convert into [music]

4:05a memory.

4:07And the nice thing about Obsidian is it

4:10then has a tagging system internally

4:12built into it so that it can then

4:14basically

4:16um

4:17semantically work across all of these

4:19different notes. So, for example, what

4:21that looks like and how that feels is if

4:24there is a key piece of information and

4:26I've done a big video on memory on

4:27another one, so please go check them

4:29out. But let's say there is a memory

4:31around uh social media content. That

4:34might be linked to another piece which

4:35might be uh preferences. That preference

4:38might be linked to another piece and it

4:39can work its way through the memory

4:42system.

4:44And that then helps produce this

4:47uh kind of layer of outputs. So, for me,

4:51one of the things that I do is I have a

4:53massive kind of content operating system

4:55and I'm going to be doing a new video

4:57showing exactly how my content OS system

5:00works, but I also have an AI SDR. We do

5:03a lot of just different cold email

5:04campaigns.

5:05>> [music]

5:05>> We uh do a lot of like internal business

5:08automations that run based off the back

5:11of this information. And the results

5:13would be subpar with all of these

5:15different outputs if this top layer was

5:18not good in the first place. This is why

5:20it is critical to have a good memory and

5:23there are You don't have to use Depths,

5:25there's plenty of other memory systems

5:26that you can use plug and play.

5:28And an internal and external context

5:31infrastructure set up uh around it.

5:35Why I really like social as an external

5:39uh context point, we are going from

5:42signal through to learning very, very

5:45quickly. So, the way that I have it set

5:46up is I'm monitoring fast-performing

5:50social media sites to find the signal.

5:52So, these will be things like X,

5:54LinkedIn, TikTok, Threads, Instagram, so

5:56on. These are things that have high

5:57velocity in terms of posts. And I'm

6:00searching for what's trending, what's

6:01doing well,

6:02>> [music]

6:02>> and wanting to tap into where is the

6:04market going? What are we looking at?

6:06What should we be looking at? Once we

6:08find something that's resonating, we

6:10then go to the longer-form content,

6:13Reddit, Substack, podcast, YouTube, so

6:16on, to understand and learn about that

6:19thesis. So, a great one here, as you can

6:20see in the example, is

6:22Claude code for a is trending, for

6:25example.

6:26It is like, "Okay, well, that's an

6:27interesting subject. Let's go find on

6:30YouTube people talking about Claude code

6:32for AIO and learn everything that there

6:34is to learn about it, take down any of

6:36the raw data, and then process that as a

6:38memory going forward."

6:41Now, [music] here's where it gets really

6:42interesting. We have this kind of raw

6:45belief to a memory system, right?

6:48>> [music]

6:48>> So, first of all, it comes in as raw,

6:50which I've uh embedded. It's in bed,

6:52it's tagged, and it's sourced. Then we

6:54do this kind of step two, which is this

6:56belief check system. So, it's looking

6:59across the raw data set, and it's trying

7:01to understand if there are X amount of

7:03repeatable actions happening. So,

7:07um

7:07it's it's consistently seeing something.

7:10It's not just a one-off, unless that's a

7:12very tangible business fact. Um it then

7:16stores it as a memory, right? So, this

7:19has to happen two, three, four times

7:21before I then convert this as into

7:22memory, unless, like I said, it is a

7:24very tangible black and white [music] uh

7:27business-oriented goal that I've stated

7:29or that it's picked up uh as well.

7:33And this then allows us to create the

7:36skill system on top, which builds the

7:39outcomes, right? So, let's say you're

7:41setting up an AI SDR, you maybe want

7:43that AISDR to have certain skills like

7:45cold email writing,

7:47uh LinkedIn writing, whatever it is,

7:49right? Those skills are nothing unless

7:53you attach that skill to the entire

7:56infrastructure that we've just talked

7:58about.

7:58>> [music]

7:59>> Yes, they might produce a slightly

8:00better output than just chatting to it

8:02raw, but you still have generic soulless

8:06piece of content that won't have any

8:09personality, depth, or meaning to it

8:12uh as if you would have add the memory

8:15and context layer into everything.

8:17>> [music]

8:18>> So, hopefully from this video, if I can

8:21get anything, you need to think about

8:23that internal and external

8:27uh context system. It's super simple to

8:29set up. It requires no heavy lifting.

8:32Simply take this video, get the

8:33transcript, plug it into your agent, and

8:36say, "Hey, help me set up this system

8:38that this person is talking about." I'm

8:41going to share this link with you down

8:42in the description. Go connect all of

8:45these tools, set up to feed the raw

8:47data, tell your agent that you want to

8:49build a context system into a memory

8:51system. It will do the heavy lifting for

8:54you. I'll also include a prompt to set

8:57up the memory system and a prompt to

9:00show or tell your

9:02agent exactly how I've set mine up so

9:05you can follow it step by step, [music]

9:08uh mirror it entirely.