Full transcript
0:00I'm going to go ahead and get started
0:01and folks can join us as they're able
0:03to. I appreciate everyone joining us
0:06today for this applications for AI and
0:08SEM with agents, coaches, and process
0:11documentation. We're excited to have Dr.
0:13Jason Trager with uh CEO and the founder
0:16of plentiful.ai as our guest speaker
0:18today. Just a little bit uh about the
0:21strategic energy management
0:23collaborative. My name is Crystal Marx.
0:24I am the executive director of the SEM
0:26collaborative and I'd love to tell you
0:29about what we do. We exist to connect,
0:32learn and advance. And what that looks
0:34like is bringing people together across
0:36the SEM field. Whether that's
0:38practitioners, PAs, impleers,
0:40researchers, partners, whoever that
0:41might be. We exist to help you learn by
0:44sharing emerging practices, research
0:46tools, and lessons from across the SEM
0:48community. And work we work together to
0:51strengthen and grow strategic energy
0:53management as a field. You can learn
0:54more about us and find a bunch of great
0:56resources on the website semhub or
0:59semhub.com.
1:00That website will be changing uh as we
1:03get through the month of August to
1:05semccolaborative.org.
1:07It's not live yet, but we're excited to
1:09welcome you to that and show you more
1:11about uh what it is that we do. And
1:13we'll talk a little bit more about that
1:14as we get through the webinar. Before we
1:17get started, I just want to let you know
1:19the webinar is being recorded right now.
1:21We really hope that you will uh share
1:24this webinar with people afterwards.
1:26We're going to publish it on our YouTube
1:27page and on our website. We'll also
1:29email you a link to it as well. Please
1:32use the Q&A function throughout the
1:34presentation. We'll have time for
1:35discussion near the end. But the Q&A
1:38function again is on the bottom toolbar
1:39of your Zoom. If you can't find it where
1:41it says Q&A, click on the round symbol
1:44with the three dots where it says more
1:46and you should be able to find it there.
1:48If you see a question that you like,
1:50upvote it so that we know that you want
1:51it answered or you can feel free to
1:53comment on it as well. And we'll start
1:56with a brief three question poll here in
1:58a moment led by Emily Lang with Cascade
2:01Energy. But I want to make sure you know
2:03to stick around at the end of the Zoom
2:06webinar for a very brief survey just to
2:08let us know uh how we did and to let us
2:11know what else you'd like for future uh
2:13programming. So now I'll introduce Emily
2:15Lang, a business development lead at
2:17Cascade Energy. She is also a member of
2:19our events committee and the key
2:21organizer of today's webinar. So take it
2:23away, Emily.
2:25>> Awesome. Thank you, Crystal. Um, can
2:28everyone everyone can hear me? I see my
2:30green box lighting up, so I'm assuming
2:31that means yes. Okay. Um, yeah. So, as
2:35Crystal said, uh we're going to start
2:37today with a quick poll just to give us
2:41a sense of where everyone's at in their
2:44AI uh journey. I hate that word, but I'm
2:47going to use it anyways. Like, where how
2:49are you feeling about AI adoption? Um,
2:52and I believe Laura is going to pull it
2:54up for me. Is that right?
2:56Cool.
3:14Okay, perfect. So, hopefully I got the
3:16poll pop up on my end, so hopefully
3:18everyone else did as well. Um, three
3:20questions, multiple choice, not too not
3:23too hard. Um,
3:25uh, I guess I'll read the questions out
3:27loud. I'll go ahead and read them while
3:28you're filling out the responses. So
3:30number one is how would you describe
3:33your organization's current use of AI?
3:37Um
3:39and then number two, where do you see
3:42the greatest potential for AI in SEM?
3:49And finally, number three, when using AI
3:52tools, which type of data are you
3:54currently comfortable using? Oh, and you
3:57can select more than one. uh I believe
4:00for yeah the last two. All right. So,
4:04we'll give everyone a minute to fill
4:05those out and then we'll take a look at
4:07the results.
4:14Oh, I was just trying to fill it out
4:15myself. I don't get a chance to vote.
4:17That's okay.
4:21[clears throat]
4:45All right. Is that enough time? Laura,
4:47do you want to pull the results up? Can
4:49you see how many people?
4:50>> Yeah, we've got uh why don't we give one
4:51more minute? We've got about 2/3
4:5470% and there's a few more clicking in.
4:56So, we'll give them just 30 more
4:57seconds. I think people are filling them
4:59in.
5:0115 more seconds.
5:05Okay, I'll end it here in just five,
5:10four, three, two, one.
5:19All right.
5:21Um, so it looks like we're we are all in
5:25kind of the middle stages of our AI
5:27adoption between pretty equally spread
5:30between we're experimenting informally
5:33using AI for specific things or have
5:36some type of organizationwide initiative
5:40um
5:42all over the map on where we think there
5:44might be a good potential for AI and SEM
5:47which is probably true that it is going
5:49to be applicable able to many different
5:51aspects of an SEM program. Uh the winner
5:55here was energy data analysis and
5:56insights. And then finally using AI
6:00tools um what are we currently
6:03comfortable with? Uh publicly available
6:06information is number one uh followed by
6:09things that are not confidential.
6:12Uh which makes a lot of sense. All
6:14right. Thank you all for participating
6:16in that. Um, and uh, I don't know about
6:20all of you, but I am super burnt out on
6:23listening to AI presentations that are
6:25just like super high level and have no
6:27real world application, which is why I
6:30am very excited for uh, the presentation
6:33today. Uh, Jason Trager, if you don't
6:36know him, he's awesome. He is a a PhD
6:40from UC Berkeley who is a serial
6:43entrepreneur. He's started multiple uh
6:46startups and um he's here to walk us
6:50through what types of processes that
6:54like we are actually using in SEM could
6:56be AI ready today. Um he's going to give
6:59us a plain language framework for how to
7:02think about chat bots, agents, and
7:04coaches and then get into some of the
7:06real case studies um on work that he's
7:09actually done in the field. Um, and so
7:12with that, I'm gonna pass it to Jason.
7:15>> Sounds great. Yeah, let's go to the next
7:17slide. Um, uh, also, uh, absolute
7:21shameless plug, um, uh, I'm giving, uh,
7:25a talk next week at ASP Flex Connect,
7:29um, on flexibility in the space, and I
7:32run or I'm booting up a renewable energy
7:34themed art gallery, um, which you should
7:37talk to me about.
7:38>> Um, all right. Uh yeah, that was a good
7:41intro. Um I've been doing AI, machine
7:45learning, data science in the space for
7:47um about 16 years now. Um and today uh
7:52we're going to do a little bit talking
7:53about like the frameworks that we can
7:56use. Um and then I thought to myself,
7:59hey, how do I explain AI to a whole
8:01bunch of SCM folks? And we are going to
8:03have a treasure hunt. Um, so there is a
8:07worksheet that um did get distributed
8:11question mark um or will be distributed.
8:14>> It will be distributed when you when you
8:17get to that portion.
8:18>> Okay, great. There's going to be a
8:20worksheet. We're going to um
8:23look at um look at our organizations
8:26like we would look at um the
8:29organizations that SEM programs serve.
8:32All right, let's go to the next slide.
8:35Um
8:37uh has anyone
8:39heard someone say uh some form of let's
8:44just AIify it or can't we just use AI to
8:48improve that and stop to say well how
8:54how do we engage in that? Um
8:57we uh
9:01at plentiful you know one of the two
9:03main things we do is AI enablement and
9:06helping organizations like implement AI
9:09with discipline and almost without fail
9:12uh we come in people say here is the
9:16pile of things how do we put AI into it
9:21and we look at it we say well the
9:24documentation here is kind of thin. The
9:28process isn't aligned. How do we um
9:34how do how could we possibly me manage
9:36what we can't measure? And if we can't
9:38manage it, we can't have a robot manage
9:40it. Um and so for us, we think of this
9:43as uh documentation is the basis upon
9:46which we build the M&V of the process.
9:47We don't have a baseline, we don't have
9:49a savings claim. Um and let's keep
9:52going. I'm going to beat this uh analogy
9:55to death. Um so we're going to start
9:58with AI 101. Let's keep going.
10:02Okay. Um so AI uh did anyone and you can
10:07just raise your hand or can you raise
10:10your hands? Do an emoji of some kind. If
10:12at some point in your childhood or one
10:15of your kids' childhoods, you had that
10:18uh little 20 questions game that was
10:21usually a little red ball and it would
10:23ask you 20 yes or no questions and tell
10:25you what plant or animal you were
10:27thinking about. Um, okay. That was my
10:31favorite example of AI because it was
10:34available in like the 80s. um it seemed
10:38like magic and it uh performed a very
10:43noticeable task in the way that a human
10:45would um by breaking down a uh
10:49complex decision tree into yes or no
10:52questions. So any system that performs
10:55tasks we associate with humans is
10:57artificial intelligence but it was
10:59static it was pre-programmed. Machine
11:02learning is a type of AI where things
11:06are not pre-programmed
11:08and the machine can actually learn from
11:11examples. Um, and everything we're
11:14getting used to as modern AI is a
11:16descendant of that. Deep learning um was
11:21uh developed uh using neural networks.
11:24It's using multi-layer decision trees
11:26and generative AI is uh a lot of like
11:29the image and text generation we're
11:31familiar with right now. Um and it
11:34predicts and produces new text and
11:35images and audio from uh current
11:39examples. All right, next slide.
11:44Um
11:45uh so we're going to talk about language
11:47models uh more because like most people
11:48talk about like integrating claude or
11:50chat GBT or um like some open source
11:53model into their work. Um they're
11:55reliably good at drafting, summarizing,
11:57translating and formatting. Um answering
12:00questions from documents
12:02uh finding uh finding patterns from um
12:08uh and exceptions in task text and
12:10following written procedures step by
12:12step. Uh, it doesn't know your
12:15programming data or rules inherently.
12:17Um, it is possible to program that in.
12:21We're going to talk about that a little
12:22bit. It can't check its own numbers.
12:24LLMs are notoriously bad at math. Um,
12:27they don't stay current without being
12:29fed sources and they don't say I don't
12:31know. Um, which leads to our favorite
12:34thing of slop. Um, which we'll talk more
12:37about later. Let's go to the next slide.
12:40Okay. I think that the the three
12:43categories of AI in the space are chat
12:47bots, agents, and coaches. Um, and I
12:52think we're a little bit familiar with
12:53each of these at this moment. Um, but
12:55chat bots are uh are just like it gives
13:00you the answer. Um, there's a lot of uh
13:04this present in uh the the Google search
13:07revamp. Um, it takes a chatbot format.
13:10it it'll search and then it gives you a
13:11Q&A over the internet. Um uh which is or
13:18is not useful depending on you know if
13:20you were good at Google search before.
13:21Uh there are agents. It'll take steps on
13:24your behalf. It'll send an email. It'll
13:25schedule a call. It'll uh match up
13:28files. Uh it it typically should have
13:31guard rails and a human in the loop. And
13:34then there are coaches that will like
13:36help you out with the process um over
13:39time um with a memory and some sort of
13:43human handoff. We've been seeing a good
13:45amount of these in programs where um the
13:48the coaches act alongside human uh
13:51coaches uh in programs. Um just checking
13:56my time. Okay, let's keep on going
13:58please.
14:00Um we have red flags to design around um
14:04which are hallucination, wrong number
14:06bias, unverifiable output, human and
14:08loop and data leakage. Uh essentially
14:11hallucination
14:14LLMs want to give you an answer. They're
14:16programmed to give you an answer even if
14:18they don't know what's going on. So they
14:20just lie. Um
14:23uh the next two are like forms of
14:25hallucination. They can't verify it.
14:28it's no good to you. If it's the wrong
14:29number, it's also no good to you.
14:31They're bad at math. Um, if you don't
14:35have a human in the loop, you might want
14:37one. And you should really know where
14:39your data goes cuz if you're putting
14:41data in, it might violate somebody's
14:43terms of service. So, we need to be
14:44aware of that. All right, let's keep
14:47going.
14:49Um,
14:51okay. Uh at the heart of all good
14:54automation is good process
14:57documentation. Let's dig into what that
14:59means and how uh how we engage with it.
15:04Uh okay. Um so uh how many of you feel
15:10like you're good at treasure hunts? Um
15:13you can respond with emojis of any kind
15:16cuz I can't see your faces which is only
15:18a little weird. Um
15:21so um
15:25you know in in a treasure hunt or in um
15:30like in a mass documentation
15:33we take the operator operational
15:36controls and document procedures um in
15:38SEM this maps pretty directly to the
15:41prompts and procedure and AI runs
15:42against treasure hunts are looking
15:45around for um wasted energy um we're
15:48we're looking for wasted attention and
15:50an opportunity register maps pretty much
15:53directly to an AI backlog um uh which
15:57you will be receiving uh shortly um so
16:01let's keep on going in fact this was
16:03kind of a perfect uh analogy um
16:07you know plan do check act is a loop um
16:12I uh tried to get this as a circle uh
16:15but Claude refused so I'm Sorry. Uh
16:19[clears throat] I didn't feel like
16:20putting in enough effort to make it do
16:22it. Um so, uh
16:27okay, fun fact about me. Um I did a lot
16:32of my PhD studying, uh the work of W.
16:34Edwards Deming, um and his work in um
16:38and and process control is applied to
16:40energy efficiency and buildings, which
16:41is how I'm just generally fascinated by
16:44SEM. Um
16:47uh and plan do check act comes a lot
16:49from that and has come down and um like
16:53originally from Toyota then into agile
16:56and lean and six sigma and then into
16:58software as devops um and in uh SEM and
17:02now is in uh is in agentic tooling um
17:06because this is pretty much the loop an
17:09agent executes when engaging with it.
17:11It'll it'll it'll make a plan. It'll do
17:13things. It'll check if it worked and
17:15then if it doesn't uh if it if it um
17:21uh if if that check fails it'll act more
17:25um uh
17:28um and go back to planning um and it
17:30improves every cycle. Um so this is uh a
17:35useful loop in AI as well as all sorts
17:38of different processes. Um let's go to
17:40the next slide.
17:44Um okay. What makes a process AI ready?
17:49Um
17:50written steps and decision rules. Um uh
17:55fewer than two uh it depends uh marks.
17:59Uh digital reachable inputs. Uh this
18:02one's actually pretty key and I think
18:03something that like a lot of
18:04organizations miss is
18:07um
18:09like structured
18:12text files with routers. Uh AI tools
18:16typically
18:18um can only like me they basically can
18:22hold a magazine in memory pretty much
18:25perfectly. But the minute you have more
18:26than a magazine like a book um it gets
18:29very confused. And so reachable inputs
18:32telling it what to put into the memory
18:34every time is extraordinarily useful for
18:37controlling what comes out and matches
18:39with the process. Um and process can
18:42then get the the the the references.
18:44[clears throat]
18:45Um we want to uh do cheap verification
18:49instead of failing and handing to a
18:51customer. And we want to have reversible
18:53failures. So we want to engineer for
18:55like our failures being able to be
18:57turned around. Um, we need to know what
18:59fails and have enough volume to pay
19:01back. Um, all right, let's keep going.
19:05So, are there questions Q&A here? Um,
19:10okay. Um, also feel free to just,
19:14you know, uh, yell or put things in the
19:17Q&A. Um, I hope I prefer interaction.
19:22Um,
19:24some things, uh, fail quietly. We want
19:26to really avoid that. If something would
19:28fail quietly, we need to make its
19:30failure loud. Um, we need to
19:34uh make sure there aren't undocumented
19:36exceptions or uh have like a a low a low
19:41frequency high stakes event. Um,
19:45let's or trapped inputs or verification
19:48is slow as doing. So, let's let's go on
19:51um cuz Okay.
19:55Um,
19:56and this is uh this is an example um of
20:03uh of what we'll find on on our treasure
20:07hunt sheet. Um we have a process
20:11um which has a trigger. Um it might have
20:15um you know it might be a monthly energy
20:17report at the end of every uh every
20:20month we we do it that how much load it
20:23has. It's how many hours? So it's like
20:2512 per year times 6 hours time two
20:27people 144 hours per year. Um
20:32uh we might have partial documentation
20:36like the exceptions are in our heads. Uh
20:39the verification is faster than doing
20:41it. Um how many exceptions we get. Um
20:44this this requires like tracking how
20:46many documents go arai which is actually
20:48a part that um I think is new in the AIH
20:53is like quantitatively tracking mistakes
20:57um and attributing them to a machine as
21:01part of an improvement loop. Um and then
21:03we have you know the AI fit. Um like can
21:06we answer it uh with the human review um
21:11and then uh our estimate of savings. Um
21:15yes there we go. Um there's now uh a
21:19worksheet which I want to look at as I
21:22talk about it. Okay. We are going to do
21:24a miniature version of an AI treasure
21:27hunt
21:28um in our in our uh in our workshop
21:32here. Um and we're going to spend like a
21:35few minutes on that. Um so
21:39it's okay if we
21:42uh
21:44fake this a little. Um and by that I
21:46mean you're supposed to have a few
21:48people involved in this. Um so let's go
21:52to the um
21:55let's go to the next slide. Um I
22:00in a treasure hunt in a in a in a
22:03facility we might have different zones
22:04like the boiler room or the control uh
22:07room or um depending if it's an
22:10industrial facility, right? You might
22:12have various thermal loads and various
22:15um uh uh processes that move along a
22:19conveyor belt. Um, I don't do SEM all
22:21the time. Um, I have passing knowledge
22:24of of it. So, if any of those are
22:26inaccurate, forgive me. Um,
22:30uh, in in ours, this is a
22:34example of of an SEM company, right?
22:37like we might have data intake and
22:39cleanup as a place to look modeling and
22:42M&V reporting participant communication
22:45um actually the opportunity register uh
22:48event logist logistics and assessment uh
22:51documentation ENMS program admin
22:54knowledge retrieval and onboarding as
22:56areas we can look for a treasure hunt um
23:00and so
23:03um
23:05uh what I'd like you to do let's go to
23:07But the next slide, we're going to spend
23:106 minutes on this. Um I I'd like you to
23:13open the worksheet. Um I made this so
23:17you can keep it um and and run it. Um
23:20it's it's designed to be a 2hour
23:22situation, but I want to demo it um
23:26within for a minute. So on page four,
23:29right, there's an opportunity detail
23:31sheet. Um, and what I want you to do is
23:36like pick an opportunity ID, you know,
23:38probably one, a zone, like what area of
23:40the business it lives in, and a process
23:42name. Just like write down saying it
23:45absolutely grinds your gears that it uh
23:50it isn't automated. Um uh and then start
23:55to go through like what the trigger is,
23:57the current owner, how many times uh you
23:59do it um per year. And I'm going to
24:04be quiet for a second while we all do
24:06this. And if anyone wants to ask
24:10questions of the worksheet or can't get
24:11through to the worksheet, please tell
24:13me. Um and please share as you go
24:15through this.
24:35Okay,
24:38I got a good Q&A question here.
25:16Um,
25:20if some of this doesn't make sense,
25:21please pipe up.
26:13Uh, start with the um
26:17start with the uh the Word doc, not the
26:20Excel workbook.
26:34Sorry if that wasn't clear.
26:58Um,
27:00you know, while you're at that, um,
27:23So on the energy intensity issue um
27:28like one of the things I heard recently
27:30and I haven't verified it but I believe
27:32it um is that uh
27:38is that like video um there's something
27:42to excel in chat. Okay. Did we
27:46send around the word doc?
27:50>> Uh, we did. We dropped it. It's in there
27:52above it, but we can drop it again if
27:54folks need it. It's
27:56>> okay.
27:56>> Crystal just dropped it again.
28:14Um
28:16and
28:19uh like the energy use of AI it is
28:24largely in the in the training phase. I
28:28mean largely in the in the usage phase
28:29like spread across all of the um all of
28:33the usages of the inference. Um so
28:38like one of the things we can do is uh
28:40is decrease the the usage cost um by
28:44using appropriately sized models which
28:46are much more energy efficient. Um and
28:49so
28:52I think like any technology it'll get
28:55more efficient over time especially if
28:57we demand it.
29:00Um and but I I do think it's a
29:03trade-off. I think it's a valid
29:04question, right? Is is how do we how do
29:08we like use like how do we justify
29:13savings? And I think that one of the
29:14things is that um some forms of AI are
29:18you know I can run them on my laptop. I
29:20can run models on my laptop which can
29:22improve the usage of a chiller
29:25um and I run it once. It uses, you know,
29:28as much energy as making toast for
29:30breakfast one day and it can save 10% of
29:36the of the energy use of that chiller
29:38over a year as an example.
29:42So, I think it it matters like what type
29:44of AI and how.
29:48Okay, I think that we're approaching the
29:51six minute mark.
29:53Um
29:57maybe we
29:59go to the
30:02next slide.
30:06Okay. Um what we want to do is when I
30:08have a poll like what verdict you got
30:12for your process.
30:15Um
30:19uh
30:25there we go.
30:27Post and panelists can't vote. Thanks.
30:30Um
30:38okay.
30:40And if we could share the results on
30:42that. How many do nows? How many? Uh,
30:47my screen's acting up a bit, so maybe I
30:49can.
30:50>> Yeah, we'll give them another second.
30:51There's only about 10 out of the 35
30:53right now. We'll give people just a
30:55minute and and [clears throat] we'll
30:57>> right now we're sitting at most people
30:59are in the document first phase.
31:14Okay, we'll go ahead and end the poll
31:16here.
31:27Well, um,
31:30I kind of thought that would be the
31:32case. So, next slide.
31:40Well, uh, now we have a baseline. It is
31:43document first, which is what I
31:45predicted. Um
31:48uh if if you found a step you couldn't
31:52actually write down
31:54that process is hard to uh like make
31:58into AI. Um but if you have document
32:01first it can go into the AI opportunity
32:03register which is the worksheet which is
32:05the answer to the question of like how
32:06is that AI and so much of AI is boring
32:10process uh enablement. Um so uh I'd like
32:15to talk about um you know some case
32:19studies where we actually did a lot of
32:24um process building um and talk a little
32:27bit about context. So let's just uh walk
32:30through these. I think I got um 8
32:34minutes until Q&A. So we're on track. Um
32:38yeah, we recently did a did a project
32:40with um our friends at PSD um and
32:44they asked us to help like do AI
32:46enablement. Um and uh to their surprise
32:52um but ultimate delight we spent 4
32:54months documenting process and building
32:56process and like and like working
32:58through it and two months building AI
33:00muscle and like a uh a a harness which
33:03is a set of tools, prompts and
33:06procedures that the AI can follow. Um
33:10and they got significant business
33:13acceleration out of it. So 4 months
33:15building and then like 2 months building
33:18a harness um and and enabling folks
33:21leads to good results. Um the other way
33:24around
33:26you're going to you're going to get an
33:27explosion of
33:30like undocumented process. Uh let's go
33:33to the next slide.
33:35Um, we recently did a project with NIA.
33:39Um, and we did four months of
33:42interviews. Um, and uh, and and the goal
33:45was to make an AI standard for how
33:48homeowners,
33:50uh, interact with chatbots to upgrade
33:52their HVAC. Um, uh, four months of
33:56research led to like a simple one step
33:58that wasn't like written is a rubric,
34:02um, for proposals to make them
34:04intelligible.
34:06Um 70% of the interviewees named uh
34:10incomprehensible incomparable bids. So
34:14what I'm trying to bang home is that if
34:17you go around and interview the folks at
34:19your company and document the process,
34:20you will find the pain points and you
34:22will gather the the the the
34:25rubrics.
34:27Um
34:29all right. Uh I I want to talk about one
34:32more thing um before we get to uh Q&A.
34:36Um so let's go to the next slide.
34:39Um
34:42I think that a lot of people um
34:46forget or don't understand how context
34:48works. Um and context is a technical
34:51term for uh how much a AI can fit into
34:54its memory. Think of it like um the easy
34:57way I explain it is you can fit one
34:59magazine but not a whole book. Um and
35:04in reality it depends on like how big
35:05the magazine is or how big the book is.
35:07But like just for simple thought process
35:10just think I can fit a magazine but not
35:11a book which means you need to be
35:13selective about what you put in there.
35:15So for me in making this slide deck
35:18which I made with Claude um and I did
35:20like
35:22six versions of it before I and then I
35:24hand edit it before I got um to a place
35:27I was happy. Uh first I ran a research
35:31report in Claude on like um good
35:35templates for energy star uh and DOE
35:37toolkits um for treasure hunts um uh
35:42field field lists uh and failure modes.
35:45And I said, "All right, put that in the
35:47context." Um and then I ran a research
35:49report saying like, "Okay, what are
35:51people like liking for AI 101 teacher
35:53material?" cuz like I have my set of
35:55things that I've had over the past few
35:57years, but um and in particular I really
36:00like IBM's videos. Um they're like 8
36:03minutes long and pretty informative. Um
36:06so I ran these two research reports. I
36:08was like great, hold those in context
36:09and then let's combine them into uh
36:14into a first draft of the report. And I
36:16got a remarkably good first draft. So,
36:18like my process on this is describe
36:21something I like and describe something
36:23else I like that I want to combine with
36:24it and then smush them together. And the
36:28thing about context is that if you put a
36:30whole book in there or if you ask it to
36:31like randomly gather stuff, it's going
36:33to hallucinate and it's going to make
36:35stuff up. So, like be selective about
36:37using your expert knowledge to to put
36:40appropriate documents into context
36:41before making things. Um, uh, next
36:45slide. Um, so like why this works, um,
36:50context is basically the budget. It's
36:52it's working me memory. If you make a
36:55cold ass, the model might go out and get
36:57things, but it doesn't have your
36:58experience. It's it's very going to be
37:00very generic. If you dump a whole book
37:03in there, you're going to uh like it's
37:07it's only going to remember a few
37:09chapters and you're going to be missing
37:11things. And so you can prime your work
37:13with research reports. Um and and and
37:16the SEM version is you wouldn't build a
37:18model without an energy review, right?
37:20Don't ask a model to build something
37:22without a research base unless it's it's
37:24very easy. Um okay. Uh I am
37:28approximately on time. Let's uh um so I
37:32I've given you a kit. You can take it
37:34home. Um you can
37:36or to work or wherever you work. Uh
37:39there's a kit on the uh AI and
37:41automation uh treasure hunt kit um with
37:44the ground rules, a three-phase run
37:46agenda. If you do this at your company,
37:48you're going to get some good results.
37:51You will improve your process. We we do
37:54it's not exactly the same. I adapted it
37:56to a treasure hunt for this audience. We
37:58do this very similarly for a lot of our
38:01clients. Um, uh, you you can adjust your
38:0510 zone map. Uh, make the printable
38:07detail sheet and like go through the
38:08scoring guide. Uh, the workbook is kind
38:11of like the the the verdict per per
38:14zone. It's your opportunity register.
38:16And I made it for a program office. Um,
38:21so,
38:23uh, you know, you should expect document
38:26first to win the the verdict, which is
38:28what I've been trying to bang home this
38:29whole, uh, conversation. and I hope it
38:32sticks is that um robots aren't humans.
38:35They need more explicit instructions and
38:38we need to give them them. Um and that's
38:40good news, not bad news.
38:43So, we go to the next slide. Um
38:46uh if you've met me, you will realize
38:49you get uh either homework prizes or
38:52both at my talks. Um
38:56uh you're not here, so no uh no prizes.
39:00Sorry, only homework. Um, but I would
39:03encourage you to document one process
39:04this month. Like log it like an an
39:07opportunity register, then
39:08[clears throat] hunt, document, and
39:10automate in that order. Um, and we'll go
39:14to questions.
39:20Um,
39:22questions, complaints, discussion.
39:29Also, if you want to put a question in
39:31the Q&A uh that and you would like to
39:34speak, just go ahead and, you know, put
39:36in speak and I can unmute you and allow
39:39you to ask a question out loud to Jason
39:41if you'd like to do so. We'll wait
39:42another couple of minutes for that. And
39:45if we don't get a lot of questions,
39:46that's okay. We will uh move to the
39:49closing and a couple of follow-up items
39:51for you, but we'll give it a moment or
39:53two.
39:54Matt,
39:55>> I'm just going to call on people if you
39:56don't have questions.
40:08Here
40:21we go. We've got one from Trevor in the
40:22chat right here. Jason, if you're able
40:24to see that one.
40:26>> Yeah. Um,
40:30okay.
40:33Yes. And
40:35it depends on like what kind of what
40:38kind of information you talking about.
40:39Are you talking about like
40:42um
40:43like time series data or
40:46um like research reports? Um generally
40:53uh like if if you're if you're pulling
40:55in time series data um
40:59in in all these I would encourage an
41:01intermediate step if you're making a big
41:02report um and by an intermediate step I
41:05mean um you can have someplace where
41:10you dump files like a set of files and
41:13then it can like assemble the files and
41:15and a lot of the modern systems will
41:17manage their own context.
41:18Um, and so you can dump a CSV, you can
41:22dump a research report. Um, if you're
41:26looking to like assemble a big report
41:29and um then you can
41:32uh
41:35uh so you have the agent pull the
41:38reports, dump them there in chunks. So
41:41you like dump the CSV of the time series
41:43data, graph it and validate that. Check.
41:46All right. leave it as an artifact. Uh
41:48pull uh like an analyst interpretation
41:51out of another report, put it in there
41:53as text, validate that it looks good,
41:55check um and so on and so on and so on.
41:58So you have like 20 documents and then
42:00you say, "Okay, here's this reference
42:01pile. Assemble from that. Here's the
42:04source material if you need to uh like
42:06grab anything else, but that's secondary
42:08material." And so in this way, you're
42:10controlling your context and controlling
42:12the workflow of the agent uh assembling
42:14work.
42:23All right.
42:27I think Sam Thomas had some good
42:29questions earlier or thoughts earlier.
42:32What do you got, Sam?
42:41I just put it in the reopen section. It
42:42says for Q3. So this is from earlier um
42:45and for question three. So it depends on
42:48what AI platform we're using. So it
42:49looks like regarding the the previous
42:52poll uh which if I believe um we talked
42:56about
42:58uh when using AI tools, which types of
43:00data are you currently comfortable
43:02including?
43:05And so
43:05>> interesting
43:06>> depends on what AI platform folks are
43:08using seems to be the the answer to that
43:10one so far from Sam.
43:13Um,
43:16so
43:18I think this is actually like an area of
43:20like big
43:22movement is on how we handle
43:25confidential information using AI tools.
43:28Um, and it it comes in two flavors,
43:34three really. Um, one is check your data
43:38management policies.
43:41Um meaning like do you have an explicit
43:44agreement with the provider that they
43:46can't train on your data and if so is it
43:50okay to upload into the system. Two can
43:53you host it locally
43:55and three and I think this is like new
43:57and moving is can you build containers
44:00that anonymize your data sufficiently
44:03that they can be uh utilized in analyses
44:06by frontier tools.
44:08Um, which is a a really interesting area
44:11that that we're we're looking at.
44:18Other questions?
44:27Here's one that uh someone mentioned.
44:29Uh, the energy use is important to
44:31consider as is the scale. For example,
44:34driving to a single site can often have
44:36a higher environmental impact than an
44:38individual's annual usage of LLMs.
44:42>> Yeah, I'm with that. Um
44:47I mean
44:51I struggle every time I fly because of
44:54because of that you know it's at the end
44:57of the day
45:00like SEM work is very important because
45:02it systematizes finding energy savings
45:05at sites that um
45:09you know that drive might be worth might
45:11use more energy than the whole the whole
45:14energy uses. But like if you're like
45:15systematizing like industrial sites
45:18energy usage, you're lowering the carbon
45:20footprint for every uh element those
45:23industrial sites put out into society.
45:26So
45:29it's it's it's a struggle to to think of
45:32like how how we we do that.
45:38Any
45:42last call for any Oh, we got one here
45:44from Chad. What does success look like
45:46for SEM firms and for program
45:48administrators with respect to AI usage?
45:51That may be too broad of a question, but
45:53if you have some general wisdom to
45:55share, that's great. For example, all
45:57orgs should be doing X, some should be
45:59doing Y, and potentially a couple may be
46:02thinking about Z.
46:04>> Um, yeah. So,
46:08I think that every like
46:12like in some ways SEM is like both
46:14incredibly procedural and incredibly
46:17bespoke, right? It's like industrial
46:20plants are these weird entities that are
46:22just like, you know, custom process next
46:27to custom load next to custom output.
46:30Um, and I think that um,
46:34like
46:36I I think for SEM orgs in particular,
46:39the challenge I hear a lot of folks say
46:41is that like like getting the folks on
46:44the same page, getting the folks
46:45involved. Um, I think that
46:49um
46:51like one of the best uses of AI in my
46:54mind is like uh
46:58making
46:59making uh interactions
47:03like be more frequent and and rewarding.
47:07Um, and so like rant like having a
47:12report out to your internal champion
47:14faster with elements that are more
47:17sharable um is is is what I would say
47:20everyone should be doing like um and I
47:23think all orgs should be um embracing
47:27using like meeting transcriptions and AI
47:30inputs. Um that's that's that's me. like
47:33that might feel a little privacy uh
47:35invasive to you, but you know, privacy
47:37is dead. So, um we should probably
47:41accept that. Um and if you're not
47:44keeping your own transcript, then
47:46Microsoft is keeping it. So, you might
47:47as well use it, I think. Um I'd be happy
47:51to brainstorm with you a lot more later
47:53if you want.
47:55>> Um
47:56>> a couple more questions came in here in
47:58the last few minutes.
47:59asks, "What are your thoughts on using
48:01machine learning on BAS systems?"
48:04>> Um, I think that
48:10I assume you mean like building
48:12automation systems like
48:16um
48:18so
48:20I think all BAS systems are programmed
48:23by some individual like weird bespoke
48:25names on all the parts. So all the
48:27boxes, all the sensors, everything just
48:29has like one weird name. And like the
48:31biggest first machine learning task is
48:33naming things in a standard way. And the
48:36second machine learning task is like
48:38fault diagnostics on that. And so uh
48:42yes, I love the thought of doing that. I
48:45would encourage you to look into um uh
48:48there's a onto a brick ontology also
48:51haststack are very good data tagging
48:54schemas for that.
48:56Um, and I actually have like tons and
48:59tons of thoughts on that. So, more than
49:02uh I could answer here politely. Um,
49:06and then virtual commissioning work. Um,
49:09yeah, call me.
49:12Um, uh, I think there's a a few folks
49:15doing uh virtual commissioning work. um
49:19uh some of them uh are uh more prevalent
49:24than others, but it's it's very um
49:29like it's it's
49:31if you have AMI data, you can clearly
49:33see when there's something wrong with a
49:35building and then you can automate
49:36outreach. Um it is not uh
49:42it's not exactly a rocket science as I
49:44say um talking over a computer which is
49:47part rocket science. Um it's it's not
49:51exactly you know uh quantum field
49:54engineering. So I think that it is is
49:58definitely doable. There's going to be a
49:59lot of folks doing it. Um I love that
50:02idea and would love to talk about it a
50:05lot. Um
50:08uh but I'm not going to name any
50:10particular companies on on that. Um
50:14okay. Uh
50:19and since I don't see any more
50:20questions, I'm wondering if you all know
50:21what the wind turbine's favorite music
50:23is.
50:28Yeah, they're all big fans of heavy
50:29metal.
50:34We started with a good joke and we ended
50:36with a good or a dad joke. I'm a I'm
50:38good with either of those. Jason, thank
50:41you so much for uh for joining us and
50:43for being our guest speaker today. Um
50:46everyone, please feel free to drop your
50:48favorite emoji in to show your
50:50appreciation to Jason and to each other
50:52for engaging in a great conversation
50:54with us. There will be a brief survey
50:57when you leave uh the Zoom webinar. It
51:00would be just five questions. It'd be
51:01great if you could take that uh on the
51:04screen if you want to get your phones
51:06out and take a quick picture of a or
51:08scan a QR code. There are many ways to
51:10get involved with the SEM collaborative.
51:12If you want to dig deeper on in issues,
51:15whether that's building performance
51:16standards, growing SEM, working in K
51:19through2 groups, uh school groups,
51:22decarbonization efforts, please scan
51:25that learn about SEMC working groups QR
51:28code. If you are interested in
51:29volunteering, putting on amazing events,
51:31you want to have a say in what it is
51:33that we do for webinars or our quarterly
51:35member roundts, scan that one as well.
51:37We also have a certification committee
51:39that is going strong and looking at what
51:41is SEM coach certification look like in
51:44this field and who gets to decide what.
51:47Finally, if you are not a member of the
51:49SEM collaborative or you're unsure if
51:52your organization is a member, scan that
51:54QR code and I will be reaching out to
51:56you as well. You'll receive all of these
51:59materials. Um, this PowerPoint uh is
52:03recorded. You'll receive access to that.
52:05You have the files that we shared within
52:07the webinar, but we'll also include that
52:10as well, and that will be sent out to
52:12you. Jason, thank you again so much. And
52:14Laura and Emily for all your volunteer
52:16work on this. Everyone else, thank you
52:19so much for your time, and we're so glad
52:21that you joined us and hope you have a
52:22wonderful rest of your day. Thank you so
52:25much.