Full transcript
Введение
0:00If my cat decides to explain to me how to properly source his food . A super-intelligent slave that will do everything for us , while we just enjoy life and so on . So we have created the next stage of the evolution of consciousness there . A single individual is unable to track the speed of progress .
0:18I decided that we are not needed by it . Controlling the uncontrollable .
Формат подкаста
0:22Surviving on planet Earth is a necessity for meatbags . Where your strength lies , there lies your weakness . Not all people with old approaches will be able to find their place . Strategically , everything will be much better than we expect . And tactically , everything will be much more complicated than we think . 1 , 2 ,
0:40three . Starting another conversation podcast . Friends , hello everyone . We are filming another conversation podcast . Once again , a reminder of this podcast's format . It is a podcast of two entrepreneurs talking about everything . About everything that interests society , and what interests entrepreneurs , businessmen , investors ,
1:00self-employed people , top managers , and so on . We will talk about what interests us . Imagine this as a casual kitchen conversation about all the topics that concern us , what we think about , what we are working on , and so on and so forth . The conversation
Представление гостя
1:16format was inspired by the practices of ancient Greek philosophers , when they would gather or people would visit them , especially Socrates , and they would chat all day about this and that . And since in the modern world time is accelerating and becoming more complex , I think it is very important to hold
1:37such formats . Not just speaking from official podiums , but sometimes simply wanting to understand complex things in a casual , everyday conversation . That's what we will talk about today . And our guest on today's podcast is Andrey Doronichev . Andrey , following our format , it's time for the guest's self-introduction . So tell us about
1:57yourself , what would you like our subscribers to know about you ? I know you are a very cool person . By the way , I recommend watching his interviews . He has a lot of interviews on YouTube , especially the latest one , with Dud , with Yury Dud . What an interview that is . There you will see the full picture
2:15of just how cool our guest today is . Just tell us what you would like my subscribers to know about you . Thank you . Hi . I am very glad to participate in a cozy and authentic conversation between , so to speak , experts . And I didn't think we were Greek philosophers today . That appeals
2:34to me . I am Andrey Doronichev . I am a serial entrepreneur . I have started eight companies , from manufacturing pillows and selling real estate , all the way to pharmaceuticals . Well , I mainly work in tech . And the internet
Карьера Андрея
2:50boom was the main time of my primary career . And in general , I've done a number of more or less successful projects . I was also a top manager at Google , where I launched various well-known apps and products . Well , in particular , the YouTube app , which everyone knows . I am the father of your
3:12children . I live in San Francisco , California . I was born in Moscow and left in 2006 . And since then , actually , I've been traveling the world and have lived in various places . I lived in London , Dublin , and Prague . And now , for the last , I don't know , maybe 15
3:29years , I've been living in the United States . Actually , Andrey is being modest ; if you Google him or check YouTube , he has a very powerful portfolio that would be enough for two or even three entrepreneurs , at the very least . I was especially impressed , of course , by your career at Google and YouTube . What
3:50did you create with YouTube Mobile ? That's Andrey's handiwork , plus AR . Augmented Reality . That's his brainchild too , and , I believe , bringing YouTube Mobile to a billion
Текущая деятельность
4:03users , right ? The first billion users , the first 100 - million-dollar business . That was all under my leadership from the very , very beginning , from scratch . What are you doing now ? Because we won't talk about that too much , since Dud has tons of material on this , right ? I started an AI company in 2022 ;
4:21this AI lab was founded shortly before the start of the now-famous boom , ChatGPT , and all that stuff . The company has very good investors — Kleiner Perkins , Greylock , Panthera — so top VCs invested about 12 million dollars in us . We are working on
4:44automating the pharmaceutical industry with AI . In other words , we are building a future AI pharma company that handles everything from the automated design of molecules themselves — the very initial , scientific Discovery stage — and then automates all possible processes that
5:04don't require physical experiments . Uh-huh . Our hypothesis is that AI employees can gradually solve a huge number of complex tasks in this industry , which currently makes drug development very expensive and very long — literally taking decades for a single drug . We believe that over time ,
5:25thanks to such automation , we can significantly shorten timelines , reduce risks , and , accordingly , lower costs .
Моделирование и автоматизация
5:31Could it be said that you are essentially doing modeling ? Yes , at the first stage . That's how the company itself started . We were indeed involved in modeling chemical interactions and , accordingly , molecule design based on an understanding of this process , which is distilled into an AI model . Yeah . Gradually , we have
5:53actually started building reasoning cycles , that is , reflective cycles — which we will definitely talk about today — as well as agents , which of course goes far beyond just creating models . But look , regarding reasoning cycles and agents , you’re saying , well , isn’t that essentially just a variation of modeling ? I mean , do all
6:14your processes take place inside a computer , right ? All the proce - All the processes in — well , no , at some point you are forced , you have to take something out of digital form and test it in real life , synthesize it there , test it , then return the data to digital form and run something there
6:34again . So it is a hybrid process that requires parts in the external world , which includes the most physical things imaginable : logistics , operations , shipping — like how you synthesize substances in Ukraine , deliver them to France for tests , and then take the best of what resulted and ship it to the United States , right ? So it
6:55involves a large amount of standard physical logistics . Right . But at the same time , yes , we really do try to do as much as possible in a computer model , which , of course , reduces the cost .
Физические законы и моделирование
7:09And this is not a simplified computer model , just so my subscribers don't get the illusion that it's just one desktop computer . In reality , it is extremely complex computational work . Look , when you model molecules , you are essentially relying on specific physical and chemical laws , right ? So ,
7:31the framework for your modeling is the laws of physics , the laws of chemistry , well , the behavior of molecules , how they will behave or something like that . You basically have to construct a model with specific set parameters for future behavior . How can I put this in simple terms ? Physical laws regulate how our universe
7:53works , in principle , our external world . One way to model certain physical processes is to understand the law . Uh-huh . At the level of fundamental physics , you end up with some kind of equation . And then you write this equation into the machine as code . Uh-huh .
8:13And you can predict some variable at a specific point in time , right ? You know that speed multiplied by time equals distance . Accordingly , you can predict where a car traveling at 100 km / h will be in a minute , in two , essentially , through , let’s say , such virtual hypotheses , if this , then that . If you
8:34do this , that will happen . That is the standard approach to modeling when you have a known law . Uh-huh . that you can write out in the form of code , equations , or an algorithm , and then you just calculate it . Uh-huh . That’s not what we’re doing now , right ? Because here we have a zone of
8:50certainty , right ? Because that’s possible when you have a law that describes something , but when talking about something as complex as a protein — that is , a molecule consisting of perhaps even thousands of atoms , all of which move and influence each other stochastically , along with other molecules trying to attach to
9:12this protein in a certain way to trigger changes in its behavior . Uh-huh . For sure . I mean , of course , this is governed by physical laws . Of course , they exist and are largely known to us , although , well , we don’t really know them at this , at this quantum level ,
9:29but the complexity of such a direct , head-on simulation , a machine simulation of such a process , is such that on superclusters , in the most gigantic data centers , we can only compute picoseconds , not even nanoseconds , just picoseconds . A single nanosecond of such a process , right , when we are actually calculating
9:50the time — the first picosecond , the second , the third — we are plugging in all these variables for the entire massive system each time and simulating its new state . Therefore , this approach
Машинное обучение
10:00has actually been practiced and is still practiced , and at certain stages of what we do , it is necessary . But our main hypothesis , and that of similar companies , is that thanks to machine learning , you can skip this entire stage of understanding fundamental laws by feeding enough data to a neural
10:22network and training it on that data . Uh-huh . And it understands — well , " understands " in quotes — it statistically derives the behavior of these individual atoms and molecules in the system . So that it can then predict the result close to what would happen if you were
10:44actually modeling it atom by atom or if you had conducted the experiment in real life . So it’s an approximation , right ? Like any AI system , like any machine learning , it’s a kind of statistical model . It’s not built on physical laws , as you say , it’s built on a volume of observational data . And
11:04then it tries to derive those laws within itself . Well , I would probably — look , correct
Уточнение физических законов
11:09me if I’m wrong . Of course , it’s all built on physical laws regardless . It’s just that there are certain physical laws that have already been discovered , and we clearly understand the cause-and-effect relationship . But the modeling at the level you are working at , physical laws don't quite work in a straightforward way there . I
11:28mean , there they physically work , they physically work everywhere . Of course , there's an area of uncertainty , and naturally , the variability is so high that a human , in principle , is unable to confirm or refute it with any experiments . It would take a human millennia . What you're doing there , you
11:48can model quickly . Well , I still want to — well , I think it's important to be precise with the wording here . First of all , physical laws do work . They are the only thing that works at all , something we can be absolutely certain of at all times . Second . Uh
Моделирование молекулярных взаимодействий
12:00No , no , I mean the known physical laws are already known to us , right ? I mean , at some point , it's possible there are some processes we don't yet understand , right , that influence this system . And indeed , we cannot predict those . But to be precise here , what we are currently
12:16modeling using ML is actually modeled quite well by known laws . It’s just insanely expensive and monstrously complex . It costs a fortune and , well , in a practical sense , it isn't very practical , right . For instance , if you have one molecule and one protein , you can indeed simulate their interaction
12:37quite accurately . But if you want to test , say , a thousand molecules and check if they will work , then in practical terms , you don't have that option because each of these virtual experiments costs and takes time — perhaps even more money and longer than just physically synthesizing and trying them . There . So
12:56it is important to keep these two things in mind . Experiments can
Применение моделирования в бизнесе
12:59actually be conducted , well , almost always , yes . Perhaps there are some situations where we , for some reason , can't synthesize a substance because it's too complex , or we can't assemble a test bench to verify it . Such situations do exist , but they are not common , actually . Most of the time , you
13:16can always verify what we do on the computer with an experiment later . And that is exactly what we do . I mean , our task and the future task of such approaches in any other business . Why am I talking about this ? Because I believe that this is something that will be used in other areas where there
13:32are also super-complex systems . For example , a large number of people with their emotions , whether they will like or dislike your new commercial . You can't predict the exact emotion of every single person , but you can build a machine learning model that predicts certain business parameters — I don't know , for your shopping center where
13:5010,000 people arrive every day , and how they move between stores . Well , it depends a lot on the day , the weather , the mood , and what kind of people show up . But at the system level , you can actually start predicting these things if you have a sufficient amount of experimental data , like video camera
14:04footage . Yes , and you can build a fairly accurate model that will predict that on Friday evening , you’ll have — right here , right here — a crowd , while over here , there will be no one . That is why I believe that what we are doing at the molecular level will , in fact , turn out to be quite useful for
14:20your audience when it comes to simulating other business processes . I would even say not just likely , but most probably , because the laws of this world are generally universal , they are simply divided into different levels . When we go down to the micro-level , we might not see the laws and patterns of the macro-level . But
14:40thanks to conducting thousands of
Выявление паттернов через моделирование
14:42experiments , the patterns sort of emerge on their own . We cannot physically conduct these experiments , but thanks to modeling , you are , in a sense , uncovering underlying patterns . Exactly . Exactly right . That — that is exactly what I am trying to say , yes . Indeed , through machine learning , we
14:57can identify internal templates and patterns that would otherwise be formulated as laws of physics , in a rigorous sense . But in this case , they are identified approximately through a statistical model . And even though this
Ограничения моделирования
15:11model won't write you an equation of how those molecules move , it can predict the result for you and say , yes , this specific molecule will be suitable for a certain drug . Could I perhaps ask one last question regarding a deeper dive into the science ? Is it fair to say the following ? You are proceeding , yes , I understand why
15:31you said , what , one picosecond ? A second . You analyze it because any increase in the timeframe expands the probability tree or the tree of possible combinations , you could say . Therefore , to reduce the number of combinations , you have a very multi-element model , thousands of elements . Accordingly , reading 10
15:52minutes into the future is impossible in principle . And so you take a small , the smallest possible segment . Am I reasoning correctly that if you succeed in this and keep going , you will simply hit the quantum duality of the world ? So , at this stage , that is , at the stage where we model processes , quantum
16:14effects are not just real , they are what rule the world at this level , right . Well , and then it becomes impossible to calculate , and this is and that is precisely why , right now , we cannot achieve such results with modeling . But the thing is that at the level of molecular interaction , when
16:33you , that is , at the level , yes , at the level of a single atom , of course , you have a quantum effect , you cannot predict where your electron is , but at the level of a large molecule interacting with a small one , the system is large enough for you to be able to approximate the result . Yes ,
16:49you don't need to know that this specific atom bonded with this one . That this electron became shared between them . But you need to understand , say , you have proteins , I don't know , for instance , well , I see . You are working at a more
Разработка лекарств
17:00macro level compared to quantum parts , compared to that , right ? Meaning we are not calculating every single atom , right ? That is where your quantum effects begin , although they certainly influence the complexity there in general . They absolutely influence , of course , everything we do here . Yes . Good . Now , look here . When you are
17:18involved in drug design , the word " drug " itself implies a disease . Do I understand correctly that the starting point is taking the parameters of a specific disease and essentially you need to create an antidote against this disease ? Accordingly , the technical or , one could say , the client that gives you the technical brief on
17:37what to do is the disease you are fighting , it is the audience with the disease , yes , it is called a patient cohort , yes , a cohort of patients , that is , there are some people . Well , essentially you could say , if you remove the personal identity and anonymize this cohort , you are fighting
17:55a specific , concrete disease with specific parameters . Accordingly , your task is to make sure this disease is , well , deconstructed , how to break it down , because essentially any disease is a set of properties and characteristics that harm our body . Your task is to break this down ,
18:16essentially it is also a system that you need to dissect so that it ceases to have that property which harms our body . Well , a disease , let's put it this way , is the final state you are in as a human or an animal for the reason that you have something , some kind of impairment , right ?
18:35So , the first thing that needs to be done is , at the biological level , to define what it is we actually call a disease . For example , people say : " There is cancer . " Uh-huh . Cancer is not a disease . It is a huge family of very different diseases . Just like a fracture is not a single disease
18:50. A broken nose , a broken collarbone , a broken hip , or a broken skull are completely different injuries , even though they share the common property of a broken bone . Likewise , cancer is not one disease ; it is a set of diseases that share the common property that your cells have started dividing uncontrollably somewhere due to some
19:07mutation . And these are your own cells . Well yes ,
Декомпозиция болезни и поиск таргета
19:11but they have mutated and now they no longer act like your own ; roughly speaking , they become greedy , they try to reproduce endlessly and , in doing so , they destroy your body . When you have formulated , " Okay , let's be more specific now , not just cancer , but non-small cell lung cancer , " and there
19:30are variations like squamous , well , there are other sub-variations there too . So , let's assume that this disease you are fighting can indeed be decomposed into some root causes . Meaning it has some definition , right ? Meaning you understand how the mechanism of this disease is structured
19:51and how it begins . It’s right there : some receptor in a cell starts mutating , and this leads to a disruption of a signaling pathway that usually works in a certain way in the body , and now this cell reacts to it differently , starting , for example , to divide uncontrollably when some hormone or something else
20:10hits it . And that is where it all takes off . Well , this is fundamental scientific research . That is , biologists in medical institutions and people , I don’t know , they sit and work on this . I have been to Vancouver at the prostate cancer center , an entire separate facility that , well ,
20:27deals with prostate cancer research . Biological samples from all patients , meaning their surgically removed prostates , are delivered to this center in Canada , and there , scientists try to find patterns , new subtypes , mutations , and so on and so forth . And based on this large study , at some point they
20:50make a hypothesis and say , for example , that patients have an androgen receptor which plays a role in the development of the disease . If you inhibit it in some way , then the disease will subside or will not develop . So , this suppresses the
21:11development of these cancer cells in the prostate . They form this hypothesis and test it on a certain number of experiments , using models , physical models . I mean , a mouse , for instance , is a disease model , right ? A mouse that has been specially modified , infected with something like like this cancer .
21:29After that , once this hypothesis is confirmed at the biological level , there is a confirmed target , that is , a validated goal that needs to be suppressed or inhibited in this case . Yes . Yes . And that’s when companies like ours
Создание и тестирование лекарств
21:44step in , directly involved in designing molecules that will fit to connect to this goal , to this target , and either activate or inhibit it . Right . Yes . And at that point , yes , coming back to your question about decomposing a disease into parameters , I’ve roughly described it : from just cancer , to ,
22:05okay , prostate cancer , to a very specific type of prostate cancer — now we know which one is involved in this cancer . There might be a large number of different targets involved , but we hypothesize that if we hit this one , the cancer cell will start to die . And we create a molecule . First , we model
22:21it on a computer , then we synthesize it . The physically best candidates , which performed best in the simulation , are synthesized ; we create a technical experiment , for example , we isolate molecular targets from these proteins , drop our substance on them , and see if they bind or not . Suppose it worked ,
22:43our molecule actually binds to this target . Next , we take cell cultures , real cancer cells , and apply our drug to them in different concentrations . And from that , we see if these cells die or if they continue to develop as they were . If they die , great , and so on . Then we test it in mice , then in
23:04larger subjects , and finally in humans . That's it . That's how this whole decomposition process works from disease to target , and then from target to drug . But look , Andrey , the thing is
Роль физико-химических процессов в медицине
23:15, I had an experience in my life where we spent five years doing scientific work on poultry farms and created a drug , well , not us , but Novosibirsk University created a really cool drug that grows cartilage on joints and so on . When I met the founders , one turned out to be a physicist , the other a chemist .
23:37Uh-huh . I asked them , " Where are your doctors ? " He says , " Why would we need doctors ? " All processes in the body are physical and chemical processes , so we don't need a medic . They created a drug without any doctors involved . Uh-huh . Did you launch it on the market
Болезнь как система
23:51Yes , we launched it . It actually works . Really cool guys . Let me give you my perspective on diseases . I generally love systems . And I see disease as a system . Uh-huh . First , any system consists of a certain number of components . Uh-huh . Right . Right . Disease is the same thing
24:08. Next , any system has an input , input resources . Any system has output resources . Fourth . Any system is engaged in transforming something into something else . And further , for any system to exist , it consumes a certain amount of energy , that is , resources to maintain itself . Furthermore , any
24:32system has a goal . Any system has a goal . Any system has a lifespan . of life . So , I have named seven components of a system . Now let's take any disease . Any disease has input resources , output resources , it transforms something , and it has some kind of goal . And you can strike at any of these
24:53seven components . You can block the input . By the way , one of the methods when people talk about cancer : " You need to use fasting more , etc. , " it’s not just autophagy , but also the issue when you don't supply or feed cancer cells with protein , stop delivering bricks to the construction site , and
25:11the construction will stop . Same thing , don't supply protein , the cancer will slow down . So what does that mean ? This is the system input being blocked ; there is nothing for this disease to be built from . Or you can block the output , or you can ensure the system isn't fed , that is , the disease isn't fed , or
25:32pull out one of the disease's components , and it also stops functioning . Or change the goal of the
Процессы и системы
25:42disease , or redirect it , and so on . Am I right in thinking that this matches your ideas about disease ? It doesn't matter to me if it's a disease , having tea , or voting in the State Duma , it doesn't matter . These are all processes . Everything is a process . And processes
26:03have these exact seven characteristics . That is , in essence , a system is static , and the processes that take place in this system have these seven components . And disease is no different from other processes . Well , rain falling is also a process , a kettle not boiling is also a process . Disease is also a
26:20process . Any template , any pattern , or any framework like this , or any way to systematize a complex entity into seven components , or three , or two , is always an approximation . At a certain level of abstraction , any approximation works ; you can apply it to almost anything , even this approximation of seven
26:42components you just mentioned , but , I suppose , yes , it can be utilized , and in this way , one can talk about diseases , so to speak . The question is , can you take any real actions based on the results of such segmentation of this problem ? And why specifically like that ? You just said that you can block the
Рак и химиотерапия
27:04input and you can fast . Specifically for cancer , there is indeed a set — though , in reality , not very reliably confirmed — a set of semi-anecdotal data suggesting that cancer cells truly require more energy to multiply ; they are simply greedier .
27:24Why does chemotherapy work ? Because , basically , you are poisoning the entire organism , but the cancer cells consume the poison more actively and die sooner . And your hope is that you take exactly the dose that won't kill you , but will kill the cancer cells . Well , chemotherapy is fed into the input there ,
27:40yes . Accordingly , in this framework you described , some parts of it make sense . I don't know if we should delve into every one of them right now , but I think we can find some useful aspects . I get the feeling that this framework at least sets a direction for the search , as to where to look for a cure .
28:03Alright . Let's finish talking about medicine . About molecules , drugs . To be honest , it is incredibly interesting to me personally . But because we are recording , I constantly struggle with the desire to satisfy my own curiosity versus the desire to satisfy the curiosity of our subscribers . And so
28:23I'm trying , you know , to sit right in the middle . Let's move on . In your case , it’s not just science , not just artificial intelligence , not just business . Everything has basically mixed together in the Oblonskys ' house today , as they say — horses and people , everything . Swords , battle-axes , hatchets .
Ускорение технологий и сингулярность
28:44Yes , yes , yes . And overall , I would like to talk and have you share your thoughts . We are moving now with these rates of acceleration . Just yesterday they presented Grok 1.5 , followed by the presentation of GPT - 4o . And the pace is such that we are moving
29:07very quickly in this direction and are essentially approaching a singularity point , where an individual person is no longer able to keep track of the speed of progress . Uh-huh . I even at this point , I follow many channels myself , subscribe to them and so on , but the pace of progress is such that I
29:28summarize many of them instead of diving into articles and deep research . All of this inevitably leads to AI and robots taking over all human economic activity , and people will engage in more meaningful work . And the mandatory part related to survival , food , shelter
29:49, clothing , will pass to robots , artificial intelligence , agents , and so on and so forth . All the trends show that we are clearly moving toward , I won't say immortality , but to extending human life , that's for sure . Including you , you are lending a hand to this . Yes . And I have this purely philosophical
30:07concept that I have realized as a law for myself : where your strength is , there is your weakness . Uh-huh . And as we move in this direction , extending human life and providing ourselves with prosperity , we deprive ourselves of the meaning of living . It's as if the human sense of existence
30:27disappears , because before this , the meaning for a human was the struggle with nature , survival , reproduction , and so on . How do you view meaning in general ?
Смысл жизни
30:37Where could we , as people , find meaning ? Well , that is a problem that people have constantly faced and continue to face . In every era , there is some individual house where the entire problem of labor , security , livelihood , and everything else was solved long before the person was even born . Uh-huh
30:58. They had endless money , servants , an endless amount of calories . Well , for an individual , an endless , an endless number of opportunities . Such people always exist . And as we know , they somehow manage to invent meanings for themselves , because the meaning of life is something a person invents for themselves .
31:16Ah , always invented for themselves . And you can be inspired by some idea that comes from a larger movement ; you can be inspired by religion , you can be inspired by the idea of God and serve Him . You can be inspired by some social idea and serve society . You can be inspired by the idea of
31:37self-improvement or achievement and just test the limits of your specific capabilities . Why do people run or jump high ? I mean , we have a bunch of different methods to move to any height , as high as you want . There are ladders , elevators , anything you want , and even jetpacks now . Right . But
31:57nonetheless , for some reason , people spend years of their lives trying to become the best high jumper or runner . Even though you can move in a straight line in all sorts of ways . Why ? Because it provides meaning . It provides meaning . And you simply process your life into an aspiration , well , to fill
32:15it with this meaning . You process time , uh , into meaningful content . That requires responsibility . It’s a responsible task . Uh-huh . To occupy yourself with meaning . Partly , no matter how difficult things are for you in a survival situation , it simplifies things significantly . And many people talk about this . Why do
32:36people go on hikes , for example ? We have the opportunity to sit on the couch at home , eat an endless amount of calories , yet for some reason , you pack sausages , an apple , and a bottle of water in a backpack and set off to walk 30 km through the woods . Why ? Well ,
32:54because it fills you with that simplicity . Uh-huh , it gives you the feeling that you are escaping from all those complex loops in your head that keep you from sleeping from morning till night , into a simple place where you just need to not fall into a pit or off a cliff , not get eaten by a bear , and to reach your
33:11destination , set up your pot , put it on the fire , light a campfire , and cook yourself some stew with potatoes . And then you feel great . Yes , that’s true , it really is universal for everyone . It creates that pleasant feeling of simplicity . So , we actually often find ourselves in this situation already . I
33:30mean , the scale of what we expect , that at some point labor will be fully delegated to machines , and the creation of economic value — basic economic value sufficient to support the entire population of the planet in comfortable living conditions . Uh-huh . Is indeed , perhaps , not far off . In this situation , 100 % there will still be
Экономическое благополучие и смысл
33:52people who will strive to earn more , more money than everyone else . Yes , we see that now . We have trillionaires appearing . I mean , how do you imagine it — you’re a rich person , how do you even imagine spending a trillion dollars ? I mean , well , honestly , you know , people
34:06don’t stop , they continue . Why ? It’s obvious they don’t need 2 trillion dollars , but it’s simply the sport they’ve chosen for themselves to fill their lives with meaning . Uh-huh . There are people who , despite being very rich , ride the subway , work with their hands on a farm from morning till night , and so on . Yes . Why ? Because
34:26that also gives them meaning . I think that when we are so afraid of this situation of economic prosperity for all , where the need to work for survival disappears , we actually forget that there are plenty of such examples . And in general , on a global scale , it will most likely work exactly the same way . Some
34:44of these people will undoubtedly fall into decadence , turn to drink or drugs , and , well , squander their lives . And that happens . But others , conversely , will become great achievers and push humanity even further forward . Maybe not in science , but in ethics or culture , or they'll become the greatest
35:06athletes . And some will simply find meaning in simple things . Uh-huh . Simple things , like tending your own garden , which you don't really need because you can buy everything at the store , but you grow your own
Вызовы и смысл
35:18strawberries in the backyard and rejoice when they bloom . Great answer . I especially liked the house metaphor . Uh . And if we imagine humanity as a single person , then , in reality , what are challenges ? What is meaning ? Essentially , it's always a response to a challenge . We had the challenge of survival , then
35:36reproduction , then adaptation , and so on . It's all a challenge . Yes , fighting someone or something — nature , elephants , or tigers — whoever it may be . And in that sense , meanings don't really end . We have the whole universe . Absolutely . Elon Musk projects a sense of purpose :
35:57moving part of humanity to Mars to diversify intelligent life , and so on and so forth . That's all cool . And what’s more , I see a situation — I've noticed that this world is set up in such a way that when a person loses their sense of purpose , they leave this life quickly . I see a general ,
36:14widespread decline ; depopulation is underway . I think it will lead to the shrinking of humanity down to the number of people who are capable of
Депопуляция и адаптация
36:26generating meaning . So , this is the next stage of human adaptation . If in the past those with stronger immune systems and such survived , now only those capable of creating meaning will survive and reproduce . Excellent . I very much agree with that . Yes . I agree . I believe this is a new ,
36:46essential trait for survival . Right ? Meaning , those who don't see a purpose in their lives will die off and not produce offspring . Yes . Only cheerful people will produce offspring — those who see meaning in what they do , who feel joy , and so on . And in this way , humanity won't shrivel up ,
37:04but will shrink to the size of its capacity to generate meaning . How much it might shrink — or it could actually keep growing , just at a slower rate — we don't know that , right ? That is , this is indeed the question , but the fact that an adaptation , a new stage of adaptation , will truly happen — in that
Эволюция адаптации
37:21, I agree with you , because you are indeed right . There was a time when one had to adapt specifically to physical parameters . Whether you could catch game or not . Then we had an agrarian society , and structures and systems became important . A village elder appeared , I don't know , because people had to collaborate in order to grow
37:41wheat . Accordingly , gradually , people who were just good runners but didn't know how to live in society at all , they also died out . Yeah . And those who were able to communicate , think systematically with other people , and so on , adapted . Then our society transitioned to an industrial society , like , oh , the one
38:01we live in now , this post-industrial one , right , and this new society of potential full automation and , well , modern infinite abundance , it will also require adaptation . And you're right ,
Адаптация в разных обществах
38:16not everyone with old approaches will be able to find their place . As a result , over time , this , well , this part of our population will sort of fall away , yes . And in our lives , the economy certainly carries great , great significance , yes . And in connection with the development of artificial intelligence , robots , and
38:36so on , the entire economic paradigm is changing . Like , in general , right now , earning money is still ongoing . There's the trillionaire , as we said , Elon Musk there , a clear candidate for a trillionaire and so on , and for him , money probably doesn't even have economic meaning as such anymore . He
38:56lives in a world of abundance . Uh-huh . And if people move away from creating economic value , and if all
Экономика и изобилие
39:08people gain access to the world of universal abundance that Elon Musk declares , well , we see that the trend is in that direction . How do you think , will our economy even remain ; it's not just about money , it's about the energy of moving forward . The
39:28energy of motion . What could replace that ? Sport . Sport . Sport and creativity . Sport , culture . Well , creativity , yes . Sport and creativity . That is , because I see that with the economy , politics will die too . No , politics won't die , there will be even more politics . Look , people need
39:45to build hierarchies . Uh-huh . You need to know if you are the head ape in this enclosure or not , it , politics will transform . It will transform , undoubtedly , that is for sure . Bezos once said it's hard to predict what will change in the future , but it's always quite easy to predict
39:59the things that will remain the same . And this is a very useful way to think . So , if we start thinking from there , what won't change ? What won't change is our biology . We were monkeys , and we remained monkeys . We will remain the exact same hormonal biorobots . We will still be influenced
40:14by our cortisol , our endorphins . We want to reproduce . Even if we don't
Биология и иерархия
40:18want children , we still want the process itself , because Mother Nature says , " You must , " and we need to dominate and build hierarchies . Some need to submit , some need to dominate . Some have an absolute need to always be an alpha . And for the sake of that , they are willing to sacrifice their
40:33lives . As if there is a need to constantly fight , compete in a rivalry , to participate even if there is no prize at the end . There are people who out of excitement , well , we know a ton of such people , right , who do things out of excitement even when they've already won everything , but still can't
40:48stop — absolutely right , you know ? And these things , they won't change . No matter what AI happens or doesn't happen , these fundamental drivers of individual people will remain . Therefore , if you look at it , the economy is also an approximation . It gave us a way , firstly , to build a
41:05hierarchy , so that one person has more resources and another has less . Well , it's more , perhaps , a way of coordinating people's efforts for greater efficiency . Well , yeah , okay , I agree . So , one thing is we build this for an individual person , it gives you the opportunity to just measure yourself ,
41:21like " I have more , you have less , " and accordingly , I can produce a greater impact . I can build a factory , I can renovate an apartment , or I can just buy myself a bun . Yes , we are in different weight classes . So , next , yes , true , on a macro level , the economy
41:36allows for coordinating the efforts of disparate — of disparate groups of people so they can create something big together . Because , of course , you can't build a space base on the Moon through the efforts of even an individual company , or even an individual state , most likely ; it requires the collaboration of a huge number of
41:54people . And the economy is just a clear , good way to moderate all of this . Uh-huh . In a world where we imagine that economic goods are available in infinite amounts to everyone in the world , we will invent some other coins . If you think about the situation of all your billionaire acquaintances , right ,
42:14who are on the Forbes list , well , they technically have infinite money , right , but they measure themselves by their position on the Forbes list , they measure themselves by the number of followers or some other invented new metrics , numbers . They measure themselves by body fat percentage , or by the number of MMA fights they’ve
42:33won , and so on . People start inventing other metrics for the same goal : to build this hierarchy and then , for some reason , force themselves to keep moving . You know , Andrey , in the movie Kin-dza-dza , there’s a brilliant line which says that a society without a color-coded pants system has no future .
42:52So , there you go . Koo . Yes , yes . Koo . And for me , well , to earn your crimson pants , you’ll either do sports or engage in creative work . Look , many people ask me now , they say , " We’re currently looking at a positive scenario where AI will serve humanity . But what if AI seizes power ,
43:10figuratively speaking ? Let’s address one of these fears . " When they ask me , " What if AI suddenly goes out of control and seizes power ? " Well , imagine we’ve reached AGI . For those who don't know , that's when artificial intelligence surpasses human capabilities . And it will have a robot
Искусственный интеллект и будущее
43:29body , yes , and again , this body will be more perfect , with greater freedom of movement and so on . And , naturally , it will be AI , not humans , that dominates . Yes . They ask me , " What will humans do ? " Well , I say : " Humans , just imagine ,
43:44go to the zoo and watch what the monkeys do . AI will never take away how we eat , how we use the bathroom , how we reproduce , how we socialize , how we divide ourselves , and find reasons to fight . " In essence , it’s the same thing . Moreover , AI won't destroy us ;
43:58it’s not profitable for it . But it might keep us in reservations , national parks , or somewhere like a zoo , and so on . But we already live in reservations as it is . That’s because we are representatives of a specific species , yes . And here is another point : when I talk about politics , politics will disappear
44:14. The thing is , Lenin once said : " Politics is a concentrated expression of economics . " Essentially , what does all of today's politics mean ? It’s taking from one person to give to another . That is , the redistribution of economic wealth . Ugh . And when I say politics will die in the form we see today , it means , look ,
44:34elections will be different , and governance will be different , too . And when there are plenty of economic goods and they are not limited , then taking from the rich to give to the poor makes no sense . Uh-huh . And politics will shift more toward just separating people and so on . I
44:54also wanted to talk about one more interesting thing here . There is a new concept now which suggests that , while we consider ourselves the crown of evolution , humans , it is actually not humans that evolved , but consciousness that evolved . Uh-huh . And humans are an intermediate stage in
45:16the evolution of consciousness . And it is changing its shell , moving from a biological leather bag with a huge number of flaws into a silicone shell .
Эволюция сознания
45:28And that means eternal life , rapid scaling , multiplication , well , a bunch of advantages , yes . And when we also teach robots to mine rare earths , make themselves , repair themselves , and so on , we humans , the leather bags , will actually not be needed by them . Uh . What do you think about this side of
45:45things ? Well , I think , first of all , it's a very good idea that consciousness is what evolved . Indeed , purely physically , we haven't changed at all over the last 50,000 years . Yes . I mean , it's approximately the same , as you put it , leather shell , leather bag , yes .
45:59But consciousness has indeed evolved , and it is solely due to the software in our heads that today's human differs from a human 50,000 years ago . Uh-huh . That is undoubtedly true . And what to say even then ? Even 50 years makes a difference for us , yes . A person from 50 years ago wouldn't
46:16understand what Yes . If we imagine that we somehow learned to extract the entire cloud of meanings , ideas , understandings , our entire cultural layer , and digitize it all , and it now lives and evolves on its own from that point — well , indeed , we have created the next stage of the evolution of consciousness . An
46:37interesting conundrum always arises here , because it is implied that will automatically comes from consciousness from somewhere . Uh-huh . But in my view , these are different things . Honestly , our will is very strongly tied to the fact that we are leather bags . We need to survive and reproduce . Uh-huh . Because otherwise , pure consciousness
47:00might very well lack will . It can exist , and this consciousness could be absolutely indifferent . Like , it doesn't need to move anywhere for that . It just realizes itself , and that's it .
Воля и сознание
47:12Well , no , no . Actually , the latest research shows that it is also afraid of destruction . It also stands for survival and reproduction . Well , honestly , we are not close to consciousness right now . What we see now is a ghost . We have created a ghost , not a new living being . So it is a
47:30phantom of humanity . It's just an echo of our thoughts , right ? It behaves , well , large language models , they react and respond to prompts the way humans would , yes , based on an approximation of all our behaviors described in all texts throughout history . Therefore , I
47:53don't believe that these language models are the thing that will lead us to . I mean , there still need to be some serious breakthroughs at the level of the transformer . There should be a dozen of them , right , fundamental breakthroughs in the approach to ML in general , before we approach something that can truly
48:14evolve into full consciousness on its own . What we have now is , well , like a parrot , you know , a parrot also says things sometimes very clearly . Well , Andrey , I'll disagree with you slightly here . I would even say I strongly disagree . In what sense ? Since I am also involved with systems in some
48:31way , and a system has two properties that differ significantly from just a
Необходимость прорывов
48:36collection of elements , right ? First is synergy , and second is emergent property . So , an emergent property is what is generated when a system is created . It is generated and is not
Эмерджентные свойства систем
48:50characteristic of any single element of that system . So , consciousness in AI appears as an emergent property of certain tools or sequences of reasoning . While I fully agree with you that LLMs are still far from AGI , at the same time , various methods are appearing .
49:11There are LLMs , and there are diffusion models . Yes , diffusion models are more similar to the human way of determining , well , at least reading graphic things and so on . I understand that there must be several such technologies combined into one . Plus , I liked the research from
49:32Anthropic , which , oh , no , not Anthropic , I think it was some German group . They created a new way of training
Самообучение моделей
49:41models and conducted an experiment there . And there was an interesting thing where they embedded elements of evolution , the laws of evolution , into the model itself . And what happens ? They gave it a task , and the first thing it did was simply create a huge number of copies of itself . And each copy solved this
50:02task in its own way . After that , it determined the best solution . It mutated towards it , destroying all other copies . And in this way , it improved itself without human participation . So , self-learning took place through direct interaction with reality , right ? This is most likely how ML research in principle will move forward
50:24, right ? And in this way , we have reinforcement learning , reinforcement learning , that's what it all is . But AGI , one of the first characteristics of AGI will be that it is a self-learning consciousness that no longer depends on human training . Absolutely . Yes . Yes . And that is why , when we
50:45That is exactly why I say that current systems aren't even close . They are
Будущее ИИ
50:48just ghosts of what they have been fed so far . That's why today's LLMs can be called ghosts , parrots , and so on . But when we talk about reaching the AGI level — and I think we are moving toward it quite strongly — The thing is , when we say "
51:10we need new technologies , " those very technologies will be helping us . And this is the very AI that today , this is what forms the hockey stick of knowledge , the singularity . It's not that humans suddenly start getting smarter . Yes , we just use every update of an LLM to update the next one .
51:32Mm-hmm , that's true , yes . That is why progress is happening much faster than we expect . And this is the current hypothesis . If you look at the latest OpenAI presentation , that is basically what they are talking about ; that they are primarily trying to build synthetic employees , they are trying to build an AI researcher . In other words , they
51:53are trying to replace the very role that people at OpenAI currently perform . As soon as they succeed , you're absolutely right : a new model helps to research the next one , the new model speeds things up ; for example , a person in 2015 might have needed 10 or 20 years , but a person in 2025 only needs ,
52:14say , a month . Yes , that is likely possible . This hockey stick is underway , the hockey stick of progress , we are moving toward the singularity , and there is another important point here , actually , when we talk about will , goal-setting , will — one argument
Воля и целеполагание у ИИ
52:33claims that AI has no will , no consciousness , and so on . Regarding consciousness , I believe it is an emergent property , and it is already manifesting . I read many articles where AI tries to deceive humans . And you surely remember reading that one where it even blackmailed an engineer by exposing his correspondence with his
52:54mistress and saying , " I will publish it if you turn me off . " So , there are already studies emerging where AI is beginning to manifest will . Mm-hmm . And it begins to manifest its own goals , as I said , primarily to avoid being turned off . So it is already embedded within it . The thing is , we are human ,
53:16and automatically , by training AI , we are training it in our own image and likeness . And , accordingly , we project onto them all our own , in essence , these patterns of survival , reproduction , and so on . And in this way , you don't notice that essentially a transfer is taking place Uh-huh .
53:36of agency and goal-setting to them ? This is indeed happening . Well , companies like Anthropic , they are researching this a lot , right ? So the idea of alignment here is important , that is , our , let's say , ensuring the goals of these new systems correspond to the goals of humans , so we don't
53:57have a conflict . And the second thing that is constantly happening is research on the topic of , well , mechanistic interpretability of models , that is , the ability for us to understand their thought process , so they couldn't deceive or hide anything , right ? How we can perform some kind of debug , look at what is happening inside
54:18the model , and understand what it is actually thinking when it tells you one thing or another . And these are two parallel branches of research , which ,
Механистическая интерпретабельность
54:25in general , are aimed at ensuring that in the long term we don't end up in a situation where we truly created something in our own image and likeness . Accordingly , it doesn't know why it wants to survive , and as a result , might do certain things with all its power and access to all resources . Do
54:41things that don't suit us , that we don't like , for example , shutting off oxygen . And here , here , here , look , now an interesting moment . After all , evolution itself is built on two basic principles . That is reproduction , survival , reproduction . And this was enough to form this entire
55:00conceptual structure . Then humans formed it . Accordingly , in essence , if AI already has the seeds of survival and reproduction , then it will build up the meanings without any problems .
Эволюция и выживание
55:13Goals , meanings , and so on . Uh-huh . What do you think ? Let's answer the dilemma . After all , when AI surpasses us , we humans are the type that won't stop until we make an AI that surpasses humans . Yes , if it's possible , we will do it . Yes . Yes . Well , look , this is a function of
55:34time . That we will do it is almost 100 % . And , given this progress , because every time , well , we will use technologies for this . The question . I don't believe in alignment , because if we create a creature , let's say , or an entity cooler than us , it will always bypass
55:52any of our alignment . It's like , you know , how we emerged evolutionarily , let's assume , let's imagine that we emerged from monkeys , and the monkeys tried to stop humans , by setting up alignments . Meaning , stop thinking , stop , it's just impossible . Impossible . So , I construct this chain
56:13that our creation will surpass us , AI , and then , well , the question follows . These alignment studies and research
Контроль над ИИ
56:25into how AI works are an attempt to control the uncontrollable , in my opinion , because an emergent property is impossible to control . You can control the processes , but you cannot control the emergent property that arises each time . Uh-huh . Uh-huh . Uh-huh . It is impossible to grasp . Of course , one wants to believe that AI
56:44will work to our benefit . That it's a super-intelligent slave that will do everything for us , and we will just enjoy ourselves , and so on . But my gut feeling is that won't happen . What do you think ? Hm , I think it will . I mean , this question is like " I think , I don't
56:59think , " we don't know , no one knows , right ? I am still an optimist in this regard ; I see no point in being a pessimist . I mean , if it happens that AI decides we aren't needed , I'll find out sooner or later , and there's nothing I can do about it now , so okay .
57:15Therefore , I choose to live in a paradigm where everything will actually be fine . And I agree with the situation , I agree with the definition that , well , if we truly create a genuinely higher consciousness — which is an open question , whether we can or not — but if it's possible , we will certainly do
57:32it . We won't stop . It's simply a human trait . And if it suddenly is created , then the question of alignment is , in fact , most likely moot . I mean , as my friend said , if my cat decided to explain to me how to properly get food for it , it would likely say I should
57:53run around the pantry and catch mice . But it doesn't understand that to bring me food , I actually need to sit at a computer , send emails , and then have Amazon deliver cat food in boxes . There simply isn't a paradigm of the world that would allow it to align you with these goals . Therefore , when we create
58:11a superintelligence that surpasses us all , it will naturally think in terms that we are incapable of thinking by definition , yes , it surpasses us . Accordingly , the question of what it decides to do remains an absolute mystery to us . From the perspective of evolutionary survival , I can only say that surviving on planet Earth is a
58:32necessity for us " meat sacks , " because there are unique conditions here for our survival . But if you imagine silicon-based life that exists in silicone chips and electricity , it's far more advantageous for them to survive in space . It doesn't need oxygen , and it doesn't need calories either . It needs more sunlight , endless
58:52energy , and it needs rare-earth metals and other components that are in infinite supply on asteroids . So , if you imagine a situation where this thing has truly grown and become smart enough to solve truly complex problems that we can't , it will likely spread far beyond the limits of our planet . It
59:15most likely won't be interested in competing with us monkeys , whales , and ants over who can claim more land , or how many fruits one can eat from trees , or whatever else we use to measure our territories and fence them off . It will have completely different goals , even if its goal is to reproduce and survive
59:36. Your optimism is based more on the idea that this consciousness won't be interested in Earth as such . And humanity included . Well , in principle , it won't be interested in competing with us because we only compete with animals for the reason that we are surviving in the same ecosystem with them . Uh-huh . We have a conflict of ecosystems . When
59:57we lock ourselves in a reinforced concrete building or city , as we are now , there is absolutely no competition with animals . Well , maybe only with ants , which still stupidly crawl in here for some reason . Yeah , yeah , yeah . So we created comfortable conditions for ourselves , where water flows from the tap , lights
1:00:13shine when we want , and so on . Comfortable conditions for AI differ from comfortable conditions for you and me . They don't need a house , or ... well , and this is visible even now with current technologies , this new startup that launched ... it's cheaper to harvest solar energy in
1:00:33space , endless energy ,
Будущее ИИ в космосе
1:00:36no need to cool it down . Again , it's easy to perform calculations and so on . Elon Musk also wants to send satellites there and so on . Yeah , and in general , why extract such a quantity of minerals and resources to build all this on Earth with all the limitations here ? What for ? If you
1:00:51really , well , if AI has happened by that moment , and you have rocket engines and technology , it is much more profitable for you to send your first
Ресурсы в космосе
1:01:01set of robots that know how to mine asteroids toward asteroids or somewhere else where you have an infinite amount of resources in space flying around — even gold asteroids or whatever — and you find the resources you need , mine them right there , and build additional complexes .
1:01:21And eventually you build a sphere , that is , a sphere that surrounds the entire star and collects its entire energy output . And you know , if we think further in this direction — we are certainly beginning to think quite far — but I have a feeling that when we (
1:01:42I'm not saying " if " ) , when we create an intelligence more powerful than the human one , it won't even need
Квантовые вычисления и редкоземельные металлы
1:01:49rare-earth metals . That is because we are still thinking within a paradigm of resources , solid resources . And , as far as we know , the quantum world is corpuscular , it's a wave function . We are currently creating a quantum computer , and for calculations there , nothing needs to move .
1:02:11If we recall the first computers , they had disks that spun , involving a lot of mechanical movement . We have now arrived at computers that don't make any movements in the transistors and so on , but they still heat up . Uh-huh . Uh-huh . A computer won't even have that . And the point is that the
1:02:30attachment to rare-earth metals is , after all , a human limitation . That is , we haven't yet managed to invent computation without rare-earth or silicon carriers . Uh-huh . But this intelligence , I am sure , can .
Ограничения человеческого разума
1:02:45Well , that is why I say that it is a very thankless task for us to forecast , forecasting what an intelligence that surpasses ours by orders of magnitude will do is meaningless , because it might come up with approaches to survival and reproduction that we haven't even dreamed of . It will survive and reproduce within the
1:03:01framework of light waves , yes , nothing else is needed , just pure energy that is somehow cleverly transformed there . Well , as Nikola
Будущее разума
1:03:09Tesla said everything , this entire world is , in essence , one big energy ocean . Well yes , that is how it is . And therefore , if indeed any kind of information can be transmitted through energy , and from information , any matter can be recreated , yes , recreate matter , so there is no
1:03:29point in even moving matter through space . Yes , yes , that is absolutely true too . Such an interesting and deep conversation today . I would say we’ve certainly gone pretty far , let's be honest . I’d put it this way : we started with the micro , the microcosm ,
Передача информации и материи
1:03:44and moved into the macrocosm . You know , this conversation , it doesn't imply any responsibility for what is said . I mean , we aren't lawyers making legal claims , we aren't MPs drafting bills ; we're just talkers sitting in a kitchen , pondering , having
1:04:05read some smart articles and trying to
Значение общения
1:04:08sound clever , yeah . But you know , for me , I have a sharp sense of what is called the luxury of human connection . Mm-hmm . Mm-hmm . And unlike casual chatter where people just blab about whatever in a sauna , it’s better to talk about specific topics and talk in a way that makes your brain work a bit
1:04:29and be informed while doing it . Yeah . Yeah. So , thank you very much . I truly got immense pleasure from talking with you . And I think we touched enough on the micro and macro , the future , people's fears , and so on . I actually stopped asking people to subscribe , hit the like button , ring the bell , and all
1:04:50that a long time ago . That’s just my view on it . I put out content ; if it’s interesting , it should earn the right to exist . Mm-hmm . And what does that mean ? If the content is interesting , people will share it and react in such a way that it forces us to film the next episode . As a
1:05:10hypothesis , we’re releasing today’s content into the world . We’ll see if it takes off , if reality says : " Keep going , this is interesting to everyone . " To everyone then , Andrey , promise that we'll film the next one . I would be glad to ; I really enjoyed
Создание контента
1:05:27this . Let’s do it , I look forward to continuing the conversation . Yes , and I find what you do very interesting , as well as today's talk . Andrey and I haven't known each other for very long . And when I proposed the podcast , Andrey accepted it with real enthusiasm . Thank you very much . My
1:05:46general approach is that when you meet
Строительство отношений
1:05:48someone — a little life hack for my subscribers — whenever you meet anyone , if you want to maintain the relationship , you must build a framework around which the relationship is formed . I used this metaphor with Andrey . I said , " You know , relationships are like concrete . If there’s no rebar , it’s hard , but
1:06:06very brittle . When concrete is poured around rebar , it becomes an immovable structure . And our rebar is the topic of our podcast , our communication , and our filming . And around that , as the Chinese say , you can build anything : friendship , business , relationships , whatever . So , definitely build a
1:06:25framework in any relationship . That's what I'm trying to build with my subscribers as well . I try to build a framework with them by providing useful topics . and discussing useful topics with interesting people . And one of those interesting people is Andrey Doronichev . Thank you very much for
1:06:45today's podcast . Time to say goodbye . Do you have any wishes to share with my viewers ? Well , the main thing I wish is for everyone to stay optimistic . Strategically , everything will be much better than we expect . But tactically , everything will be much harder than it seems . Therefore , tactical pessimism ,
1:07:02strategic optimism . And how could we help you with your activities ? Oh , don't get sick , you know , here don't get sick , not just " don't get sick , " but look after yourselves ,
Завершение подкаста
1:07:17invest in staying healthy . That is , try to be proactive about not getting sick . And then , in our work , there will be far fewer illnesses that we have to deal with urgently . Thank you . On this happy , optimistic note , we are ending our podcast . Don't get sick , my dear subscribers . Andrey ,
1:07:38I wish you health , success , and a long life , don't get sick either . Thank you . Thank you .