AI & ML Full Course 2026 | Complete Artificial Intelligence and Machine Learning Tutorial | Edureka

0:09Hello everyone and welcome to the AI

0:11[music] and machine learning full

0:12course. Artificial intelligence and

0:15machine learning are rapidly changing

0:17the way organization [music] analyze

0:18data, automate processes, and build

0:21intelligent applications. [music]

0:23This course provides a complete

0:25introduction to a key ideas, techniques,

0:27and tools

0:28>> [music]

0:28>> used in modern AI and machine learning.

0:31Throughout the course, you will explore

0:33how machines [music] learn from data,

0:35understand the role of algorithms and

0:37models, and see how AI systems are

0:40applied to solve real-world problems.

0:41[music]

0:42The course is designed to build your

0:44understanding step-by-step, making

0:46complex concepts [music] easier to

0:48understand. And by the end of this

0:50course, you will have a strong

0:51foundation in AI and machine learning

0:54and a clear view [music] of how these

0:56technologies are driving innovation

0:57across industries. So, before we begin,

1:00please like, share, and subscribe to

1:01Edureka's YouTube channel and hit the

1:03bell icon to stay updated on the latest

1:05tech content [music] from Edureka. Also,

1:08check out Edureka's postgraduate program

1:10in generative [music] AI and machine

1:12learning in collaboration with Illinois

1:14Tech. It offers a unique opportunity to

1:16explore [music] the cutting-edge world

1:18of generative AI and develop advanced

1:20AI-powered solutions. This program

1:23[music] covers in-demand topics

1:24including machine learning, deep

1:26learning, natural language processing,

1:28[music] prompt engineering, generative

1:30AI, LLMs, RAG, agentic AI, and much

1:33more. Learn from industry [music]

1:35experts through a curriculum built

1:37around real-world hands-on use cases

1:39designed to equip you with a practical

1:41and job-ready skills. So, check out the

1:43course link given in the description box

1:45[music] below. Now, let us get started

1:48by understanding what artificial

1:49intelligence is.

1:51AI or artificial intelligence is a

1:54branch of computer science where focused

1:56on creating systems that were performed

1:59task and that would be normally required

2:01by human intelligence. These tasks can

2:04range understanding of natural language.

2:07Secondly, recognizing patterns, then

2:10making decisions, and lastly, learning

2:12from the experiences.

2:14AI is a collaboration of ideas, methods,

2:18and knowledge where from the multiple

2:20academic disciplines work on a different

2:22problem-solving and share their

2:24knowledge to a better understanding and

2:26come up with a good solution.

2:28But, it can also be rule-based and

2:31operate under a set of rules and

2:33conditions only.

2:34So, I would like to tell you all guys a

2:37small real-time experience or an

2:39experiment done by Alan Turing to

2:42propagate and to establish the

2:44artificial intelligence.

2:46Alan Turing was the first person to

2:48conduct the sustainable research in the

2:50field that he called machine

2:52intelligence.

2:53The Turing test was conducted to explore

2:55whether machines could exhibit

2:57human-like intelligence.

2:59Proposed by Alan Turing in 1950, it

3:02involves a human evaluator communicating

3:05with both a human and a machine through

3:08a text interface.

3:09If a evaluator cannot distinguish

3:11between the two based on their

3:13responses, the machine is said to have

3:15passed the test. It serves as a

3:17benchmark of the assessing the progress

3:20of AI and discussions of the nature and

3:23intelligence and consciousness.

3:25So, this is to evolve and to establish

3:28that even machines can work as humans.

3:32And that is how it is made to bring up

3:35the machines' knowledge and the humans'

3:37knowledge into machines.

3:39There are two types of artificial

3:40intelligence.

3:42First, let's talk about weak AI. Before

3:44getting into what is weak AI, I would

3:46like to tell you with an example that is

3:49performed in a real world.

3:51I know you're all guys will be knowing

3:53about Alexa, Apple Siri, and also

3:55self-driving vehicles.

3:57So, all of these considered as in weak

4:00AI. It is also known as a narrow AI or

4:04an AI narrow intelligent that is trained

4:07by the AI and focused to perform a

4:09specific task. Weak AI drives most of

4:12the AI that surrounds by us today.

4:14Narrow might be a more apt descriptor

4:17for this type of AI as it is anything

4:19but weak.

4:20Next, let us learn about strong AI.

4:23Strong AI is made up of artificial

4:25general intelligence or artificial super

4:28intelligence.

4:30Simply, it could be told as where a

4:31machine would have a intelligence equal

4:34to humans.

4:35Where humans will track what AI needs to

4:37be done.

4:39It would be able to self-aware with the

4:41consciousness that would be this ability

4:43to solve problems, learn, and also plan

4:45for the future.

4:46Have you ever thought what does AI do at

4:49its core?

4:50I'm here to tell you what. AI is

4:52essential. It works by analyzing a lot

4:55of data

4:57to find patterns and a useful

4:58information.

4:59It even learns from this data to get

5:02better at the task over time. With this

5:04learning, AI can make decisions, predict

5:06future events, and do tasks

5:08automatically that would normally need

5:10human intelligence.

5:11This help business and other

5:13organizations work more efficiently.

5:16So, AI is about making computers smarter

5:19and more helpful in everyday life. So,

5:22let me tell you some use cases that is

5:24happening in daily uses or daily real

5:27life based.

5:28Firstly, I have taken is about

5:30cybersecurity. As it is a very important

5:33and a vital role for many platforms here

5:35after.

5:36Cybersecurity is a critical concern for

5:38individual businesses and also

5:40governments as cyber threats continue to

5:43evolve in a complexity and

5:44sophistication. It would play a very

5:47role for augmenting cybersecurity

5:49defenses.

5:50It is all based on the false positive

5:53effects that is made by the false

5:55information given by any criteria.

5:58Leading the alert of fatigue and reduced

6:01operational efficiencies.

6:03Cyber attacks can exploit

6:05vulnerabilities in AI models

6:08by invading detection and compromising

6:10security defenses.

6:12They have some private data where it is

6:15trained on the basis of incomplete data

6:18sets may produce the outcomes of private

6:21data. This will raise an ethical concern

6:24and also regulatory compliances issues.

6:27Having a very complicated problems, too.

6:29So, the next one will be your

6:31entertainment. As people know,

6:33entertainment is taking a very huge part

6:36in everyone's life.

6:37For example, it would be your social

6:39media, too.

6:40So, AI is transforming the entertainment

6:42industry by revolutionizing content

6:45creation, personalization, and audience

6:47engagement. So, this can be such as

6:50television, gaming, music, and also

6:52digital media platforms.

6:54One significant use of AI in

6:56entertainment is personalized content

6:59recommendation.

7:01So, personalized entertainment

7:02experiences are enhanced through AI

7:04driven. It is recommended through

7:06systems.

7:07They are even having some platforms like

7:10Netflix,

7:11Prime Amazon, and also Spotify

7:13that is making people engaged in a very

7:15hype as of now.

7:17These recommendations improve over time.

7:20So, I would like to conclude by telling

7:22artificial intelligence can do amazing

7:24things like analyzing data and making

7:27task easier.

7:28But, it also brings up huge question

7:31about fairness, jobs, and who controls

7:33it.

7:34We need to be careful in how we develop

7:36and use the AI.

7:37Making sure it helps everyone and

7:39doesn't cause harm at all.

7:41So, this makes easier for people to get

7:43into creativity and make rules and

7:45guidelines.

7:49>> [music]

7:52>> So, now let's get started with the first

7:53topic, which is history of artificial

7:56intelligence.

7:57The concept of AI goes back to the

7:59classical ages. Under Greek mythology,

8:02the concept of machines and mechanical

8:04men were well thought of. An example is

8:07Talos. Talos was supposedly a giant

8:10animated bronze warrior who was

8:12programmed to guard the island of Crete.

8:15Now, let's get back to the 19th century.

8:17In 1950, Alan Turing proposed the Turing

8:20test. The Turing test basically

8:22determines whether or not a computer can

8:25intelligently think like a human being.

8:27The Turing test was the first serious

8:29proposal in the philosophy of artificial

8:31intelligence.

8:331951 marked the era for game artificial

8:36intelligence. This period was called

8:38game AI because here a lot of computer

8:40scientists developed programs for

8:42checkers and for chess. However, these

8:45programs were later rewritten and redone

8:47in a better way.

8:491956 marked the most important year for

8:52artificial intelligence.

8:54During this year, John McCarthy first

8:56coined the term artificial intelligence.

8:58This was followed by the first AI

9:00laboratory, which was set up in 1959.

9:03MIT AI Lab was the first setup, which

9:06was basically dedicated to the research

9:08of AI.

9:09In 1960, the first robot was introduced

9:12to the General Motors assembly line. In

9:151961, the first AI chatbot called Eliza

9:19was introduced. In 1997, IBM's Deep Blue

9:23beats the world champion Garry Kasparov

9:25in the game of chess.

9:272005 marks for the year when an

9:29autonomous robotic car called Stanley

9:32won the DARPA Grand Challenge.

9:34In 2011, IBM's question-answering

9:37machine Watson defeated the two greatest

9:40Jeopardy champions Brad Rutter and Ken

9:42Jennings. So, that was a brief history

9:45of AI. Now guys, since the emergence of

9:47artificial intelligence in 1950s, we

9:50have seen an exponential growth in its

9:52potential. AI covers domains such as

9:55machine learning, deep learning, neural

9:57networks, natural language processing,

9:59knowledge-based expert systems, and so

10:02on.

10:02Now that you know a brief history of

10:04artificial intelligence, let's move on

10:06and understand what exactly artificial

10:08intelligence is. So, the term artificial

10:11intelligence was first coined by John

10:13McCarthy, like I mentioned earlier. He

10:15defined AI as a science and engineering

10:18of making intelligent machines. In other

10:20words, artificial intelligence can also

10:23be defined as a development of computer

10:25systems that are capable of performing

10:28tasks that require human intelligence

10:30such as decision-making, object

10:32detection, solving complex problems, and

10:34so on. So, like I mentioned, artificial

10:37intelligence helps in decision-making,

10:39solving complex problems, it performs

10:42high-level computations, and also

10:44increases the accuracy of your

10:46predictions. Right? These are the main

10:48features of AI.

10:49So, now let's understand the different

10:51stages of artificial intelligence. So,

10:54basically, when I was doing my research,

10:55I found a lot of videos and a lot of

10:58articles that stated that artificial

11:00general intelligence, artificial narrow

11:03intelligence, and artificial super

11:04intelligence are the different types of

11:07AI. If I have to be more precise with

11:09you, then artificial intelligence has

11:11three different stages. Right? The types

11:13of AI are completely different from the

11:15stages of AI. So, under the stages of

11:17artificial intelligence, we have

11:19artificial narrow intelligence,

11:21artificial general intelligence, and

11:23artificial super intelligence.

11:25So, what is artificial narrow

11:26intelligence? Artificial narrow

11:28intelligence, also known as weak AI, is

11:31a stage of artificial intelligence that

11:33involves machines that can perform only

11:36a narrowly defined set of specific

11:38tasks. Right? At this stage, the

11:40machines don't possess any thinking

11:42ability. They just perform a set of

11:44predefined functions. Examples of weak

11:47AI include Siri, Alexa, AlphaGo, Sophia,

11:50the self-driving cars, and so on. Almost

11:53all the AI-based systems that are built

11:55till this date fall under the category

11:57of weak AI or artificial narrow

11:59intelligence.

12:01Next, we have something known as

12:02artificial general intelligence.

12:04Artificial general intelligence is also

12:06known as strong AI. This stage is the

12:09evolution of artificial intelligence,

12:11wherein machines will possess the

12:13ability to think and make decisions just

12:16like human beings. There are currently

12:18no existing examples of strong AI, but

12:21it's believed that we will soon be able

12:23to create machines that are as smart as

12:25human beings. Strong AI is actually

12:28considered a threat to human existence

12:30by many scientists. This includes

12:32Stephen Hawking. Stephen Hawking quoted

12:35that the development of full artificial

12:37intelligence could spell the end of

12:40human race. Moving on to our last stage,

12:42which is artificial super intelligence.

12:45Artificial super intelligence is that

12:47stage of AI when the capability of

12:49computers will surpass human beings.

12:52Artificial super intelligence is

12:54currently seen as a hypothetical

12:56situation as depicted in movies and

12:58science fiction books. You see a lot of

13:00movies which show that machines are

13:02taking over the world. All of that is

13:04artificial super intelligence. Now, I

13:06believe that machines are not very far

13:08from reaching the stage taking into

13:10consideration our current pace.

13:12However, such systems don't currently

13:14exist, right? We don't have any machine

13:16that is capable of thinking better than

13:19a human being or reasoning in a better

13:21way than a human. Artificial super

13:23intelligence, basically any robot that

13:24is much smarter than humans. Now, moving

13:27on to the different types of artificial

13:29intelligence. Based on the functionality

13:32of AI-based systems, artificial

13:34intelligence can be categorized into

13:36four types. The first type is reactive

13:38machines AI. This type of AI includes

13:41machines that operate solely based on

13:44the present data and take into

13:46consideration only the current

13:48situation.

13:49Reactive AI machines cannot form

13:51inferences from the data to evaluate any

13:54future actions. They can perform a

13:56narrowed range of predefined tasks.

13:59An example of reactive AI is the famous

14:02IBM chess program that beat the world

14:04champion Garry Kasparov.

14:06This is one of the most impressive AI

14:08machines built so far.

14:10Next, we have limited memory AI. Now,

14:13like the name suggests, limited memory

14:15AI can make informed and improved

14:17decisions by studying the past data from

14:20its memory. So, such an AI has a

14:22short-lived or you can say a temporary

14:25memory that can be used to store past

14:27experiences and hence evaluate your

14:29future actions.

14:31Self-driving cars are limited memory AI

14:33that use the data collected in the

14:35recent past to make immediate decisions.

14:38For example, self-driving cars use

14:40sensors to identify civilians that are

14:43crossing the road. They identify any

14:45steep roads or traffic signals and they

14:48use this to make better driving

14:49decisions. This also helps in preventing

14:52any future accidents. Next, we have

14:54something known as theory of mind

14:56artificial intelligence. The theory of

14:58mind AI is a more advanced type of

15:00artificial intelligence. This category

15:03is speculated to play a very important

15:06role in psychology. This type of AI will

15:08mainly focus on emotional intelligence

15:11so that human beliefs and thoughts can

15:13be better comprehended. The theory of

15:15mind AI has not been fully developed

15:18yet, but rigorous research is happening

15:20in this area.

15:21Moving on to our last type of artificial

15:23intelligence is the self-aware

15:25artificial intelligence.

15:27So guys, let us fold hands and pray that

15:29we don't reach the state of AI where

15:32machines have their own consciousness

15:34and become self-aware. This type of AI

15:37is a little far-fetched, but in the

15:38future achieving a stage of

15:40superintelligence might be possible.

15:43Geniuses like Elon Musk and Stephen

15:45Hawking have constantly warned us about

15:47evolution of AI.

15:49So guys, let me know your thoughts in

15:50the comment section. Do you ever think

15:52we'll reach the stage of artificial

15:54superintelligence?

15:56Moving on to the last topic of today's

15:58session is the different domains or the

16:00different branches of artificial

16:02intelligence. So artificial intelligence

16:04can be used to solve real-world problems

16:06by implementing machine learning, deep

16:08learning, natural language processing,

16:10robotics, expert systems, and fuzzy

16:13logic. Now guys, these are the different

16:15domains or you can say the different

16:16branches that AI uses in order to solve

16:19any problem. Recently, AI has also been

16:22used as an application in computer

16:24vision and image processing. Right, for

16:26now let me tell you briefly about each

16:28of these domains. Machine learning is

16:30basically the science of getting

16:32machines to interpret, process, and

16:34analyze data in order to solve

16:36real-world problems. Right, under

16:38machine learning there's supervised,

16:40unsupervised, and reinforcement

16:41learning. If any of you are interested

16:43in learning about these technologies,

16:45I'll leave a link in the description

16:46box. You all can go through that

16:47content. Next, we have deep learning or

16:50neural networks. So deep learning is a

16:52process of implementing neural networks

16:54on high-dimensional data to gain

16:57insights and form solutions. It is

16:59basically the logic behind the face

17:01verification algorithm on Facebook. It

17:04is the logic behind the self-driving

17:06cars, virtual assistants like Siri and

17:08Alexa. Then we have natural language

17:10processing. Natural language processing

17:12refers to the science of drawing

17:14insights from natural human language in

17:16order to communicate with machines and

17:19grow businesses. So, an example of NLP

17:22is Twitter and Amazon. Twitter uses NLP

17:25to filter out terroristic language in

17:27their tweets. Amazon uses NLP to

17:30understand customer reviews and improve

17:32user experience. Then we have robotics.

17:35Robotics is a branch of artificial

17:37intelligence which focuses on the

17:39different branches and applications of

17:41robots. AI robots are artificial agents

17:44which act in the real world environment

17:47to produce results by taking some

17:49accountable actions.

17:51So, I'm sure all of you have heard of

17:52Sophia. Sophia the humanoid is a very

17:55good example of AI in robotics. Then we

17:58have fuzzy logic. So, fuzzy logic is a

18:00computing approach that is based on the

18:02principle of degree of truth instead of

18:05the usual modern logic that we use which

18:08is basically the Boolean logic. Fuzzy

18:10logic is used in medical fields to solve

18:12complex problems which involve decision

18:15making. It is also used in automating

18:18gear systems in your cars and all of

18:20that. Then we have expert systems. An

18:22expert system is an AI-based computer

18:24system that learns and reciprocates the

18:27decision-making ability of a human

18:29expert. Expert systems use if-then logic

18:32notions in order to solve any complex

18:34problem. They do not rely on

18:37conventional procedural programming.

18:39Expert systems are mainly used in

18:41information management. They're seen to

18:43be used in fraud detection, virus

18:46detection, also in managing medical and

18:48hospital records, and so on. So, guys,

18:50to sum it up, these were the different

18:52branches of artificial intelligence.

18:55>> [music]

19:00>> These are the term which have confused a

19:02lot of people. And if you too are one

19:04among them, let me resolve it for you.

19:07Well, artificial intelligence is a

19:09broader umbrella under which machine

19:11learning and deep learning come. You can

19:13also see in the diagram that even deep

19:15learning is a subset of machine

19:17learning. So, you can say that all three

19:19of them, the AI, the machine learning,

19:21and deep learning, are just the subset

19:24of each other. So, let's move on and

19:26understand how exactly they differ from

19:28each other. So, let's start with

19:30artificial intelligence. The term

19:32artificial intelligence was first coined

19:35in the year 1956.

19:37The concept is pretty old, but it has

19:39gained its popularity recently. But why?

19:42Well, the reason is earlier we had very

19:45small amount of data. The data we had

19:48was not enough to predict the accurate

19:50result. But now, there's a tremendous

19:52increase in the amount of data.

19:54Statistics suggest that by 2020, the

19:57accumulated volume of data will increase

20:00from 4.4 zettabytes to roughly around 44

20:03zettabytes, or 44 trillion GBs of data.

20:06Along with such enormous amount of data,

20:09now we have more advanced algorithm and

20:12high-end computing power and storage

20:14that can deal with such large amount of

20:15data.

20:16As a result, it is expected that 70% of

20:19enterprise will implement AI over the

20:21next 12 months, which is up from 40% in

20:242016 and 51% in 2017.

20:28Just for your understanding, what is AI?

20:31Well, it's nothing but a technique that

20:33enables the machine to act like humans

20:35by replicating the behavior and nature.

20:38With AI, it is possible for machine to

20:40learn from the experience. The machines

20:43adjust their responses based on new

20:45input, thereby performing human-like

20:47tasks.

20:48Artificial intelligence can be trained

20:50to accomplish specific tasks by

20:51processing large amount of data and

20:53recognizing pattern in them.

20:55You can consider that building an

20:57artificial intelligence is like building

20:59a church. The first church took

21:01generations to finish. So, most of the

21:04workers who were working on it never saw

21:06the final outcome. Those working on it

21:08took pride in their crafts, building

21:10bricks and chiseling stone that was

21:12going to be placed into the great

21:13structure. So, as AI researchers, we

21:16should think of ourselves as humble

21:18brick makers whose job is to study how

21:21to build components, example parsers,

21:23planners, or learning algorithm, or

21:25etc., anything that someday someone and

21:27somewhere will integrate into the

21:29intelligent systems. Some of the

21:31examples of artificial intelligence from

21:33our day-to-day life are Apple series,

21:36chess-playing computer, Tesla's

21:38self-driving car, and many more. These

21:40examples are based on deep learning and

21:42natural language processing.

21:44Well, this was about what is AI and how

21:46it gained its height. So, moving on

21:48ahead, let's discuss about machine

21:50learning and see what it is and why it

21:53was even introduced. Well, machine

21:55learning came into existence in the late

21:57'80s and the early '90s. But, what were

21:59the issues with the people which made

22:01the machine learning come into

22:02existence? Let us discuss them one by

22:05one.

22:05In the field of statistics, the problem

22:08was how to efficiently train large

22:10complex model. In the field of computer

22:12science and artificial intelligence, the

22:14problem was how to train more robust

22:16version of AI system. While in the case

22:18of neuroscience, problem faced by the

22:20researchers was how to design

22:22operational model of the brain.

22:24So, these were some of the issues which

22:26had the largest influence and led to the

22:28existence of the machine learning.

22:30Now, this machine learning shifted its

22:32focus from the symbolic approaches it

22:33had inherited from the AI and moved

22:36towards the methods and model it had

22:38bought from statistics and probability

22:40theory.

22:41So, let's proceed and see what exactly

22:43is machine learning. Well, machine

22:45learning is a subset of AI which enables

22:48the computer to act and make data-driven

22:50decisions to carry out a certain task.

22:52These programs or algorithms are

22:54designed in a way that they can learn

22:56and improve over time when exposed to

22:58new data. Let's see an example of

23:00machine learning. Let's say you want to

23:02create a system which tells the expected

23:04weight of a person based on its height.

23:07The first thing you do is you collect

23:08the data. Let's see, this how your data

23:10looks like. Now, each point on the graph

23:13represent one data point. To start with,

23:16we can draw a simple line to predict the

23:18weight based on the height. For example,

23:20a simple line W equal H minus 100, where

23:23W is weight in kg and H is height in cm.

23:27This line can help us to make the

23:28prediction. Our main goal is to reduce

23:31the difference between the estimated

23:32value and the actual value. So, in order

23:35to achieve it, we try to draw a straight

23:37line that fits through all these

23:39different points and minimize the error.

23:41So, our main goal is to minimize the

23:43error and make them as small as

23:45possible. Decreasing the error or the

23:47difference between the actual value and

23:49estimated value increases the

23:50performance of the model. Further on,

23:53the more data points we collect, the

23:55better our model will become. We can

23:56also improve our model by adding more

23:58variables and creating different

24:00prediction lines for them. Once the line

24:02is created, so from the next time if we

24:04feed a new data, for example, height of

24:06a person to the model, it would easily

24:08predict the data for you and it will

24:10tell you what its predicted weight could

24:12be. I hope you got a clear understanding

24:14of machine learning. So, moving on

24:16ahead, let's learn about deep learning.

24:18Now, what is deep learning? You can

24:20consider deep learning model as a rocket

24:22engine and its fuel is its huge amount

24:25of data that we feed to these

24:26algorithms.

24:27The concept of deep learning is not new.

24:30But recently, it's hype has increased

24:32and deep learning is getting more

24:33attention.

24:34This field is a particular kind of

24:36machine learning that is inspired by the

24:38functionality of our brain cells called

24:39neuron, which led to the concept of

24:42artificial neural network.

24:44It simply takes the data connection

24:45between all the artificial neurons and

24:47adjust them according to the data

24:49pattern. More neurons are added if the

24:51size of the data is large. It

24:53automatically features learning at

24:55multiple levels of abstraction, thereby

24:57allowing a system to learn complex

24:59function mapping without depending on

25:01any specific algorithm. You know what?

25:04No one actually knows what happens

25:06inside a neural network and why it works

25:08so well. So, currently you can call it

25:10as a black box. Let [snorts] us discuss

25:12some of the example of deep learning and

25:14understand it in a better way. Let me

25:16start with a simple example and explain

25:18you how things happen at a conceptual

25:21level. Let us try and understand how you

25:23recognize a square from other shapes.

25:26The first thing you do is you check

25:28whether there are four lines associated

25:30with the figure or not. Simple concept,

25:32right? If yes, we further check if they

25:35are connected and closed. Again, if yes,

25:37we finally check whether it is

25:39perpendicular and all its sides are

25:41equal. Correct? If everything fulfills,

25:44yes, it is a square.

25:46Well, it is nothing but a nested

25:47hierarchy of concepts.

25:50What we did here, we took a complex task

25:52of identifying a square in this case and

25:54broke it into simpler task. Now, this

25:56deep learning also does the same thing

25:58but at a larger scale. Let's take an

26:01example of machine which recognizes the

26:03animal. The task of the machine is to

26:05recognize whether the given image is of

26:07a cat or of a dog.

26:09What if we were asked to resolve the

26:10same issue using the concept of machine

26:12learning? What we would do? First, we

26:15would define the features such as check

26:17whether the animal has whiskers or not

26:19or check if the animal has pointed ears

26:21or not or whether its tail is straight

26:23or curved. In short, we will define the

26:25facial features and let the system

26:27identify which features are more

26:29important in classifying a particular

26:31animal. Now, when it comes to deep

26:34learning, it takes this to one step

26:35ahead. Deep learning automatically finds

26:38out the feature which are most important

26:40for classification compared to machine

26:42learning where we had to manually give

26:44out that features.

26:46By now, I guess you have understood that

26:48AI is a bigger picture and machine

26:50learning and deep learning are its

26:51subpart. So, let's move on and focus our

26:53discussion on machine learning and deep

26:55learning.

26:56The easiest way to understand the

26:58difference between the machine learning

26:59and deep learning is to know that deep

27:01learning is machine learning. More

27:03specifically, it is the next evolution

27:05of machine learning. Let's take few

27:07important parameter and compare machine

27:09learning with deep learning. So,

27:11starting with data dependencies. The

27:13most important difference between deep

27:15learning and machine learning is its

27:17performance as the volume of the data

27:19gets increased. From the below graph,

27:21you can see that when the size of the

27:23data is small, deep learning algorithm

27:25doesn't perform that well. But, why?

27:28Well, this is because deep learning

27:30algorithm needs a large amount of data

27:32to understand it perfectly.

27:34On the other hand, the machine learning

27:36algorithm can easily work with smaller

27:38data set. Fine?

27:40Next comes the hardware dependencies.

27:42Deep learning algorithms are heavily

27:44dependent on high-end machines, while

27:46the machine learning algorithm can work

27:48on low-end machines as well.

27:50This is because the requirement of deep

27:52learning algorithm include GPUs, which

27:55is an integral part of its working.

27:57The deep learning algorithm require GPUs

27:59as they do a large amount of matrix

28:01multiplication operations, and these

28:03operations can only be efficiently

28:06optimized using a GPU as it is built for

28:09this purpose only.

28:10Our third parameter will be feature

28:12engineering. Well, feature engineering

28:15is a process of putting the domain

28:17knowledge to reduce the complexity of

28:19the data and make patterns more visible

28:21to learning algorithms.

28:23This process is difficult and expensive

28:25in terms of time and expertise.

28:28In case of machine learning, most of the

28:29features are needed to be identified by

28:31an expert and then hand coded as per the

28:34domain and the data type. For example,

28:37the features can be a pixel value,

28:38shapes, texture, position, orientation,

28:41or anything. Fine? The performance of

28:44most of the machine learning algorithm

28:46depends on how accurately the features

28:48are identified and extracted.

28:50Whereas in case of deep learning

28:52algorithms, it try to learn high-level

28:54features from the data. This is a very

28:56distinctive part of deep learning, which

28:57makes it way ahead of traditional

28:59machine learning.

29:01Deep learning reduces the task of

29:03developing new feature extractor for

29:04every problem. Like in the case of CNN

29:07algorithm, it first try to learn the

29:09low-level features of the image, such as

29:11edges and lines, and then it proceeds to

29:13the parts of faces of people, and then

29:16finally to the high-level representation

29:17of the face. I hope the things are

29:19getting clear to you.

29:21So, let's move on ahead and see the next

29:23parameter. So, our next parameter is

29:25problem-solving approach.

29:27When we are solving a problem using

29:29traditional machine learning algorithm,

29:30it is generally recommended that we

29:33first break down the problem into

29:34different sub parts, solve them

29:36individually, and then finally combine

29:38them to get the desired result. This is

29:41how the machine learning algorithm

29:42handles the problem. On the other hand,

29:45the deep learning algorithm solves the

29:46problem from end to end.

29:48Let's take an example to understand

29:50this.

29:51Suppose you have a task of multiple

29:52object detection, and your task is to

29:54identify what is the object and where it

29:57is present in the image. So, let's see

29:59and compare how will you tackle this

30:01issue using the concept of machine

30:03learning and deep learning.

30:04Starting with machine learning, in a

30:06typical machine learning approach, you

30:08would first divide the problem into two

30:10step. First, object detection and then

30:13object recognition. First of all, you'd

30:16use a bounding box detection algorithm

30:18like GrabCut for example,

30:20to scan through the image and find out

30:22all the possible objects. Now, once the

30:25objects are recognized, you'd use object

30:27recognition algorithm like SVM with HOG,

30:31to recognize relevant objects.

30:33Now, finally when you combine the

30:35result, you would be able to identify

30:37what is the object and where it is

30:38present in the image.

30:40On the other hand, in deep learning

30:42approach, you would do the process from

30:44end to end. For example, in a YOLO net,

30:46which is a type of deep learning

30:48algorithm, you would pass an image and

30:50it would give out the location along

30:52with the name of the object. Now, let's

30:54move on to our fifth comparison

30:56parameter.

30:57It's execution time.

30:59Usually, a deep learning algorithm takes

31:01a long time to train. This is because

31:03there are so many parameter in a deep

31:05learning algorithm that makes the

31:06training longer than usual. The training

31:09might even last for 2 weeks or more than

31:11that if you're training completely from

31:13the scratch. Whereas in the case of

31:15machine learning, it relatively takes

31:17much less time to train, ranging from a

31:19few weeks to few hours.

31:21Now, the execution time is completely

31:23reversed when it comes to the testing of

31:25data. During testing, the deep learning

31:28algorithm takes much less time to run.

31:30Whereas if you compare it with a KNN

31:32algorithm, which is a type of machine

31:33learning algorithm, the test time

31:35increases as the size of the data

31:36increase.

31:38Last but not the least, we have

31:39interpretability as a factor for

31:41comparison of machine learning and deep

31:43learning. This factor is the main reason

31:46why deep learning is still thought 10

31:48times before anyone uses it in the

31:50industry. Let's take an example. Suppose

31:53we use deep learning to give automated

31:56scoring to essays. The performance it

31:58gives in scoring is quite excellent and

32:00is near to the human performance. But

32:02there's an issue with it. It does not

32:04reveal why it has given that score.

32:06Indeed, mathematically it is possible to

32:09find out that which node of a deep

32:11neural network were activated, but we

32:13don't know what the neurons are supposed

32:15to model and what these layers of neuron

32:17were doing collectively.

32:19So, we failed to interpret the result.

32:21On the other hand, machine learning

32:22algorithm like decision tree gives us a

32:25crisp rule for why it chose and what it

32:27chose. So, it is particularly easy to

32:30interpret the reasoning behind it.

32:32Therefore, the algorithms like decision

32:33tree and linear or logistic regression

32:36are primarily used in industry for

32:38interpretability.

32:45So, why exactly are we using Python for

32:47artificial intelligence? Why aren't we

32:49using any other language? Right? Now,

32:52there are a couple of reasons as to why

32:54Python is so popular when it comes to

32:56AI, machine learning, and deep learning.

32:58The first reason is less coding is

33:00required. Now, artificial intelligence

33:02has a lot of algorithms. If you have to

33:04implement AI in any code or in any

33:07problem, then there are going to be tons

33:09and tons of machine learning algorithms

33:11involved, deep learning algorithms

33:13involved, right? Now, testing all of

33:15these can become a very tiresome task.

33:18That's where Python usually comes in

33:20handy. Now, the language has something

33:23known as check as you code methodology,

33:26which eases the process of testing,

33:28right? You can check your program as you

33:30code it. Basically, as you're typing

33:32each sentence, your errors or your any

33:34sort of mistakes in your code will be

33:36given to you. Right? So, testing becomes

33:38much easier when it comes to Python.

33:40The next important reason why we're

33:42choosing Python is it has support for

33:44pre-built libraries. Right? Python is

33:47very convenient for AI developers

33:50because all of the algorithms, machine

33:52learning algorithms, and deep learning

33:53algorithms are already predefined in

33:56libraries, right? So, you don't have to

33:57actually sit down and code each and

33:59every algorithm. That would take a lot

34:01of time. And that's a very

34:03time-consuming task. And thanks to

34:05Python, you don't have to do that

34:06because they have libraries and packages

34:09that have all the algorithms built in

34:11them, right? So, if you want to run any

34:14algorithm, all you have to do is you

34:15have to call the function and load the

34:17library. That's all. It's as simple as

34:19that. Now, the next reason is ease of

34:21learning. So, guys, Python is actually

34:24the most simplest programming language,

34:26right? If you ask me, I think it is is

34:28the most easiest programming language.

34:30It's very similar to English language,

34:32right? If you read a couple of lines in

34:34Python, you'll understand what exactly

34:36the code is doing. It has a very simple

34:38syntax, and this simple syntax can be

34:41implemented to solve simple problems

34:43like addition of two strings, and it can

34:45also be used to solve complex problems

34:48like building machine learning models

34:50and deep learning models. So, ease of

34:52learning is a major factor when it comes

34:54to why Python is chosen for artificial

34:57intelligence, right? Next, we have

34:59platform independent.

35:01So, good thing about Python is that you

35:02can get your project running on

35:04different operating systems, right? And

35:07what happens when you transfer your code

35:09from one operating system to another

35:11operating system is we find a lot of

35:13dependency issues. To solve that, Python

35:16has a couple of packages such as there

35:18is a package known as PyInstaller,

35:20right? This PyInstaller will take care

35:22of all the dependency issues when you're

35:24transferring your code from one platform

35:26to the other platform. So, all of this

35:29support is provided by Python. The last

35:31reason is massive community support.

35:33This is a very important point because

35:36it is important that you have a large

35:38community that will help you out with

35:40any errors or with any sort of problems

35:43in your code, right? So, Python has

35:46several communities and several forums

35:48and groups on Facebook. So, if you have

35:50any doubts regarding any error, you can

35:53just post those errors in these groups,

35:55and you'll have like a bunch of people

35:56helping you out. Right? So, guys, these

35:59are a couple of reasons as to why Python

36:01is chosen for artificial intelligence.

36:03It's actually considered the most

36:04popular and the most used language for

36:07data science, AI, machine learning, and

36:09deep learning. To prove that to you,

36:11here is a stat from Stack Overflow.

36:13Stack Overflow recently stated that

36:16Python is the fastest growing

36:17programming language. If you look at the

36:19graph, you can see that it has taken

36:21over JavaScript and Java and C#, C++,

36:25and PHP, right? So, Python is actually

36:28growing at an exponential rate,

36:30especially when it comes to data science

36:32and artificial intelligence. A lot of

36:34developers are very comfortable with the

36:35Python language because, you know, it's

36:37a general-purpose language, first of

36:38all. So, most of the developers are

36:40already aware of Python. And then, using

36:43the same language in order to solve

36:45complex problems like artificial

36:47intelligence, machine learning, and deep

36:49learning is something every developer

36:51wants, right? They want a simple

36:52language in order to code all the

36:54complex algorithms or the complex

36:56models. Right? So, that's why Python is

36:59the best choice for artificial

37:00intelligence.

37:02For those of you who are not aware of

37:03Python programming and don't know much

37:05about Python, I'm going to leave a

37:07couple of links in the description box.

37:09Right? You can go through those links

37:11and study a little bit more about how

37:12Python works or how the coding part

37:15works. Right? I'm going to be focusing

37:17mainly on artificial intelligence, and

37:19I'll be showing you a lot of demos. So,

37:21those of you are not aware of Python,

37:23make sure you check the description box.

37:25Right?

37:26Next, I'm going to discuss the different

37:28Python packages for artificial

37:30intelligence. Now, these are the

37:31packages that are specifically for

37:33machine learning, deep learning, natural

37:35language processing, and so on. So,

37:37let's take a look at all these packages.

37:39So, first, we have TensorFlow. If you

37:42are currently working on a machine

37:44learning project in Python, then you

37:46must have heard of this popular

37:48open-source library known as TensorFlow.

37:51Right? This library was developed by

37:52Google in collaboration with Brain team.

37:55TensorFlow is used in almost every

37:57Google application for machine learning.

37:59Now, let me just discuss a few features

38:01of TensorFlow. It has a responsive

38:03construct, meaning that with TensorFlow,

38:06we can easily visualize each and every

38:08part of the graph, which is not an

38:10option when you're using other packages

38:12such as NumPy or scikit. Right? Another

38:15feature is that it's very flexible. Now,

38:17one of the most important TensorFlow

38:19features is that it is flexible in

38:22operability. Meaning that it has

38:24modularity and the parts of which you

38:27want to make standalone, it offers you

38:29that option. Right? It's very flexible

38:31in that way. It'll give you exactly what

38:33you want. Now, good feature about

38:35TensorFlow is that you can train it on

38:37both CPU and GPU. Right? So, for

38:39distributed computing, you can have both

38:42these options. Also, it supports

38:44parallel neural network training. So,

38:46TensorFlow offers pipelining in the

38:49sense that you can train multiple neural

38:51networks and multiple GPUs, which makes

38:54the models very efficient on any

38:56large-scale system. Right? So, parallel

38:58neural network training is supported by

39:00TensorFlow. Right? This is one of the

39:03most important features of TensorFlow.

39:05Apart from this, it has a very large

39:07community. And needless to say, if it

39:09has been developed by Google, then

39:11there's already a large team of software

39:13engineers who work on stability,

39:16improvements, and all of that. Right?

39:19The next library I'm going to talk about

39:20is scikit-learn.

39:22Now, scikit-learn is a Python library

39:24that is associated with NumPy and SciPy.

39:26Right? That's why it has the name

39:28scikit-learn. Now, this is considered to

39:30be one of the best uh libraries for

39:32working with complex data. And there are

39:34a lot of changes that are being made in

39:36this library. And one modification is

39:39the cross-validation feature, which

39:41provides the ability to use more than

39:43one metric. Right? Cross-validation is

39:46one of the most important and one of the

39:47most easiest methods for checking the

39:50accuracy of a model. Right? So,

39:51cross-validation is being implemented in

39:53scikit-learn. And apart from that,

39:56again, there are a large spread of

39:57algorithms that you can implement by

39:59using scikit-learn. Right? These include

40:01unsupervised learning algorithms,

40:03starting from clustering, factor

40:05analysis, principal component analysis,

40:08to all the unsupervised neural networks.

40:11Scikit-learn is also very essential uh

40:13for feature extracting in images and

40:16text.

40:17So, mainly scikit-learn is used for

40:19implementing all the standard machine

40:21learning and data mining tasks like

40:24reducing dimensionality, classification,

40:26regression, clustering, and model

40:28selection. Next up, we have NumPy. Now,

40:30NumPy is considered as one of the most

40:33popular machine learning libraries in

40:35Python. Now, let me tell you that

40:36TensorFlow and other libraries, they

40:38make use of NumPy internally for

40:41performing multiple operations on

40:43tensors. The most important feature of

40:47NumPy is the array interface. It

40:49supports multi-dimensional arrays.

40:51Right? That's one of the most important

40:53features of NumPy. Another feature is uh

40:55it makes complex mathematical

40:57implementations very simple. Right? It's

41:00mainly known for computing mathematical

41:03data. So, NumPy is a package that you

41:05should be using for any sort of

41:07statistical analysis or data analysis

41:10that involves a lot of math. Apart from

41:12that, it makes coding very easy and

41:15grasping the concept is extremely easy

41:16with NumPy. Now, NumPy is mainly used

41:20for expressing images, sound waves, and

41:22other mathematical computations.

41:24All right? Moving on to our next

41:26library, we have Theano. Theano is a

41:29computational framework which is used

41:32for computing multi-dimensional arrays.

41:34Right? Theano actually works very

41:36similar to TensorFlow, but the only

41:38drawback is that you can't fit Theano

41:41into production environments. But apart

41:43from that, Theano allows you to define,

41:46optimize, and evaluate mathematical

41:48expressions that involve

41:49multi-dimensional arrays. Right? This is

41:52another library that lets you implement

41:54multi-dimensional arrays. Features of

41:56Theano include tight integration with

41:58NumPy. An advantage of Theano is that

42:01you can easily implement NumPy arrays in

42:03Theano. Right? That's why there's a

42:05connection between Theano and NumPy

42:07because both of them effectively use

42:09multi-dimensional arrays. Transparent

42:11use of GPU. Now, performing data

42:14intensive computations are much faster

42:16when it comes to uh Theano because of

42:18its use of GPU, right? Theano also lets

42:21you detect and diagnose multiple types

42:24of errors and any sort of ambiguity in

42:27the model. So, guys, Theano was actually

42:29designed to handle the types of

42:31computations required for large neural

42:34network algorithms, right? It was mainly

42:36built for deep learning and neural

42:38networks. It was one of the first

42:41libraries of its kind and it is

42:43considered as an industry standard for

42:46deep learning research and development.

42:48Theano is being used in multiple neural

42:50networks projects and the popularity of

42:52Theano is only going to grow with time,

42:54right? A lot of people actually haven't

42:56heard of Theano, but let me tell you

42:58that this is one of the best ways to

42:59implement deep learning and neural

43:01network models.

43:03Moving on, uh we have Keras. Now, Keras

43:05is considered to be the most popular

43:08Python package. It provides some of the

43:10best functionalities for compiling

43:12models, processing your data sets, and

43:15visualizing graphs. It is also popular

43:17in the implementation of neural

43:19networks, right? It is considered to be

43:21the simplest package uh with which you

43:23can implement neural networks. In fact,

43:25in our today's demo for deep learning,

43:27we'll be implementing Keras in order to

43:29understand how neural networks work. Few

43:32of the features of Keras include that it

43:34runs very smoothly on both CPU and GPU.

43:37It supports almost all the models of the

43:40neural network, right? From fully

43:42connected, convolutional, pooling,

43:44recurrent, embedding, all of these

43:46models are supported by Keras.

43:48And not only that, you can combine these

43:50models to build more complex models.

43:53Keras is completely Python-based, which

43:55makes it very easy to debug and explore,

43:58right? Since Python has a huge community

44:00of followers, it's very simple in order

44:03to debug any sort of error that you find

44:05while implementing Keras. So, the

44:07libraries that I discussed so far were

44:09dedicated to machine learning and deep

44:11learning. For natural language

44:13processing, we have the most famous

44:15library known as the Natural Language

44:17Toolkit, which is an open-source Python

44:19library, mainly used for natural

44:21language processing, text analysis, and

44:24text mining. The main features include

44:26that it studies and analyzes natural

44:28language text in order to draw useful

44:31information from all this natural

44:32language text. It performs text analysis

44:36and sentimental analysis by performing

44:38tasks such as stemming, lemmatization,

44:41tokenization, and so on. Now, don't

44:44worry if you don't know what any of

44:45those terms mean. I'll be discussing all

44:47of those terms with you by the end of

44:49today's session. So guys, these were a

44:51couple of Python-based libraries, which

44:53are very essential for implementing

44:55machine learning and deep learning and

44:58artificial intelligence when you're

45:00using Python, right? These libraries are

45:02perfect for implementing AI. So guys, if

45:05any of you have any doubts regarding the

45:07libraries or if you want to learn more

45:09about the libraries, I will leave a

45:11couple of links in the description box.

45:13You can go through those videos as well.

45:15So now, let's move on to the main topic

45:17of discussion, which is artificial

45:18intelligence.

45:20Now, before we get started with the

45:22demand of artificial intelligence, let

45:25me tell you that AI was invented long

45:27ago. AI goes back to the 19th century.

45:29It was not something that was recently

45:31invented, even though AI has recently

45:33gained a lot of popularity. We can say

45:36that in the past decade, AI has gained

45:39the maximum popularity. But, it was

45:41actually invented in the 19th century.

45:44Now, especially in the year 1950, there

45:46was somebody known as Alan Turing. I'm

45:48sure a lot of you have heard about the

45:50Turing test. The Turing test is

45:52basically used to determine whether or

45:54not a machine is artificially

45:57intelligent, meaning that whether a

45:59machine can think intelligently like a

46:01human being.

46:02Right? This was the first proposition

46:04and this was one of the most important

46:07breakthroughs in artificial

46:08intelligence. Right? Somebody known as

46:10Alan Turing, he published a landmark

46:13paper in which he speculated about the

46:15possibility of creating machines that

46:17think. Right? So, the Turing test was

46:20the first serious proposal in the

46:22philosophy of artificial intelligence.

46:24This was done in 1950. Right? After

46:27this, we had eras of AI. We had the game

46:30AI which was in 1951. Now, since the

46:33emergence of AI in 1950s, we have seen

46:37an exponential growth in its potential.

46:39Right? AI covers domains like machine

46:41learning, deep learning, neural

46:43networks, natural language processing,

46:45knowledge base, and so on. It's also

46:47made its way into computer vision and

46:49image processing. But, the question is

46:51if AI has been here for over half a

46:54century,

46:55why has it suddenly gained so much

46:57importance? Right? Why are we talking

47:00about artificial intelligence now? The

47:02main reasons for the vast popularity of

47:04AI are the following. Right? The first

47:06reason is more computational power. Now,

47:09AI requires a lot of computing power.

47:12Recently, many advances have been made

47:14and complex deep learning models can be

47:16deployed. And one of the greatest

47:18technology that made this possible are

47:20GPUs. Since the invention of GPUs, we

47:23can compute much more with our

47:25computers. Initially, we could barely

47:27process 1 GB of data. Right? We only had

47:30hard disk to store additional memory and

47:32all of that. Now, our computers can

47:34process tons and tons of data. So, now

47:37we have more computational power, which

47:39is one of the main reasons behind why AI

47:41became so popular. So, by having more

47:44computational power, it becomes much

47:46easier to implement artificial

47:47intelligence. Next reason is more data.

47:50Now, big data is one of the most

47:52important reasons behind the development

47:55of artificial intelligence. Now, AI and

47:58data science and machine learning, deep

48:00learning, all of these processes are

48:03here only because we have a lot of data

48:05at present. Now, the main idea behind

48:08all these technologies is to draw useful

48:10insights from data. Now, since we start

48:12generating a lot of data, we need to

48:14find a method that can process this much

48:17data and draw useful insights from data

48:20such that it benefits an organization or

48:23it grows a business. That's why

48:25artificial intelligence and machine

48:27learning comes into the picture. Right?

48:28So, more data led to the demand of

48:31artificial intelligence. Apart from

48:33this, we also have better algorithms

48:35now, right? We have state-of-the-art

48:37algorithms. Most of them are based on

48:40the idea of neural networks and these

48:42are constantly getting better. Neural

48:43networks are actually one of the most

48:45significant discoveries in artificial

48:48intelligence because with neural

48:50networks, you can take in thousand

48:52layers of input data. Right? You can

48:54take in a lot of input data to perform

48:56computations. So, through neural

48:58networks, we are actually able to solve

49:00a lot of problems including healthcare

49:02problems, fraud detection problems, and

49:04so on. Another reason is broad

49:06investment. So, our universities and

49:09governments and startups and any tech

49:12giants like Google, Amazon, and

49:14Facebook, they are all investing heavily

49:16in artificial intelligence, which also

49:18led to the demand of AI. So, AI is

49:21rapidly growing both as a field of study

49:23and also as an economy. Right? It's

49:26adding a lot to the economy and I think

49:29this is the perfect time for you to get

49:30into the field of artificial

49:31intelligence because right now AI is in

49:34a really high demand. AI, machine

49:36learning, data science, all of this are

49:38of really high demand at present. All

49:41right. So, this is the perfect time for

49:42you to get started with artificial

49:43intelligence. Now, let me tell you that

49:45the term artificial intelligence was

49:47first coined in the year 1956

49:50by a scientist known as John McCarthy.

49:53Now, John McCarthy defined artificial

49:56intelligence as the science and

49:57engineering of making intelligent

50:00machines. So, now let's move on and talk

50:02about how artificial intelligence is

50:05different from machine learning and deep

50:06learning. A lot of people uh tend to

50:08assume that artificial intelligence,

50:10machine learning, and deep learning are

50:12the same because they have common

50:14applications, right? For example, Siri

50:17is an application of AI, machine

50:19learning, and deep learning. So, how are

50:22these technologies uh related, right? Or

50:24how are they different from each other?

50:26Now, artificial intelligence is the

50:28science of getting machines to mimic the

50:30behavior of human beings. Machine

50:33learning is the subset of artificial

50:36intelligence that focuses on getting

50:38machines to make decisions by feeding

50:40them data. Deep learning, on the other

50:43hand, is a subset of machine learning

50:45that uses the concept of neural networks

50:48to solve complex problems.

50:50So, to sum it up to you, artificial

50:52intelligence, machine learning, and deep

50:54learning are heavily interconnected

50:56fields, right? Machine learning and deep

50:58learning aids artificial intelligence by

51:00providing a set of algorithms and neural

51:03networks to solve data-driven problems.

51:05However, AI is not restricted to only

51:08machine learning and deep learning,

51:09right? It covers a vast domain of fields

51:12which include natural language

51:13processing, object detection, computer

51:15vision, robotics, expert systems, and so

51:18on, right? So, AI is a very vast field.

51:20Guys, I hope I cleared the difference

51:22between AI, machine learning, and deep

51:24learning. Also, a lot of you might be

51:26confused about data science. Data

51:28science is now an umbrella term, right?

51:31Data science basically means to derive

51:33useful insights from data. So, data

51:35science actually uh uses AI, machine

51:38learning, and deep learning, right? So,

51:40it implements all of these three

51:42technologies in order to derive useful

51:44insights from data, right? Now, let's

51:47move on to the most interesting topic in

51:49artificial intelligence, which is

51:51machine learning. Now guys, the term

51:53machine learning was first coined by a

51:55scientist known as Arthur Samuel in the

51:58year 1959.

51:59Looking back, that year was probably the

52:02most significant in terms of

52:03technological advancements.

52:05In order to define machine learning, if

52:08you browse the internet for what is

52:10machine learning, you'll get at least

52:11100 different definitions.

52:13In simple terms, machine learning is a

52:16subset of artificial intelligence, which

52:18provides machines the ability to learn

52:21automatically and improve from

52:23experience without being explicitly

52:26programmed to do so. In a sense, it is

52:29the practice of getting machines to

52:31solve problems by gaining the ability to

52:33think. Now the question here is, can a

52:36machine think or can a machine make

52:38decisions? Well, if you feed a machine a

52:41good amount of data, it will learn how

52:43to interpret, process, and analyze this

52:46data by using something known as machine

52:48learning algorithms. To give you a basic

52:51idea of how the machine learning process

52:53works, look at the figure on this slide.

52:56A machine learning process always begins

52:58by feeding the machine lots and lots of

53:00data. Now by using this data, the

53:03machine is trained to detect any hidden

53:05insights and trends in the data. These

53:08insights are then used to build a

53:10machine learning model by using a

53:12machine learning algorithm in order to

53:14solve a problem. The basic aim of

53:17machine learning is to solve a problem

53:19or find a solution by using data. Now

53:22moving ahead, I'll be discussing the

53:23machine learning process in depth,

53:25right? So don't worry if you haven't got

53:27the exact idea of what machine learning

53:29is.

53:30Now the machine learning process

53:32involves building a predictive model

53:34that can be used to find a solution for

53:37a particular problem. A well-defined

53:39machine learning process will have

53:41around seven steps. It always begins

53:44with defining the objective followed by

53:46data gathering or data collection. Then

53:49we have something known as preparing

53:51data, which is also called data

53:53pre-processing. Then we have data

53:55exploration or exploratory data

53:57analysis. This is followed by building a

54:00machine learning model.

54:02Then we have model evaluation and

54:04finally predictions. This is how the

54:06process of machine learning works. To

54:08understand the machine learning process,

54:10let's assume that you've been given a

54:12problem that needs to be solved by using

54:14machine learning. Let's say that the

54:16problem is to predict the occurrence of

54:18rain in your local area by using machine

54:21learning. Now, the first step is to

54:23define the objective of the problem.

54:25Right? At this step we must understand

54:27what exactly needs to be predicted. In

54:29our case, the objective is to predict

54:31the possibility of rain by studying the

54:34weather conditions. So, at this stage it

54:36is essential to take mental notes on

54:39what kind of data can be used to solve

54:41this problem or the type of approach

54:43that you must follow to get to the

54:45solution.

54:46The questions you should be asking

54:48yourself is what are we trying to

54:50predict? Right? Here we're trying to

54:51predict whether it'll rain or not.

54:54Right? You need to understand what are

54:56the target features. Target features are

54:59basically the variable that you need to

55:01predict. Here we need to predict a

55:03variable that'll show us whether it's

55:04going to rain tomorrow or not. Then you

55:07must also understand what kind of data

55:09you'll need to solve this problem. Apart

55:11from that, you need to know what kind of

55:13problem you're facing. Is it a binary

55:15classification problem or is it a

55:17clustering problem? Now, if you don't

55:19know what classification and clustering

55:21is, don't worry. I'll be talking about

55:23all of these things in the upcoming

55:24slides. So, your first step is to define

55:27the objective of your problem. You need

55:30to understand what exactly needs to be

55:32done here. Right? How can you solve this

55:34problem?

55:35Moving on, your next step is to gather

55:37the data that you need. At this stage,

55:39you must be asking questions such as

55:42what kind of data is needed to solve

55:44this problem. Is the data available to

55:46me? And if it's not available, how can I

55:48get the data? Right? Once you know the

55:51type of data that is required, you must

55:53understand how you can derive this data.

55:56Data collection can be either done

55:58manually or it can be done by web

56:00scraping. But don't worry if you're a

56:02beginner and you're just looking to

56:04learn machine learning, you don't have

56:05to worry about getting the data.

56:08There are thousands of data resources on

56:10the web. You can just download the data

56:12set and you can get going.

56:14Coming back to the problem at hand, the

56:16data needed for weather forecasting

56:18includes measures such as humidity

56:20level, your temperature, the pressure,

56:23the locality, whether or not you live in

56:26a hill station, and so on.

56:28Such data must be collected and it has

56:30to be stored for analysis. This is where

56:33you collect all the data. Now, moving on

56:35to step number three is data

56:37preparation. The data that you collected

56:40is almost never in the right format. All

56:43right, even if you collect it from a

56:45internet resource, if you download it

56:47from some website, even then your data

56:50is not going to be clean. Right? It's

56:52not going to be in the correct format.

56:53There's always going to be some sort of

56:55inconsistencies in your data.

56:58Inconsistencies include any missing

57:00values or any redundant variables,

57:03duplicate values. All of these are

57:05inconsistencies.

57:07Removing all of this is very essential

57:09because they might lead to any wrongful

57:11computation. Therefore, at this stage,

57:13you can scan the entire data set for any

57:16missing values and you have to fix them

57:18here itself.

57:20Now, actually, this is one of the most

57:21time-consuming steps in a machine

57:23learning process. If you ask a data

57:26scientist which step he hates the most

57:28or which step is, you know, the most

57:30time-consuming, they're probably going

57:32to tell you data processing and data

57:33cleaning. Right? It's one of the most

57:36tiresome task because you need to look

57:38at all the values that are there. You

57:39need to find any missing values, any

57:41data that is not relevant to you. Right?

57:43All of this has to be removed so that

57:45you can analyze the data in a better

57:47way.

57:48Now, step number four is exploratory

57:50data analysis.

57:52So, guys, this stage is all about

57:54getting deep into your data and finding

57:57all the hidden data mysteries.

58:00EDA or exploratory data analysis is like

58:03the brainstorming stage of machine

58:05learning.

58:06Data exploration involves understanding

58:08the patterns and the trends in your

58:09data. So, at this stage all the useful

58:12insights are drawn and any correlations

58:15between the variables are understood.

58:17For example, in the case of predicting

58:19rainfall, we know that there is a strong

58:22possibility of rain if the temperature

58:24has fallen low. Such correlations have

58:27to be understood and mapped at this

58:29stage.

58:30EDA is actually the most important step

58:32in a machine learning process because

58:34here is where you understand your data.

58:37You understand how your data is going to

58:39help you predict the outcome.

58:41Moving on to step number five, we have

58:44building a machine learning model. So,

58:46all the insights and all the patterns

58:49that you got from your data exploration

58:51stage, those insights are used to build

58:53the machine learning model. So, this

58:55stage always begins by splitting the

58:57data set into two parts, that is

59:00training and testing data. Now, remember

59:02that the training data will be used to

59:05build and analyze the model.

59:07The model is basically the machine

59:09learning algorithm that predicts the

59:11output by using the data that you feed

59:13to it. An example of machine learning

59:15algorithm is logistic regression and

59:18linear regression. All of these are

59:20machine learning algorithms.

59:22Now, don't worry about choosing the

59:23right algorithm. Right? First, we'll

59:25focus on what the machine learning

59:26process is.

59:28But anyway, choosing the right algorithm

59:30will depend on several factors, right?

59:32It depends on the type of problem you're

59:34trying to solve, the data set, and the

59:36level of complexity of the problem.

59:39In the upcoming sections, we'll discuss

59:40all the different types of problems that

59:42can be solved by using machine learning.

59:44Moving on to step number six, we have

59:46model evaluation and optimization.

59:49Now, after you build a model by using

59:51the training data set, it is finally

59:53time to put the model to a test. The

59:56testing data set is used to check the

59:58efficiency of the model and how

1:00:00accurately it can predict the outcome.

1:00:03Now, once the accuracy is calculated and

1:00:06any further improvements in the model,

1:00:08they have to be implemented at this

1:00:10stage. Methods like parameter tuning and

1:00:13cross-validation can be used to improve

1:00:15the performance of the model.

1:00:17Before I move any further, I don't know

1:00:19if all of you know what training and

1:00:21testing data set means. In machine

1:00:23learning, the input data is always

1:00:25divided into two sets. We have something

1:00:28known as the training data set, and we

1:00:29have something known as the testing data

1:00:31set.

1:00:32So, in machine learning, you always

1:00:34split the data into two parts, right?

1:00:36This process is known as a data

1:00:38splicing. Now, the training data set

1:00:40will be used to build the machine

1:00:42learning model, and the testing data set

1:00:44will be used to test the efficiency of

1:00:47the model that you built. This is what

1:00:49training and testing data set is.

1:00:51They're not any different data that you

1:00:53derive. They're the same as the input

1:00:55data set. The only thing is you are

1:00:57splitting the data set so that you can

1:00:59train the model on one data and test the

1:01:01model on another data.

1:01:03Now, remember that the training data set

1:01:05is always larger in size when compared

1:01:08to the testing data set. Because

1:01:10obviously, you are training and building

1:01:12the model by using the training data

1:01:13set. The testing data set is just for

1:01:16evaluating the performance of your

1:01:18model.

1:01:19Now, let's move on and understand step

1:01:21number seven, which is predictions. Now,

1:01:24once a model is evaluated and you've

1:01:26improved the model, it is finally used

1:01:29to make predictions. The final output

1:01:31can be a categorical variable or it can

1:01:34be a continuous quantity. Right? All of

1:01:36this depends on the type of problem

1:01:38you're trying to solve. Don't worry,

1:01:39I'll be discussing the type of problems

1:01:41that can be solved using machine

1:01:43learning in the upcoming slides. In our

1:01:45case for predicting the occurrence of

1:01:47rainfall, the output will be a

1:01:49categorical variable.

1:01:51Categorical variable is anything that

1:01:53has some categorical value. For example,

1:01:56gender is a categorical variable. Gender

1:01:59has either male, female, or other.

1:02:02It has a defined set of values. That is

1:02:04a categorical variable. So guys, that

1:02:06was the entire machine learning process.

1:02:09Now, as we continue with this tutorial,

1:02:11in the upcoming sections, I will be

1:02:14running a demo in Python, in which we

1:02:16will be performing weather forecasting.

1:02:18So, make sure you remember all these

1:02:20steps that I spoke about because I'll be

1:02:21going through all these steps by using

1:02:24Python. We'll be coding all of this that

1:02:26we just spoke about.

1:02:28Now, the next topic we're going to

1:02:29discuss is the types of machine

1:02:31learning.

1:02:32A machine can learn to solve a problem

1:02:34by following any one of the three

1:02:37approaches.

1:02:38You can say that there are three ways in

1:02:40which a machine learns. The three ways

1:02:43are supervised learning, unsupervised

1:02:45learning, and reinforcement learning.

1:02:48These are the three methods in which you

1:02:50can train a machine to learn.

1:02:52So first, let's discuss supervised

1:02:54learning.

1:02:55So, what is supervised learning?

1:02:57Supervised learning is a technique in

1:02:59which we teach or train the machine by

1:03:01using data which is labeled. To

1:03:04understand this better, let's consider

1:03:06an analogy.

1:03:07As kids, we all needed guidance to solve

1:03:10math problems. At least I had a really

1:03:12tough time solving math problems. Yeah,

1:03:15so our teachers always helped us

1:03:17understand what addition is and how it

1:03:19is done. Similarly, you can think of

1:03:21supervised learning as a type of machine

1:03:24learning that involves a guide. The

1:03:26label data set is a teacher that will

1:03:28train the machine to understand the

1:03:30patterns in the data. The label data set

1:03:33is nothing but the training data set.

1:03:35So, to better understand this, consider

1:03:37the figure. Right here, we're feeding

1:03:40the machine images of Tom and Jerry, and

1:03:42the goal is for the machine to identify

1:03:45and classify the images into two

1:03:47separate groups. Basically, one group

1:03:49will contain Tom images, and the other

1:03:51group will contain images of Jerry. Now,

1:03:54pay attention to the training data set.

1:03:56The training data set that is fed to a

1:03:58model is labeled. As in, we're telling

1:04:01the machine, "Listen, this is how Tom

1:04:03looks, and this is how Jerry looks."

1:04:05But, basically, labeling each data point

1:04:08that we're feeding to the machine.

1:04:10Right? If the image is of Tom's, we've

1:04:12labeled it as Tom, and if the image is a

1:04:15Jerry image, then we're going to label

1:04:17it as Jerry. By doing this, you're

1:04:19training the machine by using labeled

1:04:22data. So, to sum it up, in supervised

1:04:24learning, there is a well-defined

1:04:26training phase done with the help of

1:04:28labeled data. Right? The rest of the

1:04:30process is the same. After you feed the

1:04:33machine labeled data, you're going to

1:04:34perform data cleaning, then exploratory

1:04:37data analysis, followed by building the

1:04:39machine learning model, and then model

1:04:41evaluation, and finally, your

1:04:43predictions. Also, one more point to

1:04:45remember is that the output that you're

1:04:47going to get in a supervised learning

1:04:49algorithm is a labeled output. This

1:04:52Jerry will be labeled as Jerry, and this

1:04:54Tom will be labeled as Tom. Basically,

1:04:56you'll get a labeled output. Now, let's

1:04:59understand what is unsupervised

1:05:01learning.

1:05:02Unsupervised learning involves training

1:05:05by using unlabeled data and allowing the

1:05:07model to act on that information without

1:05:10any guidance. So, think of unsupervised

1:05:13learning as a smart kid that learns

1:05:15without any guidance. In this type of

1:05:17machine learning, the model is not fed

1:05:20with any label data. As in, the model

1:05:22has no clue that this image is Tom and

1:05:25this image is Jerry. It figures out

1:05:28patterns and the differences between Tom

1:05:30and Jerry on its own by taking in tons

1:05:32of data. For example, it identifies

1:05:35prominent features of Tom such as pointy

1:05:38ears, bigger in size, and so on to

1:05:40understand that this image is of type

1:05:42one.

1:05:43Similarly, it finds such features in

1:05:45Jerry and knows that this is another

1:05:47type of image, [clears throat] maybe

1:05:48type two. Right? Therefore, it

1:05:50classifies the images into two different

1:05:52clusters without knowing who is Tom and

1:05:55who is Jerry. Now, the main idea behind

1:05:57unsupervised learning is to understand

1:05:59the patterns in your data set and form

1:06:01clusters based on feature similarity.

1:06:04Basically, it'll feature similar images

1:06:06or similar data points into one cluster,

1:06:09and it'll form another cluster which is

1:06:11totally different from the first

1:06:13cluster. So, look at the output over

1:06:15here. The unlabeled output is basically

1:06:17clusters or groups of two different

1:06:20data.

1:06:21Next, we have something known as the

1:06:22reinforcement learning. Now,

1:06:24reinforcement learning is comparatively

1:06:26different, right? It's pretty different

1:06:28from supervised and unsupervised.

1:06:31It is basically a part of machine

1:06:32learning where you put an agent in an

1:06:35environment, and this agent learns to

1:06:37behave in the environment by performing

1:06:40certain actions and observing the

1:06:42rewards which it gets from these

1:06:44actions. To understand reinforcement

1:06:46learning, imagine that you were dropped

1:06:49off at an isolated island. What would

1:06:51you do? Initially, we'd all panic,

1:06:54right? But, as time passes by, you will

1:06:56learn how to live on the island. You

1:06:59will explore the environment. You will

1:07:01understand the climate conditions.

1:07:03You'll understand the type of food that

1:07:05grows there. You'll know what is

1:07:07dangerous to you and what is not. You'll

1:07:09understand which food is good for you

1:07:11and which is not. This is exactly how

1:07:13reinforcement learning works. It

1:07:15involves an agent, which is basically

1:07:17you stuck on the island, that is put in

1:07:20an unknown environment, which is the

1:07:22island, where the agent must learn by

1:07:24observing and performing actions that

1:07:27result in rewards.

1:07:28Reinforcement learning is mainly used in

1:07:30advanced machine learning areas such as

1:07:33self-driving cars, AlphaGo, and so on.

1:07:35So, guys, that sums up the types of

1:07:37machine learning. Before we go any

1:07:39further, I'd like to discuss the

1:07:40difference between supervised,

1:07:42unsupervised, and reinforcement

1:07:43learning. Now, first of all, we have the

1:07:45definition. Supervised learning is all

1:07:48about teaching a machine by using

1:07:50labeled data. Unsupervised learning,

1:07:53like the name suggests, there is no

1:07:55supervision over here. The machine is

1:07:57trained on unlabeled data without any

1:08:00guidance.

1:08:01Reinforcement learning is totally

1:08:02different. Here, you have an agent who

1:08:04interact with the environment by

1:08:06producing actions and discover some

1:08:09errors and rewards.

1:08:10Now, the type of problem that is solved

1:08:12using supervised learning is regression

1:08:14and classification problems. We'll

1:08:16discuss what regression, classification,

1:08:18and clustering is in the upcoming slide,

1:08:21right? So, don't worry if you don't know

1:08:22what it is. Unsupervised learning is

1:08:24mainly to solve association and

1:08:26clustering problems.

1:08:28Reinforcement learning is for

1:08:29reward-based problems.

1:08:31Now, what is the type of data in

1:08:33supervised learning? It is labeled data.

1:08:35That is the main difference between

1:08:37supervised and any other type of machine

1:08:39learning.

1:08:40In supervised, you have labeled data. In

1:08:42unsupervised, we have unlabeled data.

1:08:44Whereas, in reinforcement learning, we

1:08:46have no predefined data at all. The

1:08:49machine has to perform everything from

1:08:51scratch. It has to collect data,

1:08:53analyze, do everything on its own. Now,

1:08:55the training in supervised learning is

1:08:57external supervision, meaning that we

1:09:00have external supervision in the form of

1:09:02the labeled training data set. In

1:09:05unsupervised, there is obviously no

1:09:07supervision. There is an unlabeled data

1:09:09set, therefore there's no supervision.

1:09:11In reinforcement learning, there is no

1:09:13supervision at all. Now, the approach to

1:09:15solving supervised learning problem is

1:09:17basically you're going to map your

1:09:19labeled input to your known output. In

1:09:22unsupervised learning, the machine is

1:09:23going to understand the patterns and

1:09:25discover the output on its own.

1:09:27Reinforcement learning, here the agent

1:09:30will follow something known as a trial

1:09:32and error method. Right? It's totally

1:09:34based on the concept of trial and error.

1:09:37Popular algorithms under supervised

1:09:39learning are linear regression, logistic

1:09:42regression, support vector machines, and

1:09:44so on. Under unsupervised learning, we

1:09:46have the famous K-means clustering

1:09:48algorithm. Under reinforcement learning,

1:09:50we have the Q-learning algorithm, which

1:09:52is one of the most important algorithms.

1:09:55It is basically the logic behind the

1:09:57famous AlphaGo game. I'm sure all of you

1:09:59have heard of that. So, guys, these were

1:10:01the differences between supervised,

1:10:03unsupervised, and reinforcement

1:10:04learning. Now, let's move on and discuss

1:10:07the type of problems that you can solve

1:10:09by using machine learning.

1:10:11Now, there are three types of problems

1:10:13in machine learning. Now, any problem

1:10:15that needs to be solved in machine

1:10:17learning can fall into one of these

1:10:19three categories. Now, what is a

1:10:22regression? In this type of problem, the

1:10:24output is a continuous quantity. For

1:10:27example, if you want to predict the

1:10:29speed of a car given the distance. That

1:10:32means it is a regression problem.

1:10:34First of all, what is a continuous

1:10:36quantity? A continuous quantity is any

1:10:38variable that can hold a continuous

1:10:40value. A continuous variable is any

1:10:43variable that can have infinite number

1:10:45of values.

1:10:47For example, the height of a person or

1:10:49the weight of a person is a continuous

1:10:51quantity. Right? I can have a weight of

1:10:5350.1 kgs or 50.12 or 50.112 kg.

1:10:59This is a continuous quantity.

1:11:01Regression problems can be solved by

1:11:03using supervised learning algorithms.

1:11:06Another type of problem is a

1:11:07classification problem. Here, the output

1:11:10is always a categorical value.

1:11:13Classifying emails into two classes, for

1:11:15example, classifying your email as spam

1:11:17and non-spam is a classification

1:11:19problem.

1:11:20Here again, you'll be using supervised

1:11:22learning classification algorithms such

1:11:24as support vector machines, naive bias,

1:11:27logistic regression, and so on.

1:11:29Then we have clustering problem. And

1:11:31this type of problem involves assigning

1:11:34the input into two or more clusters

1:11:36based on feature similarity. For

1:11:39example, clustering the viewers into

1:11:41similar groups based on their interest

1:11:44or based on their age or geography can

1:11:46be done by using unsupervised learning

1:11:48algorithms like K-means clustering.

1:11:51One thing you need to understand is

1:11:53under supervised learning, you can solve

1:11:55regression and classification problems.

1:11:58Under unsupervised learning, you can

1:11:59solve clustering problems. Reinforcement

1:12:02learning is something else altogether,

1:12:05right? You can solve reward-based

1:12:06problems and more complex and deep

1:12:08problems. So, now let's move on and

1:12:12understand the different machine

1:12:13learning algorithms. Now, I will not be

1:12:16going into depth for machine learning

1:12:18algorithms because there are a lot of

1:12:19algorithms to cover, but we have content

1:12:22around almost every machine learning

1:12:24algorithm out there. So, I'll be leaving

1:12:26a couple of links in the description

1:12:28box, right? You can check out all these

1:12:30links and understand how each of these

1:12:32machine learning algorithms work in

1:12:34depth. So, I'm just going to show you a

1:12:36hierarchical diagram of how the

1:12:38algorithms are structured. So, under

1:12:40machine learning, we have three types of

1:12:42learning. We have supervised,

1:12:43unsupervised, and reinforcement. Under

1:12:45supervised learning, we have regression

1:12:47and classification problems. And under

1:12:50unsupervised learning, we have

1:12:51clustering problems. Reinforcement

1:12:54learning is completely different. I'll

1:12:55be leaving a link in the description

1:12:57specifically for reinforcement learning.

1:12:59You can check out the entire content of

1:13:01reinforcement learning there.

1:13:03Now, regression problems can be solved

1:13:05by using linear regression algorithm

1:13:07such as linear regression, decision

1:13:10trees, and random forest can also be

1:13:11used in regression problems. But,

1:13:14usually decision trees and random

1:13:16forest, all of these are used to solve

1:13:18classification problems. Famous

1:13:20classification algorithms include

1:13:22K-nearest neighbor, which is basically

1:13:24KNN, decision trees and random forest,

1:13:27logistic regression, naive bias, support

1:13:30vector machines. All of these are

1:13:32classification algorithms. Coming to

1:13:34unsupervised learning, we have

1:13:36clustering and association analysis. And

1:13:39clustering problems can be solved by

1:13:40using K-means. And association analysis

1:13:43can be solved by using a priori

1:13:45algorithm. A priori algorithm is mainly

1:13:48used in market basket analysis. Right?

1:13:51For this algorithm as well, I'll be

1:13:52leaving a link in the description. We've

1:13:55performed a very excellent demo where in

1:13:57we've shown how market basket analysis

1:13:59can be done by using a priori algorithm.

1:14:02Markov model is also explained in one of

1:14:05the videos. I'll be leaving that link in

1:14:07the description box.

1:14:08Now, to sum up machine learning to you,

1:14:10I'll be running a small demonstration in

1:14:13Python. Right? Like I promised earlier,

1:14:16I'll be using Python to understand the

1:14:18whole machine learning process. All

1:14:20right. So, let's get started with that

1:14:21demo.

1:14:22So guys, for those of you who don't know

1:14:24Python, I will leave a couple of links

1:14:26in the description box so that you

1:14:27understand Python. But, apart from that,

1:14:30Python is pretty understandable. If you

1:14:31just look at the code, you'll know what

1:14:33exactly I'm talking about. Right? So,

1:14:35don't worry. And also, I'll be

1:14:36explaining everything in the code.

1:14:39So, I'm using PyCharm in order to run

1:14:42the demo.

1:14:43Right? So guys, like I said, if you

1:14:44don't know Python, I'll leave a couple

1:14:46of links in the description box. You can

1:14:48go through those videos as well. The

1:14:51main aim of our demo is to build a

1:14:53machine learning model that will predict

1:14:55whether or not it will rain tomorrow by

1:14:58studying the past data set. Now, this

1:15:00data set contains around 145,000

1:15:03observations on the daily weather

1:15:05conditions as observed in Australia.

1:15:08Right, the data set has around 24

1:15:10features and we will be using 23

1:15:13features out of that to predict the

1:15:15target variable which is rain tomorrow.

1:15:18So, this data set I collected from

1:15:19Kaggle. Right, for those of you don't

1:15:21know Kaggle is a online platform where

1:15:23you can find hundreds of data sets and

1:15:26you know, there are a lot of

1:15:26competitions held by machine learning

1:15:28engineers and all of that. It's an

1:15:31interesting website.

1:15:33Now, the problem statement itself is to

1:15:35build a machine learning model that will

1:15:37predict whether or not it will rain

1:15:39tomorrow.

1:15:40This is clearly a classification

1:15:42problem. The machine learning model has

1:15:44to classify the output into two classes,

1:15:47that is either yes or no. Yes will stand

1:15:50for it will rain tomorrow and no will

1:15:52basically denote that it will not rain

1:15:54tomorrow. Right, this is a

1:15:55classification problem.

1:15:57So, I hope the objective is clear.

1:15:59Right, so we'll begin the demonstration

1:16:01by importing the required libraries. So,

1:16:04first of all, for mathematical

1:16:06computations, we'll be importing the

1:16:07NumPy library. We'll also be importing

1:16:10the Pandas library for data processing.

1:16:13Next, we will load the CSV file.

1:16:15Basically, my data is stored in a CSV

1:16:18format in this file. weatherAUS.csv is

1:16:21my data set.

1:16:22So, basically I've saved this file in

1:16:24this path. Right, so that's what I'm

1:16:26doing here. I'm loading my data set and

1:16:28I'm storing it in a variable known as

1:16:31DF.

1:16:32Next, what we'll do is we'll see the

1:16:33size of our data frame.

1:16:36Let's print the size of the data frame.

1:16:38We'll also display the first five

1:16:40observations in our data frame.

1:16:42Let's look at the output.

1:16:45Basically, around 145,000

1:16:47observations and 24 features. Now, 24

1:16:51features are basically the variables

1:16:53that are there in my data set. You know,

1:16:55for example, date is a variable,

1:16:57location is a variable, minimum

1:16:59temperature till rain tomorrow. All of

1:17:01these are variables. So, I have around

1:17:0224 features in my data set, right? Now,

1:17:05the variable that I have to predict is

1:17:07rain tomorrow. Okay? If the value of

1:17:10rain tomorrow is no, it denotes that it

1:17:12will not rain tomorrow. But, if the

1:17:14value is yes, then it will denote that

1:17:16it will rain tomorrow.

1:17:18So, rain tomorrow is basically my target

1:17:20variable.

1:17:21Right? I'll be finding out whether it's

1:17:23going to rain tomorrow or not. So, this

1:17:25is my target variable, also known as

1:17:27your output variable.

1:17:28My input variables will be the other 23

1:17:31variables. Date, location, minimum

1:17:33temperature, rain today, risk, all of

1:17:36this will be my input variables. Now,

1:17:38these variables are also known as

1:17:40predictor variables. Basically, they're

1:17:42used to predict your outcome. So, these

1:17:44are also known as predictor variables.

1:17:47Now, the next step is checking for null

1:17:49values.

1:17:50This is basically data pre-processing.

1:17:53Let me just comment it for you.

1:17:55This is data preparation or data

1:18:01Right. So, this stage is data

1:18:03pre-processing. Right here, we start

1:18:05checking for any null values or any

1:18:07missing values. This is exactly what I'm

1:18:09doing over here. I am checking for any

1:18:11missing or null values in my data set.

1:18:14If you notice the output, it shows that

1:18:16the first four columns have more than

1:18:1840% null values. Right? So, it's always

1:18:21best for us to remove features or such

1:18:23variables because they will not help us

1:18:25in our prediction.

1:18:26Now, during data pre-processing, it is

1:18:28always necessary to remove the variables

1:18:31that are not significant. Unnecessary

1:18:33data will just increase our

1:18:35computations. That's why it's always

1:18:37best if you remove the unwanted or

1:18:39unnecessary variables.

1:18:41Now, apart from removing these four

1:18:42variables, we'll also remove the

1:18:44location variable, and we will remove

1:18:47the date variable. Right, I'll come to

1:18:49this variable in a minute. We'll also be

1:18:51removing location and date variable

1:18:54because both of these variables are not

1:18:56needed in order to predict whether it'll

1:18:57rain tomorrow. Right, we do not need to

1:19:00know the location and the date.

1:19:02Now, we'll also be removing this a risk

1:19:04mm variable. Risk mm variable basically

1:19:07tells us the amount of rain that might

1:19:09occur the next day.

1:19:11Right, now this is a very informative

1:19:13variable, and it might actually leak

1:19:15some information to our model.

1:19:17By using this variable, we'll easily be

1:19:19able to predict, and there's no point of

1:19:21doing that. So, this variable will give

1:19:23us too much information. And so, that's

1:19:25why we're going to remove this variable

1:19:26as well. It'll leak a lot of

1:19:28information. So, after that, if you

1:19:30print the shape of your data frame,

1:19:33we have only 17 variables and so many

1:19:37observations.

1:19:38Now, after this, we'll just uh look at

1:19:40any null values, and we'll remove them.

1:19:43This drop.any function will just remove

1:19:45all the null values. Right, then if you

1:19:47print the shape of your data frame,

1:19:49we'll have around 112,000 rows with 17

1:19:53variables. This is the shape of the data

1:19:55set after removing all the null values

1:19:57and all the redundant or unnecessary

1:20:00variables.

1:20:01Now, it's time to remove the outliers in

1:20:03the data. So, after you remove any null

1:20:05values, we should also check our data

1:20:07set for any outliers. An outlier is a

1:20:11data point that is very different from

1:20:13your other observations.

1:20:15Outliers usually occur because of

1:20:17miscalculations while collecting the

1:20:19data.

1:20:20These are some sort of errors in your

1:20:22data set.

1:20:23So, in this whole code snippet, we're

1:20:25just getting rid of outliers.

1:20:29This is the output that we get. All our

1:20:31outliers.

1:20:32Next, what we'll be doing is we will be

1:20:35assigning zeros and ones in the place of

1:20:38yes and no.

1:20:39The only thing is we're going to change

1:20:40the categorical variables from yes and

1:20:42no to zero and one. Right, that's

1:20:44exactly what we're doing over here.

1:20:46Now, if there are any unique values such

1:20:48as any character values which are not

1:20:50supposed to be there, we'll be changing

1:20:52them into integer values.

1:20:54That's all we're doing over here.

1:20:56After this, we'll be normalizing our

1:20:58data set.

1:20:59This is a very important step because in

1:21:02order to avoid any biasness in your

1:21:04output, you have to normalize your input

1:21:06variables.

1:21:08Right, to do this, we can make use of

1:21:09the min-max scalar function which Python

1:21:11provides in a package known as

1:21:13scikit-learn. You can use that package

1:21:16in order to normalize your data set.

1:21:18So, after normalizing our data set, this

1:21:20is what our data set looks like.

1:21:23This is before normalization. You can

1:21:25see that these are in two digits,

1:21:27whereas these values are in single

1:21:29digits. Right? This causes a lot of

1:21:30biasness. But once we normalize the

1:21:33values, we know that all of the values

1:21:35are in a similar range. We have

1:21:37everything in decimals. Right, so

1:21:39normalization is something that has to

1:21:41be performed because if you have a data

1:21:43set like this, your output is not going

1:21:44to be correct. And that's why we perform

1:21:46normalization.

1:21:48So, now that we are done with

1:21:50pre-processing, what we're going to do

1:21:51is it's time for exploratory data

1:21:54analysis.

1:21:55Let me just comment it for you. This is

1:21:58exploratory data analysis.

1:22:01So, basically here, what we're going to

1:22:02do is we're going to analyze and

1:22:04identify the significant variables that

1:22:07will help us predict the outcome. To do

1:22:09this, we'll be using the select key best

1:22:12function which is present in the

1:22:13scikit-learn library.

1:22:15There's a predefined function in Python

1:22:17called select key best, which will

1:22:19basically select the most significant

1:22:21predictor variables in our data set.

1:22:24When we run that line of code,

1:22:26we get these three variables to be the

1:22:29most significant variables in our data

1:22:31set. Right, the main aim of this demo is

1:22:33to make you understand how machine

1:22:35learning works. That's why to simplify

1:22:37the competition, we'll assign only one

1:22:39of these significant variables as the

1:22:42input. Instead of taking all three

1:22:44variables as input, we'll select one

1:22:46variable and we'll take that as the

1:22:48input and the output is the rain

1:22:50tomorrow variable.

1:22:52So, basically, we are creating a data

1:22:54frame of all the significant variables.

1:22:56Basically, we're choosing this variable

1:22:58in order to predict our outcome.

1:23:00Obviously, our outcome is rain tomorrow

1:23:02variable.

1:23:03So, our input is humidity level and our

1:23:06output is to detect whether it'll rain

1:23:08tomorrow.

1:23:09The next step is data modeling. All of

1:23:11you are aware of what data modeling is.

1:23:13To solve this, we'll be using

1:23:15classification algorithms over here.

1:23:17We'll use logistic regression. We will

1:23:20use random forest classifier, which is

1:23:23another machine learning algorithm.

1:23:25We'll also use the decision tree

1:23:26classifier and support vector machine.

1:23:30Right, we'll be using all of these

1:23:31algorithms in order to predict the

1:23:33outcome. We'll also check which

1:23:35algorithm gives us the best accuracy.

1:23:38So guys, we're just using multiple

1:23:40algorithms or multiple classification

1:23:42algorithms on the same data set. We're

1:23:44not doing anything very complex over

1:23:46here.

1:23:47So, we start by importing all the

1:23:48necessary libraries for the logistic

1:23:50regression algorithm.

1:23:52We're also going to import time because

1:23:54we'll be calculating the accuracy and

1:23:56the time taken by the algorithm to get

1:23:58the output.

1:23:59So, the first step is data splicing.

1:24:01I've already mentioned data splicing is

1:24:03splitting your data set into your

1:24:05testing data set and into your training

1:24:07data set. That's exactly what we're

1:24:09doing over here. So, 25% of your data is

1:24:12assigned for the testing data and the

1:24:14remaining 75% is your training data.

1:24:17Here, you're creating the instance of

1:24:19the logistic regression algorithm. This

1:24:21is an instance that you created. Then

1:24:23you'll fit the model by using your

1:24:25training data set. So basically, to

1:24:27build your machine learning algorithm,

1:24:29you'll be fitting your training data

1:24:30set. So X_train and Y_train variables

1:24:33have your training data set.

1:24:35After that, you will be evaluating the

1:24:37model by using your testing data set.

1:24:40Then you'll calculate the accuracy

1:24:42score. Right? I'll also be printing the

1:24:44accuracy using logistic regression and

1:24:47the time taken using logistic

1:24:48regression. Let's look at the accuracy.

1:24:51Don't worry about these warnings. They

1:24:53are not important. So accuracy using

1:24:55logistic regression is around 0.83%,

1:24:59which is 83% accuracy, approximately

1:25:0184%. And this is the time taken. So the

1:25:05accuracy is actually pretty good, right?

1:25:0684% is a good number. Then we have

1:25:09random forest classifier. Here again,

1:25:11we'll import the libraries that are

1:25:13needed to run random forest classifier.

1:25:16Then we're again calculating the

1:25:17accuracy and the time taken by the

1:25:19classifier.

1:25:20Data splicing, like I mentioned,

1:25:22splitting the data into testing and

1:25:24training data set. Then you're just

1:25:26building the model by using the training

1:25:28data set. After that, you'll evaluate

1:25:31the model by using the testing data set

1:25:33and you'll finally calculate the

1:25:35accuracy. The accuracy using random

1:25:37forest is again approximately 84%, which

1:25:40is a really good number. Then we have

1:25:42decision tree classifier. Here again,

1:25:45we'll be importing the libraries needed

1:25:47for this classifier. We'll be

1:25:49calculating the accuracy and the time

1:25:51taken by this classifier. Data splicing

1:25:54followed by building the model by using

1:25:56the training data set, evaluating the

1:25:58model by using the testing data set, and

1:26:00finally calculating the accuracy and

1:26:02printing the accuracy.

1:26:04So let's see the accuracy using decision

1:26:06tree classifier. Again, we have an

1:26:09accuracy of around 83 to 84%.

1:26:12This is a pretty good number. And last,

1:26:14we're going to do this by using another

1:26:16classification algorithm known as

1:26:18support vector machine.

1:26:20Here again, we're importing the needed

1:26:22libraries. Then we're calculating the

1:26:24accuracy and the time, performing data

1:26:26splicing.

1:26:27Then we're building the model by using

1:26:29the training data set, testing the model

1:26:31using the testing data set, and finally

1:26:33printing the accuracy.

1:26:35So guys, all the classification models

1:26:38gave us an accuracy score of

1:26:39approximately 84% to 83%.

1:26:43So this is exactly how a machine

1:26:44learning process works. Right? You begin

1:26:47by importing all your data, then you

1:26:49perform data pre-processing or data

1:26:51cleaning. After that, you perform

1:26:53exploratory data analysis, where you

1:26:55understand the important patterns or the

1:26:58important variables in your data set.

1:27:00After that, you build a model, then you

1:27:03will evaluate the model by using the

1:27:05testing data set, and finally calculate

1:27:07the accuracy. I showed you all the steps

1:27:09in the machine learning process by using

1:27:11a practical demonstration in Python.

1:27:14So guys, give yourself a pat on the back

1:27:16because we just understood the whole

1:27:17machine learning process with a small

1:27:19implementation in Python.

1:27:22Now let's move on to our next topic,

1:27:24which is limitations of machine

1:27:26learning.

1:27:27Before we understand what deep learning

1:27:29is, it's important to know the

1:27:30limitations of machine learning, and why

1:27:33these limitations gave rise to the

1:27:35concept of deep learning. One major

1:27:37problem in machine learning is machine

1:27:40learning algorithms and models are not

1:27:42capable of handling high-dimensional

1:27:45data.

1:27:46Right? We can take in data with 20 to 30

1:27:49feature variables, but when it comes to

1:27:51data sets which have thousands of

1:27:53variables, machine learning does not

1:27:55work. Machine learning is not capable

1:27:58enough to process that much data.

1:28:01So high-dimensional data cannot be

1:28:03analyzed, processed, and modeled by

1:28:05using machine learning.

1:28:07Another limitation is that it cannot be

1:28:09used in image recognition and object

1:28:12detection because these applications

1:28:14require the implementation of

1:28:16high-dimensional data. Another major

1:28:19challenge in machine learning is to tell

1:28:21the machine what are the important

1:28:23features it should look for in order to

1:28:25precisely predict the outcome. So,

1:28:28basically you're selecting the important

1:28:29features for the machine learning model

1:28:31and you're telling them like these are

1:28:32the important features and this is what

1:28:35you should use in order to build the

1:28:36model.

1:28:37This process is known as feature

1:28:39extraction.

1:28:40Now, in machine learning this is a

1:28:41manual process. You're going to manually

1:28:43input as a programmer, you're going to

1:28:45tell that these are the important

1:28:47predictor variables. But, what happens

1:28:49when your data set has hundreds of

1:28:51variables?

1:28:52How are you going to sit and choose

1:28:54every variable and perform analysis on

1:28:56each variable to understand which is a

1:28:58really significant variable? That's

1:29:01going to become a very tedious task,

1:29:03right? It's not possible for you to

1:29:04manually sit down with 100 variables,

1:29:06check the correlation with each variable

1:29:08and understand which variable is

1:29:09significant in predicting the output.

1:29:12So, performing feature extraction

1:29:14manually is very tedious and that is one

1:29:16of the major limitations of machine

1:29:17learning. Now, deep learning comes to

1:29:20the rescue to all of these problems.

1:29:22So, let's understand what deep learning

1:29:25is and why we have deep learning in the

1:29:27first place.

1:29:28So, deep learning is actually one of the

1:29:30only methods by which we can overcome

1:29:33the challenge of feature extraction.

1:29:35This is because deep learning models are

1:29:37capable of learning to focus on the

1:29:39right features by themselves requiring

1:29:42minimal human intervention. Meaning that

1:29:45feature extraction will be performed by

1:29:47the deep learning model itself. You

1:29:49don't have to manually tell that this

1:29:50feature is important, that feature is

1:29:52important, choose this feature for

1:29:54predicting the output. All of this is

1:29:56not needed in deep learning. The model

1:29:58itself will learn which features are

1:30:00most significant in predicting the

1:30:02output.

1:30:03Also, deep learning is mainly used to

1:30:05deal with high-dimensional data, right?

1:30:08It is based on the concept of neural

1:30:10networks and is often used in object

1:30:13detection and image processing. This is

1:30:15exactly why we need deep learning. It

1:30:17solves the problem of processing

1:30:19high-dimensional data and manual feature

1:30:22extraction.

1:30:23Now, how exactly does deep learning

1:30:25work? Now, deep learning mimics the

1:30:27basic component of the human brain

1:30:29called the brain cell. The brain cell is

1:30:32also known as a neuron.

1:30:34So, inspired from a neuron, an

1:30:37artificial neuron was developed. Deep

1:30:39learning is based on the functionality

1:30:41of a biological neuron. So, let's

1:30:44understand how we mimic this

1:30:46functionality in an artificial neuron.

1:30:49Now guys, an artificial neuron is also

1:30:51known as a perceptron.

1:30:53Let's understand what this biological

1:30:55neuron does and how deep learning is

1:30:57based on this concept.

1:30:59In a biological neuron, you can see

1:31:01these dendrites, right? In this image,

1:31:03you see something known as dendrites.

1:31:06These dendrites are used to receive any

1:31:08input. These inputs are summed in the

1:31:11cell body and through the axon, it is

1:31:13passed on to the next neuron.

1:31:16So, similar to the biological neuron, a

1:31:18perceptron or a artificial neuron

1:31:20receives multiple inputs, applies

1:31:23various transformations and functions,

1:31:25and provides an output.

1:31:27Right? So, that's how artificial neural

1:31:29networks or that's how deep learning

1:31:31works.

1:31:32Now guys, the human brain consists of

1:31:34multiple connected neurons called a

1:31:36neural network. Similarly, by combining

1:31:39multiple perceptrons, we've developed

1:31:41what is known as deep neural networks.

1:31:44The main idea behind deep learning is

1:31:46neural networks and that's what we're

1:31:48going to learn about. So now, let's

1:31:50understand what exactly deep learning

1:31:52is.

1:31:53Deep learning is a collection of

1:31:55statistical machine learning techniques

1:31:57used to learn feature hierarchies based

1:32:00on the concept of artificial neural

1:32:02networks. So, the main idea behind deep

1:32:05learning is to use the concept of neural

1:32:08networks.

1:32:09A deep neural network will have three

1:32:11layers. Okay, there's something known as

1:32:13the input layer followed by the hidden

1:32:15layers and then we have the output

1:32:17layer. The input layer is basically the

1:32:19first layer and it receives all the

1:32:21inputs. So, all the inputs are fed into

1:32:24this input layer.

1:32:25The last layer is obviously the output

1:32:27layer. This layer will provide your

1:32:30desired output. Now, all the layers

1:32:32between the input and your output layer

1:32:34are known as the hidden layers.

1:32:36Now, the number of hidden layers in a

1:32:39deep learning network will depend on the

1:32:41type of problem you're trying to solve

1:32:42and the data that you have.

1:32:44We'll get into depth of what exactly a

1:32:46hidden layer does, but for now this is

1:32:48how a neural network is structured in

1:32:51deep learning. So, guys uh deep learning

1:32:53is used in highly computational use

1:32:56cases such as face verification,

1:32:58self-driving cars, and so on. Right? So,

1:33:00let's understand the importance of deep

1:33:02learning by looking at a real-world use

1:33:05case.

1:33:06So, I'm sure all of you have heard of

1:33:07the company PayPal. Now, PayPal makes

1:33:10use of deep learning to identify any

1:33:13possible fraudulent activities.

1:33:16So, the company makes use of deep

1:33:18learning for fraud detection. Now,

1:33:20PayPal recently processed over 235

1:33:24billion dollars in payments from 4

1:33:27billion transactions by its more than

1:33:30170 million customers. So, basically it

1:33:33processed this much data by using deep

1:33:35learning. PayPal uses machine learning

1:33:38and deep learning algorithms to mine

1:33:40data from the customers purchasing

1:33:42history in addition to reviewing

1:33:44patterns of any sort of fraud stored in

1:33:47the database and it will do this to

1:33:49predict whether a particular transaction

1:33:51is fraudulent or not. Now, the company

1:33:53has been relying on deep learning and

1:33:55machine learning technology for around

1:33:5710 years.

1:33:59Initially, the fraud monitoring team

1:34:01used simple linear models, right? They

1:34:03used machine learning, but over the

1:34:05years the company switched to more

1:34:07advanced machine learning technology

1:34:09called deep learning. This shows how

1:34:11deep learning is used in more advanced

1:34:14and more complicated use cases.

1:34:17The fraud risk manager and the data

1:34:19scientist at PayPal, he quoted that what

1:34:23we enjoy from more modern advanced

1:34:25machine learning is its ability to

1:34:27consume a lot more data, handle layers

1:34:30and layers of abstraction, and be able

1:34:32to see things that a simpler technology

1:34:35would not be able to see. Even human

1:34:37beings might not able to see. This is

1:34:40exactly what he quoted. He said that a

1:34:42simple linear model is capable of

1:34:44consuming around 20 variables, but with

1:34:48deep learning technology, you can run

1:34:50thousands of data points.

1:34:52He also quoted that there is a magnitude

1:34:55of difference. You'll be able to analyze

1:34:57a lot more information and identify

1:35:00patterns that are a lot more

1:35:02sophisticated.

1:35:03So, by implementing deep learning

1:35:05technology, PayPal can finally analyze

1:35:07millions of transactions to identify any

1:35:10fraudulent activity.

1:35:12This is how PayPal makes use of deep

1:35:14learning.

1:35:15Not only PayPal, we also have Facebook,

1:35:17right? Facebook makes use of deep

1:35:19learning technology for face

1:35:21verification.

1:35:22You've all seen the tagging feature at

1:35:24Facebook where we tag our friends in

1:35:26photos. All of that is based on deep

1:35:29learning and machine learning.

1:35:31So guys, that was a real-world use case

1:35:33to make you understand how important

1:35:35deep learning is.

1:35:36Now, let's move on and look at what

1:35:39exactly a perceptron is, right? We'll be

1:35:41going in depth about deep learning.

1:35:44A perceptron is basically a single-layer

1:35:47neural network that is used to classify

1:35:49linear data.

1:35:51It is the most basic component of a

1:35:53neural network. Now, a perceptron has

1:35:55four important components. It has