Drone Programming With Python Course | 3 Hours | Including x4 Projects | Computer Vision

Hello everyone, welcome to my channel, this is the

drone

programming

course

with Python. Here we will learn the basics of a

drone

,

including

the components and how the drone flies, then look at basic installations and movements along with image capture. There we will create interesting

projects

including

surveillance drone face tracking and line tracking drones. If you are a beginner, don't worry, this

course

is for you and we will focus on the implementation part so that you can get the most practical experience of this. Although we will be using a specific drone for this course, eighty percent of the material covered in this course can be used with any drone.

If you want to see more such content, don't forget to like and subscribe, so let's get started. The term drone means a low, continuous hum. This may not be the best way to represent these unmanned aerial vehicles, but it is still the most used term. Drones have gained a lot of popularity in recent years as technology has advanced and motors have become cheaper, allowing drones to be used in much more commercial sectors such as photography, film, surveillance and even delivery systems. Despite this, drones have a long way to go, so learning how to program a drone to perform different tasks is becoming an increasingly important task.

Popular skill drones come in different shapes and sizes. A popular term you may have heard for a drone is quadcopter. This refers to a drone with four propellers or motors. Similarly, a hexacopter has six propellers and an octocopter has eight propellers, so let's look at the main ones. components of a drone a drone will consist of frame motors propellers electric motor controller power distribution board a flight controller battery or bluetooth receiver a camera video transmitter and some sensors the frame or body is what holds everything together most Of the good drones use carbon fiber as the frame material as it is quite light and strong nowadays, many DIY drones use wood and 3D printed frames, also the motors can be considered as the engine of the system that creates the lift. and allows the drone to fly.

There are two main types of engines. Brushed and brushless motors used in a drone are simple and cheap, while brushless motors are expensive but have a better power-to-weight ratio. The propellers are what generate the lifting force when the motors start rotating. There will be two types of propellers for any drone. one that rotates clockwise and the other one that rotates counterclockwise, this is because if all the propellers rotate in the same direction, the drone will also start to rotate, but as we have half of them spinning in one direction and the other half spinning in the other direction, they can cancel each other out and the drone doesn't spin the propellers can have multiple pairs of plates the more blades you have the more thrust it will generate but the more inefficient they become The electric motor controller or the ESC is what controls the speed of the motors.

They convert the DC signal from the battery into AC signal for the motor. You can have a separate ESC for each motor or a single board with multiple ESCs. Power distribution boards allow the battery to power all components, including the motors and flight controller. The controller is the brain of the system and decides what the speed of the motors should be based on the information received from the sensors and the receiver. The battery will power all the components of the drone. This must be light and efficient at the same time. To allow maximum flight time, the receiver or bluetooth allows the transmission of signals between the remote control and the drone, most of the time we will also have a camera connected to the drone along with a storage SD card or antennas for transmission wireless video now to transmit video we need a video transmitter this device is responsible for transmitting video through the antenna to your cell phone or to the fpv remote control there are multiple sensors connected to a drone for example the pressure sensor measures the altitude or The distance between the ground and the drone The GPS locates the position of the drone while an IMU measures the acceleration and the angles at which the drone is The movement of a drone seems quite complicated since it has four or more motors to allow all movements but it is actually quite simple and intuitive.

The drone can move with four degrees of freedom, which means it can move in three directions and turn in one, as mentioned above. Two propellers on a quadcopter rotate clockwise and the other two rotate counterclockwise, generating zero angular momentum that allows the drone to remain stationary rather than rotating in one direction. . Let's first look at translational movements. First we will run all the motors at the same speed, which will generate a lift greater than the weight of the drone. This will make the drone move. above this is the phase translation movement if we want to go down we will decrease the speed of all the motors so that the lift is less than the weight of the drone if we want to float in the air we will change the speed of the motors so the lift is equal to the weight of the drone, this will allow the drone to float above the ground, now we go to the second translation, if we want to move to the left, we will reduce the speed of the left motors and increase the speed of the right motors.

To move to the right we will decrease the speed of the right motors and increase the speed of the left motors. In a similar way, we can move in the third direction, which is to move forward and backward by decreasing the speed of the front motors and increasing the speed of the rear motors. drone will move forward to move backward we can decrease the speed of the rear motors and increase the speed of the front motors as two motors rotate clockwise and the other two rotate counterclockwise of the clock, we can use this knowledge to rotate our drone to rotate the drone clockwise we will decrease the speed of the motors moving clockwise and increase the speed of the motors moving in the counterclockwise direction similarly if we want to rotate counterclockwise we will decrease the speed of the motors in the counterclockwise direction and increase the speed of the motors in the clockwise direction. clockwise Now that we understand how the drone flies, let's take a look at the drone we will be using for this course.

The drone we will use for this course is the teledrone. Now it is recommended to use this drone with this course, but that doesn't mean you can't apply the same techniques to other drones, in fact, 80 of this course are applicable to any drone and only 20 are specific to the teletron. The teledrone is produced by the ryze company and contains DJI and Intel technologies for flight control. It is a compact drone with an attached camera that can record videos up to 720p at 30 frames per second. It is quite easy and safe to use indoors because it has multiple safety features.

Now this video is not sponsored by any of the companies including rise dji or intel and this is my honest opinion that telo is one of the best drones to start with if you want to learn how to program drones, the best part about this drone is that it is programmable, which means that using an SDK we can control the movement of the drone and apply

computer

vision

techniques without any external wiring. That's right, everything will be done via Wi-Fi and you don't even need a router for it, in fact, they have added two antennas that will allow stable video transmission throughout the flight.

The time for this drone is approximately 13 minutes and the flight distance is approximately 100 meters with this maximum speed of 8 meters per second. When you get the drone, you have multiple options, the first is a normal package and the second is a giant package, there are also two. types of tello drones, one is normal and the other is educational, which is educational. This course is compatible with both types of drones. If you get the normal package, it will come with a single battery, extra propellers and the propeller guards, the charging would be On the other hand, if you get the telo jumbo package, you will get everything as before plus two extra batteries and a battery charging center three slots.

The jumbo pack is much better for

programming

because with three batteries. You can do many more tests and while using one battery you can charge the other so that you don't have any delays, so first we will go to App Store or Play Store and search for telu now. once we find it, we will install it and open it. The first thing we will do here is connect the telo device, so we need to turn it on. You'll see the lights and once they stop flashing all these different colors. it means it's ready and it's ready now it's blinking and it's just one color it means it's ready we can go to our app and we can click on connect hello and then we're going to connect to our wifi. so here we will connect to the telo once this is done we will come back now we can see the live stream from our telo device so here you can see the live stream and now it means that it is running so once you have the tele working you will go to settings and you will go to more and here you have to make sure that your firmware is updated so here you will see the firmware version and you will have to click on update so it needs to be downloaded first so you will have to do it. connect your wi-fi again you will have to remove the wi-fi from the telo and you will connect to your home wi-fi and download the firmware and then once downloaded you will go back and upload it to the telo device connecting it again so here it can see that it is already the latest firmware, if it is not the latest, it will allow you to download it and once downloaded you will be able to upload it to the teledrone.

Once everything is set up, we'll try to run it to see. everything is working fine so we will click on the take off button and we will slide to take off and now you can see the drone is flying so we can go forward we can go back we can rotate we can rotate back and then we can go up and then we can go down, so I think I'm about to hit the couch, so I better come back now. One thing to keep in mind here is that while this is fine, let me, okay, we have two options here, tap to land and then manual landing, so most of the time when you're programming you'll be working with tap to land, so I want to mention here one thing which is very important even though this is an indoor drone and it is very suitable for flying indoors.

You have to be early when you operate this drone, so it has a lot of safety features, but you still have to take special care of your surroundings, you have to be aware of where others are and you have to be aware of where you are so that no one gets hurt, so a Once this is done, we can go to the facilities on our

computer

and we can start programming this queue to program this drone. We are going to use the Python programming language. Now to do this, first we will install Python and then we are going to use an ide which is an integrated development environment to write our code, so first we will go to

python

.org and here we will go to the download section and you can download the latest version or my recommendation is to download 3.7.6 so that is the most stable version that I have found that works well so you can go to all versions and here you can find 3.7.6 so you can download this and then you can go to the pycharm download section here.

You can see you have for Windows, you have for Mac, you have for Linux and here you can download the community version, which is free and open source. If you want to understand the idea a little more, you can think of it as a notepad but with more functionality, so every time you write the code it will tell you where you are making mistakes, help you debug the code and generally It has much more functionality than a normal notepad, that's why we need the IDE, one of the most famous. ides for Python is pycharm, so we'll use this now.

The thing is, I already installed both things on my computer, so I'm not going to do that again, but I have the footage from one of my previous courses. which was about the opencv

python

course so it follows exactly the same procedure so I will show the footage right now and you can follow the same steps the only difference will be that when you name your project in pycharm name it tello. course or telo programming course or something like that so let's start with the installations first we will go to python.org and we will go to the download section here we are not going to download the latest version but we are going to download 3.7.6 As this works fine with opencv, we'll click download and then head over to the Windows version.

If you have a Mac, you can also download the Mac version. Once the download is complete, we will install it. Make sure to add Python 3.7 to the path so once the installation is finishedwe will close the dialog and go to pycharm, so we will go to its download section and download the community version, since it is free. Once the download is complete, we will run the installation file here. We will associate the dot pi files with pycharm and also add the launchers directory to the path. Then we will restart the computer and run pycharm, so now we will create our new project here before starting the project, we will analyze the project. interpreter now this has to be 3.7 so if it is detected automatically then we are good to go so let's type opencv python and press create so now this is the pycharm environment here we have the area where we will write the code and on the left On the other On the other hand, we have the project files and the project folders, so once the installations are completed, let's create our project.

Now I have called it stello course, so it will come with this main dot pi file that we can write to this file or we are. We are going to create a new one in which we will store all our code so we can delete this one, we can right click and we can go to new and then to the Python file and here we can say basic basic moves now to program the telo we need its library so to install it we will go to the file, we will go to settings and then we will go to the project, then the project interpreter and then we will add here, we will type dgi tello and here you will see that we have the dgi tello pi library, so we will click on that and press install.

Now while it's installing let's take a look at who actually created this library so let's go to the link and this is the guy his name is damia funitis he's the one who created this dji tello wrapper which is the dji tello sdk wrapper original dji tello. So what this does is it makes it a little bit easier to communicate with the telo and sort all these different things and perform different tasks, so there's documentation for that as well. This is an example, so you need to import the library. and then you have to say connect, you can say move left, move right, move forward and you can give it some values, so a big thank you to the guy who actually built it and to all the contributors who contributed to this Gift Hub repository and gave us made it easy to use the telo library and the license for this is mit, so it basically allows you to use it commercially, you can modify it, you can distribute it, but you have to make sure you include it in the distribution, so this is the license.

If you are distributing, you must also add this to the license. Okay, with that being said, let's go back and now you can see that it's already installed. Another library we will need later is the opencv library, so we can install it now. it's called opencv python we can click on that and we can press install and this will install both and now you can see we have all these new libraries that we didn't have before so we can press ok and now we can continue our coding and first of all we're going to import our library so here we will write from dgi from gi telepi import tello and then we will need the time as well so we will write from time import sleep so we want to add some delays between each command so we need this to connect to our telo es very simple all we have to do is create a telo object so you can call it whatever I want to call it so I'm equal to tello dot tello t-e-l-l-o and this will create our telo object and now we can just type me dot connect and that's how simple it is , just write me dot connect and it will take care of all the IP address and all the communication. part for you, all you have to do is run this and to make sure it's working properly, we'll print the battery, so how much battery do we have in our drone so we can write here?

The dot gets the battery, the underscore of the battery. So there you have it and if you're wondering where do I get all these functions from, then you can click control and then you can left click and this will take you to all the functions that you have available so you can get height get temperature get altitude you have all these different things that you can use, okay, that being said, let me show you more, you have to turn left, you have to go back, you have to give the value of x, the value of x is basically in centimeters, so this is from 20 centimeters to 500 centimeters, you have more advance to the right, but we are not going to use them because it is very simple and you can also use it in one of the third party applications that advance 10 centimeters, not 10. actually let's say half a meter, then move half a meter to the left, and then turn 90 degrees.

You can do all this with third-party libraries. What we're going to do is use the rc command and later I'm going to show you what that looks like, but before we go there, let's run this and see what happens, so we're going to right click and hit run, but before we do that, we need to make sure that the telo is connected to your wifi, so we will turn it on. the telo drone and then we will go to wi-fi and we will connect so i turned on the drone right now and then we will go to the wi-fi settings and here we should have the telodrone popup there you go so now it is connected , I hit connect automatically, that's why it connected automatically, so once that's done, we can go back and we can right click and run basic moves, so if we run that, we get the response first of all, we connect, so we got the answer okay and then we asked for the battery so we got the answer 61.

So right now my drone's battery is 61. That being said now we need to see how we can control our drone so as I mentioned before you can simply write, for example. go ahead, so if you want to go forward, you can write me dot, go, it's go ahead or just move forward, so it's me, period, go forward, so again you can read about all these functions, so it tells you telo, fly forward . with distance to control the speed, as we mentioned in the introduction, the drone can move in three directions and it can turn in one direction, so we want to control that, so we will write here the point send script under rc control and within that we are going to hit En the four figures that we were discussing, first of all, as you can see here, it says velocity left right, so let's say this is zero and then it will say forward and backward, let's say this is also zero, then we have up and down. set it as zero and then we have the yaw rate which means how much do you want to turn so we have all these speeds and if we click on it and if we go there you can see here this is the speed from left to right that we have towards front to back. up and down and its speed, then we can use all this from minus 100 to 100, so if we want to go forward we will write for example 50, it will go forward, we will write minus 50, it will go backwards with half the speed, so we can go back here and let's write here, let's go forward 50, then we'll give it a delay, so we'll write here sleep for two seconds and then we'll tell it to land, so we'll say I, period, land now, one thing I didn't write before was takeoff, so that before we send the command we have to make sure that we are asking our drone to take off so we will write here the take off point and then we can send our command then we can sleep and then we can land now if you want to land correctly and if you don't want to make the drone move forward while landing, then you need to repeat this and write 0 here as well, otherwise the drone when it lands will. it will go forward as well so first we will stop it completely and then we will ask it to land so it will land like a helicopter instead of a plane so let's try this and see what happens so when you saw it take off it went forward. for two seconds at a speed of 50 and then it landed so now we can move the drone to the right as well so we can copy this and as we know the first one is basically left and right so we can change this to let's say 30 .and let's see what happens, so as you saw the drone took off first, went forward for two seconds, then went to the right side for two seconds and then landed, so if we write here minus 30, it will go to the left if we write. here minus 50 will go backwards, so this is the basic idea and if we want to rotate it, we can rotate it, for example, we can say here we can say 30 and let's try what happens when you saw that the drone moved forward first and then turned to the right, so this time it is in yaw, not actually on the right side and then it landed, so these are the basic movements that we can control, we can control up and down, left and right, forward and forward back, and we can control.

If you like the video so far, please like it and don't forget to subscribe, so now we will learn how to capture images from our drone so that we can process it and do many different experiments with it. we will go to our tello course and we will create a new file and we will call this file image capture and the easiest thing to do is just copy the standard, you can call it the standard model for the telodrome, you can copy it and now we can continue with our code to that the image acquisition part is not so difficult.

What we have to do is write me dotstream and we will turn on this stream so that this stream gives us all the frames one by one and we can process them now because it is a continuous number of frames we have to use a while loop so we will write while true and here we're going to write our image equals me dot get frame get underscore frame read and then we're We're going to write a dot frame, so this is going to give us the individual image coming from the teledrone and now what we can do is change the size of this image if we want to process it faster, to do so we need to import our opencv library. we can delete this this time, we are not going to use it, so we can write here import cv2 import cv2 and at the bottom here we are going to write image equals cv2 dot resize and then we are going to give our source image, which is img and then we have to give it the size that we want to resize, for example we can say 360 by 240, so we keep the frame size small to make it faster, so we can format this and then we can write. here cv2 dot I am show so this is for us to create a window to show this result so we will write here I am show and then we will write here image and we will say this is the name of the window and then what we do we want to show we want to show img, so this is the idea and then we're going to write the cv2 point weight key.

Now we are writing a weight key because if we don't write this, the frame will close before we can see it. so we have to give it a delay so we're giving it a millisecond delay so this should work fine and actually let's do one thing let's remove this for now and let's try it without resizing and then we'll try it with the size change. so let's right click on image capture now again. We have to make sure the teledrone is connected. Now it's already turned off because I haven't turned it on for a while or haven't used it for a while, so let's go. wait for it to connect fine, so now that it is connected let's go to the image capture, right click and we will run the image capture so that we have 40 batteries left for the image capture, it should not be a problem and that's it, like this that this is the current coming from my uh drone at this point you can see there you go so now it's one two eight zero times seven 720 and what we can do is if we resize this it's going to get quite a bit smaller so there you go so now it is much smaller and you will get some warnings here, don't worry about this, as long as you get the frames it is fine and you will see a delay in the frames.

This is because of the transfer and this is because of the display in Actually this delay is not much but because we are watching it we are continuously trying to do all the graphical calculations that is why it is giving us this delay so which we'll normally keep it at this size so that processing is a little bit easier and it's going to be a little bit faster with all the different functions that we use, so now we're going to use our keyboard to fly the drone. To do this we first need to get the keyboard commands and then we can implement it with our basic moves and we can execute them together so to do this we are going to create a new module called keystroke module so a module is basically a code snippet that can be run individually and can also be run from another script so if there is another script wants to use its functions it can also do that so we will create our module we will write it as push modulewe're going to remove the rest and we're going to paste it, so the first thing we have to do here is write our parameters, so, as we mentioned before, we need the velocity and also the angular velocity. and from that we will calculate the distance and we will calculate how much angle we have now first we are going to write the forward speed now the point is that we already have the forward speed that we are sending, for example 50 means 50 centimeters per second, but the point is that when I experimented with it and tried to figure out my own values ​​to see if they were correct or not, many times the values ​​differed, so the teledrone is not very repeatable and it is not.

It's accurate but we're still trying to learn the concept and from my testing I found that if I move for example 10 seconds I get a distance of 117 so 117 divided by 10 is basically our speed so I can write it here. speed of travel which is basically in centimeters per second so this is the centimeter how many centimeters it traveled and these are the seconds it traveled so it traveled in 10 seconds 117 and it was giving the actual speed of 15 centimeters per second and you can see it's coming in at about 11.7, about 12 centimeters per second, so again doing some manual testing can also help and again with the angular velocity I tested that as well, so the velocity, let's write it here in capital letters, just like the angular velocity it took about 10 seconds to rotate 360 ​​degrees, so our velocity will be as fast as angular velocity in degrees per second, so this will be our angular velocity in degrees per second.

Now in the example we took the interval as one second so we were checking the value or we were plotting the value after one second now that is a long time we don't want to do that we will reduce the interval so that actually the interval is equal to 0.25, so based on this information, we now have the speed and we have the interval so each unit on our map will represent a certain amount of distance and a certain amount of angle, so distance and angle that we can find, we can write here that the distance interval is equal to the forward speed multiplied by our interval and then the angular interval is equal to velocity multiplied by interval so this will give us the distance and the angle every time we move one unit okay these are basically our parameters now you can play with these you can try it yourself you can time it using a stopwatch you can write some actual code yourself to test these different things but these are the values ​​that I found to be a little bit better so which again this is not very accurate so don't expect it to be very good but the idea is that if you understand this concept and you come across a better drone then you can use the same methodology to create a more accurate model.

That said, what we can do next is try to plot something to see how opencv actually works. works, so to plot, let's delete this to plot, we're going to create an image and we're going to create a black image, so we're going to call it image equals to and we're going to use the numpy library now that we haven't used. numpy before is basically a math library that allows all these different math or matrix calculations, so we're going to import that, so we'll say import numpy as np, so we'll use that, so the image is basically an array of pixels, a array of numbers, that's why we can use the numpy library which is known for handling arrays, so we can use it to call numpy.0s, we want an array of zeros and we want it to be the size of one thousand times one thousand times three, so uh the three are basically the channels so we wanted a color image so we have rgb channels and here we will say that the values ​​that are stored in these arrays are basically unsigned integers this is how you write an unsigned integer and it should be 8 bits 8 bits means that we can have a value of 2 to the power of 8 2 to the power of 8 which is equal to 256, so our values ​​will range between 0 and 255, so these are the values ​​that will be stored here okay, with that being said, let's draw something on it, so let's create a function called draw draw points and we will call this function when we want to draw, so now we will draw something simple, uh, let's just draw a circle, like this that cv2 point circle and we're going to give our image, so the image that we want to draw the circle on, then we're going to give our points, let's say 300 and 500 and then we're going to say that our, I think the next part is the size.

I check radio wait it's not, it's not supposed to be radio, what's happening is cp2 imported, oh it's not imported, that's why it imports cv2, let's try it now, yes, there you go, so now it tells us what is required, so yeah, next up will be radio, then color. thickness then radius let's say 20 color let's say red then let's write 0 0 and 255 then it's bgr in opencv we don't use rgb we use bgr so this is blue green and red and then we're going to say thickness we want to fill so we'll write it here filled so oh , I typed it wrong here, so okay, so we can do some formatting here and okay, let's run this and see what happens.

Oh yeah, we don't actually show anything, so we have to show this point cb2, I'm show and we're going to write the name of the window, so let's say output and then we're going to write the image and then we're going to write the hold key of the cv2 point to have an intermediate delay of one millisecond, so let's run this and that's it. So this is our output screen, it's 1000 by 1000 and you can see the circle is drawn here, so we can make it a little bit smaller, it's very big, let's make it five, let's see and there you have it, now this is one point and If we continue adding more points, then we can create the graph we need.

Okay, now what happens is we have to find the value of how far we have traveled, so let's call this, these are our final values, let's call them x and y. So we'll keep it very simple, we'll call them x and y, so here we'll say that x and y are both 500, so let's write that at the top, so the idea is that at any given point we want to plot this x. and and then we can just go here and we can say instead of this we can say y and we can say , this value of x and y should change, so what type of change do we have to put here?

Let's take a look, let's look to the left first, so here what we're going to do is add our distance and then We have to add to our angle, so at any given time, every time we press the key, we have to check the angle and the distance, we have to update them so that this is the final value in the x and y coordinates, but there should be another value that is basically the distance and the angle and we will check each time what is happening to them, so the angle we will keep the same, so the angle should not be reset to zero every time because it will continue to increase, but we will reset the distance every time. time, so we'll write d equals 0 inside here and a equals 0 outside, okay, now we need to update this every time we press a key, so for the left we'll update our interval, so we'll write here d equals our dp our d interval, so normally when we travel, we travel with this distance, but we do not travel in the forward direction, we travel towards the west, so we have to change our angle, so we will set the angle to minus 180 degrees now to the right. we will copy this to the right we are going to change our angle and the angle will be 180 degrees and our distance will be negative so we will go up now up our interval will be positive but our angle will be north be 270. and then to lower our angle, so it's not really up and down, it's basically back and forth, so we have to write that our interval is again negative and then our angle is negative 90.

Okay, this is basically simple. because we are simply moving from left to right back and forth, but when it comes to the actual rotation, it gets a little complicated, so we need to change the angle, so here what we will do is check our previous angle and add it. So a will be our yaw equals plus equals our interval a, so it's not just going to be a new value, but it's going to be added to our old value and in the other direction we're going to remove it, so we're going to write here minus a. interval now we don't declare yaw, so we need to declare it and we will write here its equal to zero and we can write here, let's say global yaw, okay, after doing that, we have updated all the values ​​that we need to update now that we need. to convert it to Cartesian coordinates, so here we're going to update our angle so that the angle is more equal to the yaw and then we're going to get our x and y, so finally we're going to write x is equal to actually plus equal to because we have to add it to our previous value will initially be 0 0 will then be added, so we will write an integer here and then we will write the distance multiplied by the mathematical cosign of the mathematical point.

I think math is not imported, so we need to import here. import math and if we go down, okay, so math.cosine and then we'll write math point radians because this function actually uses radians, so we have to convert them to radians, so math.radians and then we can enter the value of a so that there is a problem most likely because of the brackets this is for this this is for this oh no ok spelling then it's good there is still an error for some reason crazy dot cosine math dot radians a and then ok I think the problem is because x is not global so we'll put it as x and y there you have it, then we can copy this and paste it here and we can say this is y and this is sine instead of cosine, so do we need to change anything else?

No, this seems fine, so what we can do now is send this x and y here. It's already submitted, it's already been updated, so we don't need to worry about points, but we could actually write it from here. we can send it back from here and then from here we can send let's say our image that we want to draw and we can send the points so we can do this but it should work automatically as well but this is a better way to do it so we can write here points and our points are basically points equal to the values ​​in four and the values ​​in five, okay, so let's run this and see what happens, point zero and then point one, so let's run this and see what happens, right?

OK? we get an error local variable referenced before assignments where is a, so this should go here, let's try again and that's it, this is our buy game, we have to click here to make it run, Wow, that goes very fast. Okay, so it goes back and forth, um and then it rotates, yeah, okay, so it's working, but there's a problem. What is the problem? Let's try to find out. Okay, first thing is that I forgot to add the delay. So if the time is not accurate then all our calculations will be wrong so we need to write here the sleep time point or I think we import the sleep directly so we can write here for example 0.25 so which must be exactly the same as the interval. so we will copy it from here and write it here, so this has to be our interval and now it will not be as fast as before because of that interval, so let's try it, yes, now it is much slower and it looks much better. and now you can see that it is also rotating when I press some keys, but again the problem is that we can't see the path, so it always prints this point, but we don't just want this point, we want all the points that have been plotted. since what we can do is create um points, we can go here and we can type points and we can put them as an empty list, then we can go down and instead of sending these values ​​directly here, what we can do is we can add so we can say points dot add and we want to add these values ​​so that this keeps adding to the points, so the other thing we have to do is here again, we have to loop through these points and then we have to print. take them out or we have to draw them, so we need to add a loop here, so we're going to say that for points on points we're going to write dot on 0 and dot on 1 or we could just write dot as well, it should work. let's run that and see if it works, we get an error argument that is not a tuple, okay, so which line is okay?

I think we just need to change the square brackets here because they are for a list, so I hope this works, yeah, okay, there it is. let's go and now if we click and there you have it, now if I go forward you can see that I've held down the forward button and now if I press d you can see that it's spinning but it's spinning in the wrong direction, okay,now if I press You can see it's moving and if I press left it's going to go left, if I press up it's going to go up and give us the estimate so we can just print out the x and y value and those are going to be the coordinates or those are going to be. the distance traveled so we can print it here we can say put cv2.put we will put it in the image and we can write the points so we will write here f and then inside that we are going to write a bracket is here the points we refer to this and not this, we are outside the loop, so we'll write the dots in the last, so we want to put this text at the end, so we'll take the last value and take the zero element of it. which is the same so we're going to copy it, paste it here and we're going to write -1 and here we're going to write 1 and then at the end we're going to write meters, that should be good, so this is basically the distance or the x and y coordinates printed out, but now we have to mention where we really want to put this, so we need to define the points, the point where we want to put. so we don't want to put it at exactly the same point, we want to change it a little bit, so we'll write the points minus 1 and 0 and we'll add 10 to it to move it forward a little bit and then we can copy this and paste it here and this time we'll add, let's say, 30 so it's down a little bit and we'll do this one so we can format this like this, let me bring it here a little bit right so it should be good enough and there was another problem that it wasn't turning in the right direction so when I was pressing a and d, the values ​​were reversed, so this is supposed to be more and this is supposed to be less, okay, so that's one thing.

Now, another thing that we did in the previous chapter of what you call keyboard control is that we typed a single value here, the velocity as 15. But in this case we will have a different angular velocity and we will have a different angular velocity. the ground speed, so here we have to type it here or change it down here, so let's keep it as speed, so we'll keep it at 15 because that's the speed that I used for testing and the angular speed that I used. was 50. so I used 50 so we can write a speed is equal to 50. so we don't need to change this, but we need to change this down here, so there you have it, now if we run it right, something is missing. oh yeah we didn't complete it we didn't complete this one font is missing so we have to write the font cb2 dot font plane and then we have to write the thickness what else we have to write the color we forgot? many things two five five zero two five five and let's put the scale too so this is the scale and this is the thickness so let's put a comma there and run it there so now it says 4.95 and 4.95 uh why?

Are you saying it should be zero zero at this point? Where did I make a mistake? points in minus one um I think you are using zero zero maybe so we need to check here that if points minus 1 in 0 are not equal to values ​​in 4, which means it is the previous value or points in -1 one not is equal to the values ​​in five now this just makes sure that we're not using, we're not plotting the same thing over and over again, then we're going to add the opposite we're not going to add, why is it giving this, let's run it.

The index is out of range. The indices are out of range. Alright. So what we can do is give it some values ​​at the beginning so that it's not empty, so we're only giving two values. at the beginning so it can be compared, there you go, but the value is still 495, why are we going to check our value here? I think we need to add a square bracket here, maybe let's run this, yeah, that was the problem, okay, now we have zero zero and Let's move it forward and that's it, now we get the distance and the coordinates in both directions and we get the map.

As I move it, I can see what's happening and I can press these different keys. and it will keep spinning and at the end of the day it will tell me how far I've moved, so what I can do at the end is create another circle to be for the header so I can change the color of the page. let's make it green and make it a little bit bigger, so we'll do it, let's say 8 and we'll use the last dots, so let's write again, okay, I'm missing something again, it's dots, okay, there you go, now it's green and if I move it, there you have it, now it looks much better, it gives us an estimate of where the drone is, so now let's try it on the actual drone and see what happens.

In this project we will see facial tracking. about how we can make our drone track a particular object or a face or even a whole body, so we will do it with face tracking, but later you can implement it with any of the other objects as well, so first of all, let's have a Look at the concept now, when we have to move the drone and we have to track our face, we need two different things, we need the forward and backward movement and we need the rotation movement, so we need its and forward and backward, so let's first take a look at front and back, so we will treat them individually and then combine the code for them, so let's take a look at the concept of front and back, so if the object or face looks small, then it means I need to move forward and if the object or the face looks quite big then we have to move back to be too close we have to move back so if this is my drone and this red area is where it is too close and this red area is where it is too far away, so in both the drone has to move.

This green region is where you don't have to move at all, so let's say the person here moves backwards. So now the drone will have to compensate and it will go forward, it will go up to that green zone and then it will stop. Now let's say it moves forward, then the drone will have to counteract the movement by moving back and it has to go. it moves back until it reaches the green zone, so this is the idea that there will be a green zone in which the drone will not move, it will be stationary and if it is too close it will retreat, if it is too far it will move forward, so this area we go to define by area so let's say the area here is 6200 and the area here is 6800 so these are the values ​​that I found after some testing so we will use these values ​​to put our drone so this is the area, so if you take the number of pixels and multiply the width by the height you will get this value so they are again in pixels, you can change these values ​​later according to your needs if you are making a full body.

The area is going to be different than if you're making a face or you're making another object, so with that being said, let's take a look at the yaw angle, so the idea here is that we can go forward and backward, but what if the person starts? Rotate the idea here is that we always want our object to be in the center of the image, so let's say this frame is the image we have and the person is not in the middle, so if it is on the left side, you will move . the drone you will rotate it and this will bring the person to the center and if, for example, they are here and then you can rotate it and this will bring them back to the center.

Now we can give it a value, for example. Let's say at this point our object is on the left side, then we will say that it rotates counterclockwise with a speed of 20, let's say it starts moving at a speed of 20 and when it reaches the center it cannot stop immediately, it will move. a little bit forward because of the momentum, think of this as a fan, so if you turn off a fan, it will not stop immediately, it will keep spinning and after some time it will stop, so the same thing will happen here, so if we tell you go 20 degrees or sorry, go with a speed of 20 centimeters per second and when it reaches the center, if we tell it to stop, it will not stop immediately but will go a little further.

This is called overshooting, so the same thing will happen. If we say she's on the other side and then we do it here, what happens is when it overshoots, the image won't be in the center, it will go here, then it will try to recover and it will go here, so it will keep coming back. and forward and it won't stabilize, so how can we improve it so we can improve it by slowing down as we get to our destination? So let's say the speed here is 20 and once we get close, let's say the person is here now. then we will slow down, let's say now it is 10, then it will become 5 and when it reaches the center it will become 0.

This way instead of having plus 20 and minus 20, it will have a value that will change depending on how far away we are of distance or how far we are from the actual point, so this method is known as pid. I won't go into detail about how pid works more than this. I think this is a very basic explanation that will give you a basic understanding, but if you want to learn more about pid, it is basically proportional integral and derivative, so these are the terms, it is the type of controller that you can use, you can relate two different entities that can't be related by default, so these things are a little bit complicated if you haven't seen them before but for this course this explanation is more than enough that you are able to go back and forth and when gets to the center it has to slow down so we will use this methodology in our implementation so let's take a look at the implementation now so first we will create a new file and we will call it face tracking so this project is more towards opencv, so first we will see the opencv part and then we will see the integration towards stello. so let's import cv2 and then we will import numpy as np so what we will do first is run our webcam so I will use my standard webcam instead of the teledrone and once everything is done we will integrate it afterwards so I will write here . cap equals cv2.video capture video capture and I have multiple webcams so I'm using number one, if you don't have multiple webcams you'll most likely use number zero okay and then we'll write as long as it's true , TRUE. let's get our image from cap dot read and then we'll say cv2 dot im show and we'll say this is our output, let's say and this is our image and cv2 dot waits for the key and we'll give it a delay of one so this is the standard code to run the webcam with opencv so let's run this and see what happens so right click and run so here is my webcam so I'm going to use my image for the face tracking parts and once that's done, we will try it. with the telodrome it's okay so now the first thing we will do is track or detect the face first so we will create a new function and call it say find face and we want to find it in an image so we will pass the image in which we want find it and to find these faces we will use a method proposed by Viola Jones.

This is a very famous method of using Har waterfalls. This is basically a file that has all the parameters and information of a model that helps us to detect different objects, in this case the file that we have will help us to detect faces, so let me copy that file so that it is available on the website opencv official and also available on my website in this course so you can download it from any of the places so you can go to resources and we will paste it here so the name of this file is our default waterfall front face, actually we're going to click rename and copy it. this name because it is too long and now we have to reference this file so we will say face cascade equals cv2 point cascade classifier and then we will write to the image path sorry this file path now this file is inside resources. so we have to write here resources and then we can write the link or the path of our file, then we are going to convert this to grayscale so we can do it by writing cb2 dot cvt color now, if all this is overwhelming for you, what do you think?

What you can do is go back and see my opencv course. It has been very popular. It now has over a million views and 99 positive comments. If you check the opencv part this will be very easy for you but if not. I don't have much background with opencv so just take my word for it that for example this line is to convert to grayscale so we will write the image and then the cv2 point color will underline bgr to gray so this is it will convert to gray and then We will detect all faces so we will write faces equal to multiscale face cascade point detection and we will pass our image so that the gray image that we just made and then we will pass some parameters now these parameters normally the first one is basically the scale factor and the second one is the minimum neighbors, so what you can do is play with these values.

The ones I found good are 1,2 and 8. So again you can read about these in their default function or you can check it in the documentationonline. I won't go into that much detail here, so we'll create a list of all the faces and then create a list of all the areas of these faces. So there can be multiple faces, so we need to find the largest one and we'll assume that's the one we want to track because it's the closest to the camera, so we'll detect it and write my list of faces here. is equal to empty and then we can write the area of ​​my face list, in fact, let's write here c for the center, so it will have the information of our center point, so cx and cy, where is our face detected and this will have the area value?

There will be two lists and then we will create a for loop, so we will write 4 x and w and height, so basically every time it detects these faces it will return these values ​​to us so that we can unpack these values ​​inside these faces and we can write x and z to the faces. and then we will write that we want to draw the rectangle around this face, so we will write here the cv2 point rectangle and for the rectangle we have to send an image, we have to give it the x y coordinate and then this is the starting point and then we need the point end, so the end point will be x wider and then y higher, then we need the color, so let's set it red to be bgr, so 255 will be at the end and then we have the thickness.

Let's put it. like two, okay, so that's good, now let's run this and see what happens, let's test it before we continue and we need to call this function, so we'll call it by typing it here and let's run it, so that's it. You can see the face is being detected correctly and it doesn't seem too bad even if I'm moving, it's 30 and it stays there. If you don't find good results, you can always change these values ​​to get something better. Now once we have this, what we're going to do is we're going to get our center x and center y, so we're going to write here c x equals x plus width divided by 2 and then c y equals y plus height divided by two, so once we have this , we can also find that the area is equal to the width times the height, as simple as that times the height and then we're going to write that we want to add this, but we should print, okay, let's add it first. so my face list c dot append we are going to add the value of cx and cy okay and then we are going to add my face list dot append area and we are going to add the area so once we do both of those things what will happen? happens, okay, let's print it first so we can draw a circle at this position cx and cy so we see where exactly the center is so we can write here the point circle cv2 and we need an image, what else do we need? center so the center will be cx cy and then we need the radius, let's say the radius is 5, then we need the color, so let's say zero two five five and zero and then we need the thickness, we'll say fill cb2, dot fill, we want it full uh, full from the inside and that should be fine, so let's run this and see what happens and voila, now you can see the center.

The center value is also displayed, so this value is what we will return, but what? What we can do is because now we only have one face, so obviously we're going to return this value, but the reason we made these two lists is because we want to find the maximum area and we only want to return that value, so here we do it. will. check if the length before doing any calculation on this we need to check if it is empty or if it is full so if it is empty there is no point in doing calculations so first we will check my faces we can choose any of these areas of my faces . is not equal to zero if it has some values ​​inside then we will say it is equal to the index of points of the area of ​​my face we want to find the index and then we want to find the index of the maximum value so that the maximum value of our area my list of areas is okay so this will give us the maximum value of our area and this will give us the index of that maximum value so once we have that index we will say return image and then we will also return the list of my face in c in i and we also want to return the area because we want to use it, then from that list we will use this index to find the maximum value of the area and from that area we will get the cx and cy values ​​of that corresponding face, but if this is not the case, then we must give it, we must return something more, so we'll say: another thing, return the image and then return 0 0 for cx and cy and 0 for the area, so we'll pretty much return. nothing right, so now what we can do is get the values ​​to have an image and then we have what else we have.

We have cx and cy. We can call it collectively as information, so this will be the information that we will get. Come back and once we have it, what we can do is print it so we can write here information in let's say 1, this will give us the area so we can write here area is information one, so let's try that and that's it. now you can see the area that is being detected, as we mentioned at the beginning of the explanation, if we move forward, the area will increase, if we move back, the area will decrease, so based on this we are going to move our drone, okay, that's good . and then we also have the center value, so the center will be zero. um if we print it all together maybe it'll be easier to read so let's print it here like this so let's run it again and that's it now we're getting the center values ​​and we're getting the area so the center values ​​will be used to rotate and the area value will be used to go forward and backward so this is good now what we can do is enter our drone and we can also enter the range so let's do the tracking first so here we will define a function for what we will say track the face and what we need is the drone, then the variable me and then we need the information and we also need the width and the pid. and the p error now I'm going to explain everything they are so don't worry about that at least for now so what happens is we need to create a range so let's do the forward and backward right now.

We need to create a range, so here we're going to write that our forward and backward range is equal to uh, as I mentioned before, it was six to eight hundred, sorry, six two hundred and then six eight hundred, so these were the values for our range and then we go to where it is and then we're going to write here that if our information or we should call it area, let's call it area so it's easier to look at what's happening, area is equal to information at zero, now it was at one, so if the area is greater than the front, uh, the forward and backward range on number two, then we need to go back so that the forward and backward speed is minus 20, okay? so this is that and then we can write something else, whether the information or not, the area information is less than the range forward and backward at the number zero, this should be 1. so we will say that forward and backward is equal to 20. so at At this point, it says if it's too far, move forward, if it's too close, move back.

Now we need to add a factor of safety here and the reason is that if we just say that the area is less than this value, which is 6000 something. so 0 is also less than 6000 so if nothing is detected it will start moving forward so we have to mention here that area is not equal to 0. So if it is not equal to 0 and it is less than our rank, then advance. otherwise it will move backwards and we also need to add a condition where it will remain stationary so that should be the first priority, if it has to remain stationary then remain stationary so we will write here if the area is greater than our range forward and backward at zero and our area is less than our range forward and backward at number one, so this is the idea and then we will write forward and backward equals zero and we can do this another way way, if so now we have these three conditions, the first is where we have the green zone, the drone will not move at all, the second is where it is too close, it is too big, so it will move backwards and this is where it is too small or too far away and it will move forward, so now all we have to do is send our command to the drone, we can send this fb value back and forth, that's it.

Now let's take a look at the pid for our angle, so for the pid we're going to enter the error. is equal to, let's unpack our x and our y as well, so let's delete this, bring it here and then we can say that our x and y are basically zero information, so here we'll say that our error is basically the x position, so let's say that the width divided by 2. So what is this width divided by 2? It is the center of our image. So we want to know how far our object is from the center.

So this is the value of the object or the. face and this is our center so when we subtract that we find the offset of how far away it is once we have that so this w is basically the width of our image so we'll define it when we call this function and then we'll use the pid equation now again, you don't have to go into too much detail about what this does, if you just want to know how it works you can copy the code; Otherwise, if you want to go into details, I highly recommend you check out any of the other videos on YouTube that have a lot of information about pid, so let's write here that our velocity is equal to pid, this is a constant value, so which we're basically changing the sensitivity of our error by this value, so we'll declare it. here we will call it pid is equal to and we will give the values ​​of our pid we are going to use 0.4 0.4 and 0 in our case so this is proportional p this is integral and this is derivative so we are only going to use p and d we will not use i so here this is proportional, it will be p and then what we will do is multiply it by the error and we will add the pid1, which is its derivative, we will multiply it by the error minus the previous error, then this will give us the speed and what we will do is cut this speed between minus 100 and 100, we don't want it to go over 100, we don't want it to go below 100 minus 100, so we'll do it. type here speed is equal to an integer, we're going to convert it to a injured integer and we're going to say numerous point clip and we're going to clip the speed between minus 100 and 100, so this is our new speed for our yaw that we can send to our drone If you remember, for the drone we can send the value something like this so we can say yo dot send underscore rc underscore control and then we send the four values ​​so that the first one is zero and the second one is forward and backwards then will be this and the third will be zero and the fourth will be the yaw which is our speed this is how we can send our command to the drone now one more thing we have to do we have to return this value return error because we need to use the previous error so this error when it comes back here it will come back as the previous error okay here we are going to write the error uh it is a global error no it is not global but we can write the error e as global so yeah , we can do that and then when we call it, let's call this function, after detecting that we are going to send all this and we will get the error p which we will send in the The next one is okay, so here we have to include the code for the drone.

We will still wait for us to send the information, which is good. So we need to send the width. So where can we define the width? We can define it here. We can say that our width and height are equal to, say, 360 by 240. As we did before, we can also resize our image based on these values, so we can write here that image equals cv2 dot resize and we will give it our image and then we will say we want it as width and height so this will be the value and then we can send this and we already have the pid at the top declared so it should be fine now again you can change these values If not I don't find good results, you can change these values ​​and try different parameters.

I didn't put much effort into adjusting this because the adjustment usually takes a lot of time so I'll show you the basic method and then you can adjust it yourself and see how it works well, so with that said what else is missing, do we have something on the face of the track ? So on the track face, one more thing we have to do is define that if we don't get anything, then we have to stop, so we've already defined it for this forward and backward, but we didn't define it for speed. , so just to make sure we will write here if let's say x is equal to zero, then our velocity is equal to zero and our error is equal to zero, so our previous error will be zero, okay, that's enough and now we can enter the code for our or print it, actually let's delete this and print these values, so let's delete this and delete this and We'll print the speed value and forward and reverse, so we'll write here print and error and fb, so let's run this okay, so we have an error, so fb is not declared, so let's go back up and does it change everytime? so we can write here fb equals zero, okay, there you go, let's look at the webcam and there you go, so we get the center value and this is, let's remove the other prints because it's annoying, okay, let's try it. again so you can see the first one we're getting, so you can see here, we're going forward so we're going back because it's too close and then if we go back I think it's too big to show here because the value of 6000 is too small so we can try it later, but for the angle we can see that if we move to the left or to the right, you can see that the speed changes and when it is in the center the speed will be close to zero and then it starts to move away, the speed will increase more, so that's the basic idea, but here we are seeing that it goes to negative 180, so let's see why that happens, so here our speed, oh, because we are showing the error, we should show the speed . and not the error okay now if when it's zero it shows -100 it should show zero yeah because we need to print it here so this is right before it goes to our drone and there it goes the speed is zero so that once it is detected there Go and yes it works fine so now we can test it in the real environment but before doing that we need to add the part for our drone so let's go to the image capture and we will copy the code from there . all this and we also need to copy dgi telepi so we will paste it here and delete the cv2 import and then we will go to image capture again and copy these two lines of code or maybe just this one and copy it. from here and if we go down we can delete this, we can comment it out and we can paste it here and that should give us the frame and we can comment this out and we can uncomment this part that will send the values ​​to the drone, so Let's not print anything and then let's test it and see what happens.

Now we also need to add a safety feature so that when we press the q button it lands and we also need to add the takeoff part, so here we will write me. take off point and what I have observed is that when it takes off it is at a lower level than the average height, so what we can do is make it a little bit higher so that it is easier to detect faces instead of going. down and crouching down so we can type here, that's actually let's copy it from the bottom, where is that, there you go so we can copy it from here and we can paste it here and we can say we're not going to go forward back or wink instead. we want to go up and down, so we'll go up at a speed of 25 and we'll do it for, say, the sleep time, we'll do it for two point two seconds, so it depends if it goes up a little bit higher, then you can reduce it if it's lower, you can increase it and then for the landing, we can write something here.

This will be a bit confusing but understand that this part of the code, when we press the q button, it will stop and land, so We are using the cv2 library to do this, so we will write here if cv2 weight key 1 and 0 xf is equal to ord, we will write q, so if the q key is pressed I dot and then we will break, break means we will finish the time. loop so it will end so that's the idea and it should work so let's try it. If you like the video so far, please like it and don't forget to subscribe to this project.

We are going to create a line following the drone. Now the idea behind the line following drone comes from the line following robot so let's try to understand how it works and then we can implement the same methodology in our drone so let's say we have a line following drone that looks like this if we search on Google. The results will look something like this so you can see that we have many different types, most of them have a long chassis and then at the end they have sensors, so it can be three, it can be 16, it can be eight, so you can see a lot of different types here so let's say it's the exact same robot it has these two wheels and we're looking from above and these are the three sensors that can be on or off so if let's say the robot is on the black line then will be on, if it's not on the black line then it will be off, so at this point these two will be off and this one will be on, so let's take a look at the concept, so let's say I want to move. forward here this middle one will be on and the left and the right one will be off so the robot will move forward now once it reaches this point now both the right and the middle one are on because the black line is available on the to the right and on the center sensor, then we can tell it to rotate a little bit now, if it continues moving forward and reaches a point where only this one is left, it means it needs even more rotation, and in that case we will do it. rotate even more to go back to the middle part until you reach this point, then it will keep moving forward and at some point you will see that maybe this one and the one in the center will start to detect the black line, then at that point you will have to go back. counterclockwise, this way, if we have this sensor on, we must turn counterclockwise, if we have this one, we must move clockwise so that we always let's be moving towards the center of our line now if we talk about binary now if we talk about numbers, if there is no sensor uh with the line available, then we can say that it is 0 0 0.

Now, if one of them is on, we can write it and then we can write it as 0 1 0. So we can have all of them. these different conditions where we can say if it is 0 1 0 then go ahead, so let's say we have this, in that case it will be 0 0 1. So if we have 0 0 1, we can go to the right and we will rotate to the right and if we have let's say 1 0 0, which will be this case, then we can say go left, okay, so this is simple, but because we have three different sensors, the possibilities are two to the power of three, which means eight, so which we have a total of eight different possibilities, so we have already written three of them, so here we can say that it is 2 to the power of 3, which is equal to 8.

So we have written 3, so we can also have 0 one and one , so we'll say slides. to the right instead of going to the right when it is like this when it is in the middle it will have zero one and one this way they are both on so we just have to rotate a little bit before it was zero 0 1 we had to rotate a lot now it's 0 1 1 so we have to rotate it a little bit so it's slightly to the right the same way we're going to have 1 1 and 0 so it's going to be slightly to the left and then we're going to have zero zero and zero and then it's going to have one one and one like that which many times uh maybe one of them is not possible like for example one one one maybe not possible and zero zero zero maybe not possible if it does not turn off, but in some cases it may be possible, so we can write, for example stop and stop for these cases so now we have one, two, three, four, five, six, seven cases and the last one is a zero and one so again this should not be possible so we will write stop or It doesn't need to be defined, so here we have eight different possibilities that can happen.

Based on this, we can move our robot to follow this line, so that it turns, then moves forward, turns a little, and then holds. Moving forward this is the main idea so let's see how we can implement this on the drone so here we have the rotation of the drone so let's say our drone is up here and it is moving forward and the camera is facing towards down, so we're looking directly down. Now we're going to use a clip to attach a mirror so you can look down. This clip is available on thingiverse now if you can't print it or don't have a 3D one.

Printer, you can always stick a mirror on the teledrone. I've tried it with the glue gun and it works perfectly fine, but a neater and cleaner way is to use the thingiverse file and this file here is provided by Works of Clay, so thank you. to the guy who actually built this and you can see it can be seen from above if you add an emitter, there you go, I think this is the guy and yeah, so you can print this, it will print pretty quickly and then you can use this to see. down, so this is what we're doing, we're looking down and what we're going to get is an image, so what we're going to do is split this image into three different parts, so we're going to have the center left and the center right, like this that this will be the exact same method of our line follower, so this line follower had these three what you call sensors.

In the same way, we will have three regions where we will detect our black line, so when we get the image, if the image is in the middle, then We will say move forward, then if it is on the right, we will say turn right to let the drone turn to the right and then it will come back to this position and then when it goes a little bit here, it will move. to the left and it will travel like this, this way it will follow the line completely, so the idea is exactly the same as the line following robot.

All we are doing is converting our image into three different sensor zones, so now it is in the center and to the left. Because these are virtual zones, what we can do is divide them into more regions and we can also make them five, we can make them ten, but the problem is that if we have more than three, let's say we have five, then we will have two for the power of five different combinations so that's a lot of combinations to handle so I would suggest going with three and if you want to try it you can try five too.

Now this is part of the rotation because in a line follower. On the robot we can only rotate, so it's a little easier and that should be enough to follow the line, but on a drone, in addition to rotation, we can also translate, so maybe the drone is not here, maybe time you're here, then you're not actually seeing the line correctly. is it like that or maybe it is like that so we have to make sure that the drone is always in the center and then when it is in the center it will be able to tell when to turn correctly;

Otherwise, if you are here, you won't be able to do it. say how to rotate or how much to rotate, then we also have the problem of translation, so this is the idea of ​​translation: when the drone moves at a certain point, we will try to find the center of our black line and where the drone is . so if the detected image or line is right in the center we will keep moving forward, if not if it is skewed we will add some translation towards the center so same thing if it goes this way let's say it is like this. then we will reduce it back to this so at any given point we will measure it so that it keeps coming back like this so this is the idea of ​​the translation so let's start with the implementation so the first thing we will do is right click and go to the python file and we will create the bottom line so we will open it and here we will mainly use the opencv part so we will process images many times so what I will do is use my webcam and once everything is done we can deploy it to the drone, so the idea here is that we are going to set the a4 papers as our path because it is very easy to find the a4 papers and everyone has them, so this will be a very easy way to create a path, so we will use a4 paper, but you can use anything you want.

One good thing about a4 paper is that it's plain white, so we can use color detection to actually detect our white color, so start with that, so what we'll do is type here import cv2, then we'll add our webcam , so we'll write cap equals cv2 dot video capture and we'll put it as one, you can put it as zero if I only have one webcam, I have several so I'm using one, then we'll write while is true and then we'll write the image of underscore equals capital dot reading and that should give us our image from our webcam and then we're going to type here cb2 dot uh key weight one and we also need to show so we're going to type cb2 dot i am show and we're going to say output and then image , so this should give us our image and we can also resize it so it's not too big, so we can type here image equals cv2 dot resize and we'll give it our image and then we'll resize it to let's say 480 by 360 .Again you can also use other values ​​if you want, now the thing is when We will be using the telodrone the image will be flipped vertically because we are using a mirror to find the image so it will be flipped so we need to flip it back so we can write here the image is equal to cv2 dot flip and we want to flip our image in the vertical direction, this is how you can do it, but for now we are going to remove this and later when we implement it on the drone we will add it again.

Okay, by the way, for now I have these images, so I have created these simple images that we are going to try, so this represents the path of the a4 paper and we have a straight line and a curve so we can use the webcam and these images to prove them. Put these images on my website too so you candownload them from there. By the way, everything will be available on my website for free. You can download the code and all the resources from there, so the next step would be to find our way to To do that, we will use the color method, create a threshold function and give it an image that we want to find the color of, so we'll copy this and write here threshold and use the color method. hsv color space to find our color now, if you are not familiar with this again, you can go back to my opencv course and you can check it out.

I highly recommend you do it, but if you don't want to do it, move on. and I will explain as much as I can so let's convert it to hsv so this will be our hsv image so we will write here cv2 dot cvt color and we will write that we want to change this image to hsv the reason why We are converting it to hsv , which is the hue saturation value, because it is easier to find colors in that color space, so we are converting from rgb to hsv, actually from bgr to hsp because opencv uses pgr, so we will write cv2 here. bgr, sorry, color to color bgr. to color bgr to hsv so this is what we will use and then we will give our limits so that we have minimum hue and maximum hue then we have minimum saturation and maximum saturation and minimum value and maximum value so we have to enter these values, but how can we get these values ​​well?

I have written a script again, this is from my opencv course, so this script basically helps us find these six values, so let me create a new file here, color picker, we will create a new file. and we'll paste it, so this is the code again, it will be available on my websites, so what we can do is read it from our capture device, so we can uncomment this and we'll comment it out, so again you can use this color picker . with the drone i also added the code for the drone so when you want to tune the code for the drone you will use this as well so now we are using it for the images so let's run this so there we go so here you can see how I think it's flipped, let me remove the flip, so here, because of the drone flip, I also added the flip here, so it's a little confusing, so we can run it without that and there you have it.

Now you can see my screen, there is black and white, so we need to find this white, so what we have to do is move these values ​​and we have to keep the color white and we have to remove it. You can look here. you have to keep the white color and you have to remove the black, there you go, now it is being removed and there you have it, now we are getting just the white, so you can see here that the white that was here is now gone, so we can increase it. better and we can play with this as well and there you go, so these are the values ​​that we got, what we're going to do is stop this and then we're going to copy our color from here, so we're going to copy all of this. and we're going to go back to our line follower and here we're going to declare a new variable and we're going to call it hsv values ​​so here we're going to call it hsv values ​​or let's say waltz is equal to this so now we can use these values ​​to actually create the minimum and the maximum, so here we will call them white dot dot or we will just call them bottom and then we will say numpy.array, so here we have to import numpy as np, so we will say np dot array. and then we'll add our hsv values, then we have the top equal to a numpy dot array and then again we have to add our hsv values ​​so we can write here the hsv values ​​at zero, then the hsv values ​​at 1 and then we have the hsv values ​​at 2 So we will copy the same from here and paste it here and these values ​​are basically 3, 4 and 5.

So if we want to create that image, the mask image so we can write here the mask is equal to point cv2 in the range. and we will type hsv and we want to use bottom and top, so this will be the image that will only contain the paths, the white parts that we have and nothing else so we can return this return mask, return mask and then we can go. Back here and we can get this mask, the mask is equal to this, so what we can do now is call it image threshold, so what we can do is show this image threshold so we can say path, so let's run this so that it is there.

Wow, now only the white part is shown as you can see and if we go to the curve, you can see even the curve. The only part shown is the curve, so it detects only the white parts, so here you can see the settings. the white portions and now for the white paper for A4 paper will be a little different, so you may have to adjust it again. In fact, let me drop a document and show you what it looks like, so here you can see what it is. the white paper, the a4 paper now again, as I mentioned before, once we're done with the code, we're going to tune it on the drone from the drone camera so it's a little bit different colors, so we're going to tune it.

So, but for now it looks good, but as you can see my carpet is gray and it's also detecting it a little bit, so once we adjust it, it should be fine, so now that we have the image of our path we still don't we have her. to know where the center is or how much curve there is, for this we need to convert this into a box and from that box we will get the center point, so we will create a bounding box around this path and we will get the center point. from it and based on that center point we can translate it, so if it is to the right we will go back to the center again, if it is to the left we will go back to the center, so we will try to keep it in the center, this is how we do it.

This we will create a new function and we will call it get contours and we will send our image threshold and we will also send our actual image in which we want to draw the bounding box, so let's create this function get contours and we will have an image threshold and we will have the image okay, so the first thing we will do is get the contours so we will use the cv2 dot find contours function now the contours are basically edges so we are looking for the edges in our image and based on them. edges we will find the path or the bounding box, so here we will write that we want to find it in our image again.

If you want to go into detail on how this works you can check out my opencv cv2 dot course, let's use the external method which will give us the outer region and the cv2 dot string approximation again you can use simple or none let's use none and what this is going to return is it's going to return contours with tours and it's going to return us in hierarchy now we're not interested in the hierarchy, but because it's going to return us, we're just going to unpack it here, okay, so we have to find the largest contour of this because that will be our path.

We are assuming that the largest contour or largest region is found. will be our way so here we will say that bigger equals by the way these contours are basically the dots or the dots actually let me draw the contours so you can see what it is, let's delete this so let me draw like this cv2 dot drop contours and we will give our image that we want to draw on that and then we will draw all the outlines and this one minus one means that you have to draw them all and if they are only one or two it will draw the first one or the second one and so on, then you have to define the color, like this let's set it to purple, purple and set the thickness to seven, so let's run this and see what happens.

We're getting an error, let's see what that is, look, we found contours. something is missing, I think um, oh okay, here it shouldn't be an image, it should be an image threshold, so we need to find it from the image threshold. We just got the route from here. Let's run it again and voila, now you can. See the outline is getting the outline this way, now the problem is that my computer also shows the white regions on the edges so it's a bit problematic otherwise if I zoom in you can see they are getting the edges or let me. try the other one, there you go, so now you can see that it is detecting the path the right way, so if I go here or here, if I go back, it will also get the largest one because I have white regions in other places, but if it is expanded.

It should be fine on it, so while we use it with the drone, the background should be a different color so that it is easily detectable. Now that we have these contours, we need to get the largest one, as you saw. we can get the largest by writing largest equals maximum contours and what is the key that we want to use, the key will be area, so we will write cv2 point contour area of outline, this will give us the largest outline of the points and then we will convert all these points because it will have a lot of points that we don't want, what we want is the bounding box x y and the width and height, so we will change all these points to just a bounding box or a rectangle so we will say that x and we can also find the center of this, so the center x is x plus the width divided by 2 and the center y is y plus h divided by 2.

So these will be our points and as we did before, we can draw these points cb2 dot what do we do? you call it a circle, we'll draw it on our image, we want the center to be cx and cy, we want the radius to be, say, 10, then we want the color to be, let's say, green and then we want the thickness to be cv2, dot fill , so that should work, what else and we can change this instead of drawing them all, we can only draw the largest one, so there you have it, now we also get the center, so if we zoom in correctly, you can see that the center is here.

So if we get something like this, we'll say our main center is here somewhere in the center of the image, so we have to backtrack until it's in the center, so it's going to be like this and the drone is going to move like this, using this. let's check the other one there you go, you can see we are getting the center, so using this method we can find the deviation and from that deviation we can tell the robot to go left or right, so do we want to draw this? Yes, why not? Let's leave it there, you can also draw the rectangle using these points.

Point rectangle cv2, you can use that to draw it, so all we need from this function is the cx, so we'll return cx and we'll come back here and get the cx. value here, so now we have done the first part, which is basically for translation, so I can write here, this is for translation. Okay, now we need something for the rotation, so since we discussed the three sensor method, we're going to use that, so we're going to call. this like get output from sensor we will create a function called get output from sensor we will send our image and we will send the number of sensors that we want to use so that the image has the same image threshold and the number of sensors is three so now also you can change them, but for now we'll keep it at three, let's copy this and create a new function here, so now the idea is that we want to divide our image into three different portions, so if we have three sensors, we want to divide it into three different sections to that we can use the numpy library because these are arrays, we can use the numpy responsibility to split it, so there is a very nice function for that, let's say our images are called images. is equal to the horizontal di

vision

of numerous points because we want to divide it horizontally, we will give our image what the threshold of the image is and then we will give it how many numbers we want, so we want sensors, which are three, we want to divide it in three ways.

Now one thing is that you need to make sure that when you divide the image size here, the width of the image should be divisible by three. If it is not divisible by 3, you will not be able to divide it into equal portions, so you have to make sure that 480 is divisible, so it should not be a problem, so what we can do is check if it is working, we can print cv2 dot I am show and we can write, let's say we can create a for loop here. for the image or let's say I am in the images we want, we want to display the image and we have to give a unique name so that we can list it, it lists and we can get the value of x, so here for the name we can put the value. of x we ​​can convert it to a string and we can write here I am, so every time it repeats it will create a new image, so we will have three images to display, so let's run this and see what happens.

There you go, so now I can see, so this is our main image and then we get these three converted images, so let's say let me change this screen, let's say it's in the center or let's say it's on the right, uh, it's a little to the left. bit, then the one on the rightadd a little bit of this there you have it and then maybe a little bit of this okay these are the values ​​we've found it and it should be fine so we can stop it we can copy it from here and we can go to the line follower and we can paste it into the hsv here if we run this now, there you go, now we're getting our path, this is the center of our path and as you can see we have a little bit of right and left and then we have the forward speed, no there is curve, but if we rotate it, we should get some curve, so now you can see we have a curve, if I go to the other side, you can see that the curve value is changing the last one here 15.

So if I go back and I think the battery died, okay, anyway, the idea is that everything is working fine, all we have to do now is test it, we'll take it off and run it and see what happens.