HOWTO create a Line Following Robot using Mindstorms 52

The easy way of learning how to built your own Line follower Robot using a PID Controller without any of the mathematical expressions that you will find on most tutorials. Code included!

The first thing you have thought about after unboxing your LEGO Mindstorms was building the first robot, and just after that you would love to make it follow lines, isn’t it?

mindstorms-line-following-light-sensor

Line following is one of the most common problems on industrial robots, and it is one of the most useful applications because it allows the robot to move from one point to another to do tasks.

There are several ways of making a Line Follower, the one that I am going to explain you about is using the Light sensor. As you know, both Mindstorms and EV3 sets come with a little light sensor that it is able to get a reading of reflected light, apart of seeing colors. On this tutorial I will explain how to do line following with just one sensor. The more sensors you have the better and faster the robot will be able to follow it.

Building the line follower robot

So first thing is build yourself a little robot much like Track3r, either with wheels or with tracks. You can download the instructions provided by LEGO. It is a simple construction.

TRACK3R_large

Or base it on one of the LEGO Education models…

45544_Brick two

Whatever you build, just try to keep the distance between wheels to a minimum… because the bigger the distance, the harder for the robot to follow the turns on the line.

Ok, ready? Time to code. Let me explain how we get to the “best” solution with a serie of intermediate steps that will help you understand it better.

Building your playground for line following

Ok, the robot is done. But before start coding, we need the line that the robot will follow. If you happen to have the Mindstorms NXT 2.0 it has a mat with a big elipse-like line that works great for this tasks.

But if you don’t just do like me. I have used a black tape and with the finger I have sticked it to the floor creating a continuous path that the robot will follow. You don’t need a closed loop ( although it is a good idea to do it in that way ).
My floor is done of marble that it is white and brown at times and even with that it works. So it may work too on yours unless it has even less contrast than mine.

Line Following Problem definition

It is quite important to understand the line following problem first. So let’s describe the problem.
We have a thick black line on a white surface and we want our robot to move along the line following it in the fastest possible way. Right? Well, first thing that we need to understand is that we don’t want to follow the line ( wtf?! )…

No, serious, we don’t follow the line but its border in what it is called left-hand approach.

We want to follow the line where there is a 50% of white and a 50% of black.

midpoint-light-sensor-mindstorms-line-following

So, next step is defining what it is black and what it is white. I hope you have a clear idea of what these two colors are, but unfortunatelly your robot don’t. So the best thing you can do, before starting anything else is calibrate the robot.

Light Sensor calibration

Ok, as you know Color sensor can also work as a Light sensor, so we choose the Measure reflected light mode and we are going to store in two variables the white and black colors.

The reflected light value is just a number between 0 and 100 with the amount of light the sensor is getting back.

So the pseudocode would be


function CALIBRATE

	print "WHITE?"
	Wait for Touch Sensor to change
	white = Read Light Sensor
	
	print "BLACK?"
	Wait for Touch Sensor to change
	black = Read Light Sensor
	
done function

Do you get the idea? We add a Touch sensor to our Robot to record the light value, you can also do it using Brick buttons, as you prefer.
Here is the EV3 code that I used for it

light-callibration

So the idea is that you place it on the white surface, press the touch sensor, place it now on the black surface and press the touch sensor again, now we have the white and black readings and can start working.

I do it each time I start the robot but you can safely ignore it while light conditions keep stable.

Line Following with On/Off Controller

Ok, we have the robot, we have the calibration data… let’s go code. MaaaaaaahHH! Bad!

Let’s think first what we are going to do. Let’s start with the simplest possible way ( and perhaps the worse ) of doing line following.

We place the robot on the line, we get a reading if it below the middle ( black-white ) we move to one side and if it is above we move to the other side. Simple? Clever?

Let’s see again the pseudo-code


program LINE FOLLOWING

	white = 0, black = 0
	CALIBRATE()
	
	midpoint = ( white - black ) / 2 + black

	repeat
	
		value = Read Light Sensor
		
		if value < midpoint then
			motor B set power 50
			motor C set power 25
		else
			motor B set power 25
			motor C set power 50
		done if
		
	done repeat
	
done program	

The idea is pretty simple just make one wheel turns faster than the other. Here it is how it works.

Here is the EV3 program

on-off-controller

( You can download all the source codes on the bottom of the page )

Does it works? Well... it does... to some point. If the corner is step enough the robot will miss it and as you can see it is missing the straight line and it starts oscillating around it.

Why? Because we have only two states, so the robot is either turning left or turning right. What can we do? Exactly... add an extra state to make it go straight. So we have left, straight and right... so if we want to improve it even more we could add even more states, far left, left, a bit left, straight, a bit right, right, far right, and so on...

So why not make the turn proportional to the error, the difference between the midpoint and the value read.

Line Following with a P Controller

Do you like Maths? I don't. I have a deep understanding of them but I really can't stand the complex way of explaining simple things... so let me explain you the P Controller without any kind of mathematical notations.

If you follow the reasoning of the On/Off controller with several states you may end up thinking of a controller with infinite states, each one of them with an value proportional to the error, the difference between where we want the robot to be and where it really is.

If you want something to be proportional to a quantity you just multiply both factors so we have


how much to turn = Kp * ( midpoint - Light Sensor Reading )

Where K is a constant that helps us tune the P controller. The bigger Kp is the higher will be the turning when there is an error, but if Kp is too big you may find that the robot overreacts and that it is uncontrollable.

You can watch what happens when you change the value of Kp from 0 to 10 on this video.

So start with 1.

So our P-Controller would be like this pseudo code


program LINE FOLLOWING

	white = 0, black = 0
	CALIBRATE()
	
	midpoint = ( white - black ) / 2 + black
	kp=1;

	repeat
	
		value = Read Light Sensor
		correction = kp * ( midpoint - value ) 
		
		Turn B+C Motors by correction
		
	done repeat
	
done program	

If you are not using the EV3 module just move one motor a value + correction and the other motor to value - correction. It is pretty much the same.
Here it is the EV3 program

p-controller

( You can download all the source codes on the bottom of the page )

Tunning the Kp parameter

Start with Kp=1 and run your robot. If the robot can't follow the turns on the line then increase Kp, if on the other hand your robot turns violently on the line, decrease the Kp value. The bigger the effect the bigger the change need on Kp. Just keep playing...

The P Controller has two problem that can be "easily" fixed with a complete PID controller: one is that it will oscilate after corrections and the other is that there is a small error that it is a direct result of the P Controller.

So let's see the complete PID controller.

Line Following with a PID Controller

A PID controller is basically a P controller with two other components, the integral part and the derivative part.

pid.jpg

Don't be scared. Let me explain the overall idea and we move to the pseudo code.

Proportional part of the PID Controller

This is exactly the part that have just seen above.

Integral part of the PID Controller

The idea behind the integral part is that the error must tend to zero over a period of time. I will avoid you a very ugly mathematical expression explaining this... but keep in mind that as you are working inside a loop we can write

integral = integral + error

So over a number of interation we want this to be zero. Why? Because we want to correct the error, so the only way of the robot being exactly over the midpoint between black and white is that this part of the PID controller is zero too.

So our controller would be like

controller = kp * error + ki * integral

As you can imagine the only way for the integral part to be zero is to cancel the error by applying another negative error. So the robot will move from one side to the other of the midpoint in order to cancel the error and be exactly on the midpoint.

Derivative part of the PID Controller

The final part of the PID Controller is the derivative part. If we can think of the integral part as a memory of the controller, the derivative part is like an economist that watch the future and try to guess it based on past errors.

So if we know the last error and we know the actual error we can compute what the next error will be. So the derivative part of the PID controller will help the controller to correct the error faster.

One problem of the derivative part is that it is highly sensible to the noise in the controller signal ( our light detector ) so if the signal is noisy it may not estimate the correct error. And yes, light sensor is quite noisy :)

So the final PID Controller is like...


error = midpoint - value 
Turn = kp * error + ki * integral + kd * derivative

Pseudocode for PID controller

So this is the pseudocode for this PID controller


program LINE FOLLOWING

	white = 0, black = 0
	CALIBRATE()
	
	midpoint = ( white - black ) / 2 + black
	kp = 1, ki = 1, kd = 1
	lasterror=0

	repeat
	
		value = Read Light Sensor
		error = midpoint - value
		integral = error + integral
		derivative = error - lasterror
		
		correction = kp * error + ki * integral + kd * derivative
		
		Turn B+C Motors by correction
		
		lasterror = error
		
	done repeat
	
done program	

And well I won't be a true nerd if I don't show you the crazy mathematical description.

So how good does it works?

Disappointed? Keep reading...

Mindstorms EV3 PID Controller Program

So, here it is the EV3 code for the PID controller.

pdi-1

And here is the EV3 file...

Ok, got it? Now a big problem appears... what are the best values for kp, ki and kd?

How To set kp, ki and kd params in a PID controller

Well... again, I will save you all the mathematical representation of it. So after reading Wikipedia and reading about various approaches, I think the one that best suits our tasks it the heuristics approach.

So copying from Wikipedia: "If the system must remain online, one tuning method is to first set K_i and K_d values to zero. Increase the K_p until the output of the loop oscillates, then the K_p should be set to approximately half of that value for a "quarter amplitude decay" type response. Then increase K_i until any offset is corrected in sufficient time for the process. However, too much K_i will cause instability. Finally, increase K_d, if required, until the loop is acceptably quick to reach its reference after a load disturbance.

However, too much K_d will cause excessive response and overshoot. A fast PID loop tuning usually overshoots slightly to reach the setpoint more quickly; however, some systems cannot accept overshoot, in which case an over-damped closed-loop system is required, which will require a K_p setting significantly less than half that of the K_p setting that was causing oscillation."

Got it? No problem, I am here to help you.

Just set Kd and Ki to 0. Set Kp to 1 and run your robot. Does it follow the line smoothly? No, try decreasing it by a half? Can it still follow the line? No, increase it a bit... keep changing until you have the robot following a straight line without oscillations.

Then increase Ki a bit and check if it is smoother? Yes, increase a bit more... does it works worse... so a bit less...

Finally set Kd to 1 and check how fast the robot follows now the line? Too unstable, decrease it...
Keep repeating until you are happy.

How different Line following approaches compare

One video is more than enough to see it :)

Miguel says...

As a final word, you now need to know that each kind of line require a Kp, Ki and Kd params tunning to get the best of the robot.

That's all folks.
Hope you enjoyed the tutorial as much as I did. Don't miss my post about a self-balancing robot that I built using a gyro sensor I have just got from LEGO. Did you know that I run a Youtube channel with almost daily LEGO reviews and HOWTO? Please subscribe to The Technic Gear LEGO Reviews

Downloads

Download Mindstorms EV3 PID Controller.

Ads by Google

52 thoughts on “HOWTO create a Line Following Robot using Mindstorms

  1. Pingback: Asi se programa un robot LEGO para que siga lineas [EN]

  2. Reply Tristan Vukosich (SuperSluether) Mar 13,2014 8:58 pm

    A very informative and well-said post. Thank you for your help! Now I understand how line following works. Just so you know, there is a program that does something very similar on http://www.nxtprograms.com, but this post does a lot better job explaining how to program your own.

    • Reply Miguel Mar 13,2014 9:50 pm

      Thanks for your kind words.
      I remember reading it when I built almost a year ago the Segway robot… I understand not enough and ended up copying the code and uploading it to the brick.

  3. Pingback: The Best Ten LEGO Technic Sets ← LEGO Reviews & Videos

  4. Pingback: Beacon tracking with LEGO Mindstorms EV3 IR Sensor ← LEGO Reviews & Videos

  5. Reply EV3Newbie Mar 23,2014 6:42 am

    Thanks for an informative post. I’m new at EV3 but I have a basic math question/point. In your EV3 code for the P-Controller, you have the equation as “1*(a-b)” but I don’t understand the point of having the multiplication at all. Isn’t “1*(a-b)” exactly the same math outcome as just “a-b”? You have something similar in another post (IR beacon following) where your equation is “0.5*(a-0)”. In this second example, subtracting zero does nothing, unless I’m totally missing something?

    • Reply Miguel Mar 23,2014 9:05 am

      Well, let’s see the two parts:
      a) The EV3 P-controller, yes. 1*(a-b) is the same that (a-b) the point of having a 1 there is that it would be a Kp constant that in this case is 1 but on others can be 0.5, 7 or 124. So why multiply? For making clearer that there is a Kp constant with value 1.

      b) On the IR Beacon, yep completely right. Again, by definition a P-controller is a controller that acts proportional to the error, and error is defined by ( reading – goal ). I want heading, the goal to be 0, so it is there to make it clearer that it is just a P-Controller.

      Hope it helps.

  6. Reply Steve Apr 4,2014 12:59 pm

    Thanks for your guide! I have a couple of questions. How do you set up the Kp,Ki and Kd parameters in the PID Controller? I tried changing the number on this equation “0.07*a+0.3*b+1*c” but I couldn’t figure out whats happening? Also, I’m working on the WRO 2014 High School(Space Station) challenge and I want to change the line-following so it does’nt only follow the black line but also some gray areas and the green area at the starting point. Can you give me some tricks? Thank you in advance!

    • Reply Miguel Apr 4,2014 4:47 pm

      It is set by trial and error. It is explained in the final part of the article. I am writing a special article about how to set them up and a program to configure the robot in an easier way.

      It can be tricky… one way would be to just poll again the Color sensor but this time in color mode instead of reflected light intensity. I am myself working on a similar project ( but for fun )

      • Reply Steve Apr 16,2014 12:01 pm

        Do you set up the Kp,Ki and Kd parameters in this “0.07*a+0.3*b+1*c” equation? And which number is Kp, which Ki and which Kd?

  7. Reply Tim Apr 15,2014 7:43 pm

    Thanks for the very nice code and explanation. After looking through the programs I have a couple of questions
    1) Why the motor reset block in the bang-bang controller?
    2) Why the light sensor calibrate blocks on the PID controller? Does it actually give you more dynamic range in the reflected light measurement, or is it somehow just easier if the target intensity is about 50?

    • Reply Miguel Apr 16,2014 10:27 am

      1) Yes, because I attached an extra motor to control the P-value. I am still writing the article.

      2) Calibration will make your robot works best under any condition, you can’t assume white to be 100 and black to be 0. Just calibrate and let the robot guess them :)

  8. Reply Flavio Andrade Apr 15,2014 11:23 pm

    great article, help me a lot.. tnks…

  9. Pingback: The LEGO Mindstorms EV3 Laboratory Review ← LEGO Reviews & Videos

  10. Reply Laverato Mosler May 7,2014 3:29 pm

    hi, I’m come from Viet Nam, can I ask you how to loading the program into the robot sd card, i just know control the robot by bluetooth and I use pascal. Can you help me?

    • Reply Miguel May 7,2014 4:33 pm

      Ummm… as far as I know you can’t use Pascal to program the EV3… You can use NXC, Java, C# or even the EV3 own language… just use any of the tools provided there on each of the languages to upload the program to the brick.

  11. Pingback: Draw a Square: Using Motors in leJOS EV3 ← LEGO Reviews & Videos

  12. Reply Rebecca May 25,2014 5:45 am

    My son (age 9) and I are new to the lego mindstorms- he built the lego mindstorms ev3 (took him 4 hours). The code written by the designer looks like greek to us both. We are attempting to get this robot to follow a black line maze. We literally spent over 13 hours today and can only get it to move backwards with a simple code we designed. Any suggestions would be greatly appreciated.

    • Reply Miguel May 25,2014 8:46 am

      Well, in the article there are three ways of doing it… from simplest to hardest. The maths of the article are a bit hard for a 9yo but you should be able to handle them: just multiplications and divisions.

      The code is available to use but you should adapt it to your robot ( or use one like mine ).

      Anyway, cool thing about Mindstorms is that you are learning everyday with it.

  13. Reply Tsvi Katchalski May 27,2014 4:08 pm

    Hi, Nice job here. We tried the code and it worked well. We now added transmission gear to the two motors making it run x4 times faster. PID fails running with same params and we can’t seem to be able to tune them with reasonable results. Running power down to 20% (returning to original speed) still is uncontrollable. Any suggestion?

    • Reply Miguel May 27,2014 4:31 pm

      Yes, I have suffered it myself when I tried my robot with bigger wheels.
      The problem here is that the light sensors see a very small area so when you increase the speed the area the sensor see moves too fast when there is any kind of correction.
      For faster speed you need two or more light sensors… as a rule of thumb… the more the faster. I may give it a look in a few months… when I write the linefollowing in java using leJOS.

  14. Reply John Jun 24,2014 2:44 pm

    Hallo , there is something I didn’t understand in the integral , in the EV3 program , you did intergal*0.5+eror = intergal

    • Reply Miguel Jun 24,2014 7:14 pm

      Yeah, because if you don’t divide the integral ( making the error smaller on each iteration ) you will have a robot that once the error is too big it will be unrecoverable and it will start spinning over itself.

      Give it a try and change 0.5 by 1 and let it miss the line just a bit.
      We are just making the robot to forget about errors that happened a few iterations ago and to concentrate on the actual error instead…

      • Reply John Jun 24,2014 8:43 pm

        Thanks for your answer and also really thanks for the guide , its allot of help for me!
        in the explenation you didn’t write anything about dividing it and also in the pseudo code its with no devision , just intergal = intergal + eror.
        which i didn’t really understand also..
        I would really appreciate if you can try explaining this to me.. say the first eror is 5 , the intergal would be 5 , then in the next time if the eror would be 3 the intergal would be 8? how is that helping to decrease the eror? I know it may be stuped questions im a teen and i would really aprpreciate you trying to help me.

        • Reply Miguel Jun 27,2014 2:46 pm

          Oh, it isn’t stupid at all… what you are exposing is what it is known as integral windup… the error goes up and up and up and there is no physical way for the robot to correct that error.

          The idea of the integral is that using a proportional controller you would eventually reach a state of zero error. The integral just makes that state to be reached earlier by adding error after error forcing so the proportional part to correct it sooner.

          http://blog.opticontrols.com/wp-content/uploads/2011/03/P-PI-PID-Responses.png

          In the above image you can see that P controller gives a base error during time and I and D parts of the controller just make the controller to reach the zero error sooner.

          Hope it helps.

  15. Reply John Jun 24,2014 2:58 pm

    Also , I didn’t really understand that part of intrgal.. if the eror for example would be 5 what would the intergal be?

  16. Reply Matheus Paracampos Jun 29,2014 12:46 am

    (Sorry for my english, i’m brazilian and needed use Google Translator) Nice work guy! You really helped me alot. Before I just know how P works. With your help I understand I and D. Congratulations and thank you!

  17. Reply uzziel Jun 30,2014 5:33 pm

    Thanks for sharing your knowledge in EV3! as I can download the code?
    because pressing on only rare signs me out

  18. Reply Jordan Ryan Curry Jul 3,2014 4:49 pm

    Hey a group of high school students are using the ev3 as a research project. we read the article and it has help so much. but we came up with a problem. for some reason it wont turn right. any suggestions to what could be wrong?

  19. Reply spicy777 Jul 5,2014 4:13 pm

    Hey, a group of high school students are building a line follower for a research project. We thought you article was really helpful and made us understand the while concept so much better. But we came up with a problem. Our line follower wont turn right while following the line. It will correct itself when it hits the white on the right side of the black line. But it will not correct itself right when on the left side of the black line. Do you have any idea what could be wrong?

    • Reply Miguel Jul 7,2014 7:17 pm

      It sounds like something wrong with an absolute value. Try printing the values of the P(ID) controller to see why it isn’t doing negative numbers…

  20. Reply Matheus Paracampos Jul 23,2014 1:34 pm

    Hey Miguel, can you answer me? I don’t understend the part of Integral. I know that integral is error+integral=integral. But on your code you multiplies integral by 0.5 and save this like be integral. What this number do in the code? Thank you to the help!

    • Reply Miguel Jul 23,2014 5:10 pm

      It is explained a few comments above this one. The idea is to make the PID controller “forget” about old errors and correct the actual one. Otherwise you can end up with a integral wind-up

  21. Reply Mezello den Aug 14,2014 3:07 pm

    Do you set up the Kp,Ki and Kd parameters in this “0.07*a+0.3*b+1*c” equation? And which number is Kp, which Ki and which Kd?

  22. Reply raaja Aug 17,2014 2:13 am

    which follower is fastest one?

  23. Reply Ivan Yiu Aug 19,2014 5:52 pm

    Hello, sorry for my poor english
    Your article is really inspiring
    I have learnt pid controller few years before and your article still teaches me a lot.
    I am working a tough project about ev3 and i have couple of troublesome questions to ask:
    First, the project require my light sensors place in a high position (i think is about 2.5cm above ground) and as you know, light sensor doesnt work well in this height, it only give me about 25 degree range(from white to black). Eventhough i am using two light sensors pid follower(e=s1-s2+sensorDiff), it still cannot work well(swing). I have tuned the pid constant for long time and it still doesnt work. So i want to ask do you think it ia possible to make pid controller work well under a small range of light value? If yes, what method you would suggest me to tune the pid constane?=)
    Moreover, for this strange problem, i have some idea but i am not sure about the answer.
    1. How does the distance between sensor and wheel(i am using the most commom big wheel) affect the performaoine of the robot?
    2.for the integral part, instead of multplying the integral error by 0.xx. i reset it to zero when the error is in the range of -1 to 1. What is the difference between this two methods and, which method do you think it is more suitable to my case?
    Sorry for a long comment with poor english =p hope you can reply me soon thankyou!

    • Reply Miguel Aug 21,2014 6:27 pm

      Let’s start with the simplest one:
      2) I think the approach of multiplying for 0.xx is better that abruptly setting it to zero when it is outside of the range… but I don’t really think it is a big deal.

      1) I tried myself a bigger wheels robot with the same PID controller and it didn’t worked and even it was QUITE hard to tune it for following a straight line… I think that the bigger the wheel the higher the sensor should be… but you have the problem with distance and sensor sensibility.

      I want to give it another try on the future for a faster line tracking robot… but my advise is to get a decent Kp value before attempting to play with Ki and Kd.

      Hope it helps.

  24. Reply CoolAlex Tan Aug 21,2014 5:40 am

    miguel, can u tell me more about kp? i sitll no understand about the kp very well

    • Reply Miguel Aug 21,2014 6:31 pm

      kp is just a number that makes the robot correct the error between where it is and where it should be ( over the line ) so that the correction is PROPORTIONAL to the error… the bigger kp is the more aggressive the correction will be.

  25. Reply CoolAlex Tan Aug 21,2014 5:49 am

    if i copy the program given for the p controller, does it work without any changes

    • Reply Miguel Aug 21,2014 6:28 pm

      Yes, it should as long as your robot is similar to mine… and that the sensor is close to the center of the line that joins both wheels.

  26. Reply Daniel Aug 24,2014 9:03 pm

    Miguel, I read your article and found it very good, but I still have some questions. First of all, in which language does the codes you wrote are? It isn’t Java, right? Secondly, the EV3 Color Sensor are set to mesuere the reflected light intensity, so it can see in which part of the line it’s? And one more thing, I didn’t understand why we have to add the black to the average when setting the midpoint.

    • Reply Miguel Aug 26,2014 9:48 am

      1) No, it is Mindstorms graphic language…
      2) Yes, exactly. You are suppossing that black is 0 but it isn’t true most of the times… so you either do (white+black)/2 or write in a more complex but clearer (white-black)/2+black both ways is the same.

      • Reply Daniel Aug 27,2014 12:20 am

        The second part I understood, but my first question was about what you called “pseudo codes “.

        • Reply Miguel Sep 5,2014 4:47 pm

          Oh, just something close to natural language that it is simple to understand and follow yet describes the process of designing the algorithm accurately.

  27. Reply CoolAlex Tan Aug 29,2014 11:22 am

    How to adjust programming if the color sensor is not close to the map

    • Reply Miguel Sep 5,2014 4:46 pm

      Not really sure… this is the part that I have to do myself… but I would guess that as the change from black to white is now a smaller range I would reduce P until it works.

  28. Reply CoolAlex Tan Aug 29,2014 12:22 pm

    Miguel I tried but it cannot follow the straight line

  29. Reply Leszek Sep 8,2014 6:58 am

    Pretty cool idea, thanks for sharing.

    BTW, the “easiest” way to tune a PID controller is the Ziegler-Nichols method (http://en.wikipedia.org/wiki/Ziegler%E2%80%93Nichols_method). It might be tricky to pull out with your robot though.

Leave a Reply