Line following robot with OpenCV and contour-based approach

4 min readOct 23, 2018

In my previous story, I told about PiTanq — a robot-tank I built. Then a big goal is to make it autonomous. I was inspired by Udacity course for self-driving cars and the first task of this course is to recognize lanes on roads.

As my robot is pretty far from hitting a road I put a white tape on the floor.

The idea was to implement tracking a line based on lanes detection approach from the Udacity course.

A general solution for this task would look like:

Filter an image by color
Find edges with Canny edge detector
Crop irrelevant parts of the image
Detect lines with Hough transform

Filter image by color

Udacity course task requires to detect both yellow and white lines. For that purpose, they use HSV or HLS conversion. I need to detect only a white line so I decided to use only a grayscale filter.

Next step I picked a threshold to have a binary image:

Detect the line

Then, applying Canny edge detector, I got that picture (inside a region of interest):

Using Hough line detector I found some odd lines:

Making detection rules more restrictive:

Not good in terms of relevance (and reliability as well).

Spent some time with these experiments I was not able to get satisfying results. Then I decided to try another approach. Instead of lines detection, I used contours detection. Assuming, the biggest contour is the line and taking a centerline of the bounding box, I got a plausible direction.

Light problem

The next problem I encountered with, was different light conditions on the line. One side of the line turned out in the shadow of a couch and it was impossible to find a grayscale threshold working along the whole line loop.

The solution was to adjust the threshold individually for each image depending on a ratio of white pixels to the whole area.

def balance_pic(image):
    global T
    ret = None
    direction = 0
    for i in range(0, tconf.th_iterations):
        rc, gray = cv.threshold(image, T, 255, 0)
        crop = Roi.crop_roi(gray)
        nwh = cv.countNonZero(crop)
        perc = int(100 * nwh / Roi.get_area())
        if perc > tconf.white_max:
            if T > tconf.threshold_max:
                break
            if direction == -1:
                ret = crop
                break
            T += 10
            direction = 1
        elif perc < tconf.white_min:
            if T < tconf.threshold_min:
                break
            if  direction == 1:
                ret = crop
                break
            T -= 10
            direction = -1
        else:
            ret = crop
            break  
    return ret

Make driving decisions

Based on computer vision technics we got a direction to move. The real decisions were made depending on the angle of this vector and its shift from the image middle point.

Determine turn actions (if required):

def check_shift_turn(angle, shift):
    turn_state = 0
    if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle:
        turn_state = np.sign(90 - angle)
    shift_state = 0
    if abs(shift) > tconf.shift_max:
        shift_state = np.sign(shift)
    return turn_state, shift_statedef get_turn(turn_state, shift_state):
    turn_dir = 0
    turn_val = 0
    if shift_state != 0:
        turn_dir = shift_state
        turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step
    elif turn_state != 0:
        turn_dir = turn_state
        turn_val = tconf.turn_step
    return turn_dir, turn_val

Do self-driving:

while(True):
            a, shift = get_vector()
            if a is None:
              # there is some code omitted related to line finding
              break
            turn_state, shift_state = check_shift_turn(a, shift)
            turn_dir, turn_val = get_turn(turn_state, shift_state)
            if turn_dir != 0:
                turn(turn_dir, turn_val)
            else:
                time.sleep(tconf.straight_run)

Results

There is a debug visual info:

Algorithm settings

## Picture settings# initial grayscale threshold
threshold = 120# max grayscale threshold
threshold_max = 180#min grayscale threshold
threshold_min = 40# iterations to find balanced threshold
th_iterations = 10# min % of white in roi
white_min=3# max % of white in roi
white_max=12## Driving settings# line angle to make a turn
turn_angle = 45# line shift to make an adjustment
shift_max = 20# turning time of shift adjustment
shift_step = 0.125# turning time of turn
turn_step = 0.25# time of straight run
straight_run = 0.5# attempts to find the line if lost
find_turn_attempts = 5# turn step to find the line if lost
find_turn_step = 0.2# max N of iterations of the whole tracking
max_steps = 100