Line following robot with OpenCV and contour-based approach

In my previous story, I told about PiTanq — a robot-tank I built. Then a big goal is to make it autonomous. I was inspired by Udacity course for self-driving cars and the first task of this course is to recognize lanes on roads.

As my robot is pretty far from hitting a road I put a white tape on the floor.

The idea was to implement tracking a line based on lanes detection approach from the Udacity course.

A general solution for this task would look like:

• Filter an image by color
• Find edges with Canny edge detector
• Crop irrelevant parts of the image
• Detect lines with Hough transform

Filter image by color

Udacity course task requires to detect both yellow and white lines. For that purpose, they use HSV or HLS conversion. I need to detect only a white line so I decided to use only a grayscale filter.

Next step I picked a threshold to have a binary image:

Detect the line

Then, applying Canny edge detector, I got that picture (inside a region of interest):

Using Hough line detector I found some odd lines:

Making detection rules more restrictive:

Not good in terms of relevance (and reliability as well).

Spent some time with these experiments I was not able to get satisfying results. Then I decided to try another approach. Instead of lines detection, I used contours detection. Assuming, the biggest contour is the line and taking a centerline of the bounding box, I got a plausible direction.

Light problem

The next problem I encountered with, was different light conditions on the line. One side of the line turned out in the shadow of a couch and it was impossible to find a grayscale threshold working along the whole line loop.

The solution was to adjust the threshold individually for each image depending on a ratio of white pixels to the whole area.

def balance_pic(image):
    global T
    ret = None
    direction = 0
    for i in range(0, tconf.th_iterations):
        rc, gray = cv.threshold(image, T, 255, 0)
        crop = Roi.crop_roi(gray)
        nwh = cv.countNonZero(crop)
        perc = int(100 * nwh / Roi.get_area())
        if perc > tconf.white_max:
            if T > tconf.threshold_max:
                break
            if direction == -1:
                ret = crop
                break
            T += 10
            direction = 1
        elif perc < tconf.white_min:
            if T < tconf.threshold_min:
                break
            if  direction == 1:
                ret = crop
                break
            T -= 10
            direction = -1
        else:
            ret = crop
            break  
    return ret

Make driving decisions

Based on computer vision technics we got a direction to move. The real decisions were made depending on the angle of this vector and its shift from the image middle point.

Determine turn actions (if required):

def check_shift_turn(angle, shift):
    turn_state = 0
    if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle:
        turn_state = np.sign(90 - angle)
    shift_state = 0
    if abs(shift) > tconf.shift_max:
        shift_state = np.sign(shift)
    return turn_state, shift_state
def get_turn(turn_state, shift_state):
    turn_dir = 0
    turn_val = 0
    if shift_state != 0:
        turn_dir = shift_state
        turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step
    elif turn_state != 0:
        turn_dir = turn_state
        turn_val = tconf.turn_step
    return turn_dir, turn_val

Do self-driving:

while(True):
            a, shift = get_vector()
            if a is None:
              # there is some code omitted related to line finding
              break
            turn_state, shift_state = check_shift_turn(a, shift)
            turn_dir, turn_val = get_turn(turn_state, shift_state)
            if turn_dir != 0:
                turn(turn_dir, turn_val)
            else:
                time.sleep(tconf.straight_run)

Results

There is a debug visual info:

Algorithm settings

## Picture settings
# initial grayscale threshold
threshold = 120
# max grayscale threshold
threshold_max = 180
#min grayscale threshold
threshold_min = 40
# iterations to find balanced threshold
th_iterations = 10
# min % of white in roi
white_min=3
# max % of white in roi
white_max=12
## Driving settings
# line angle to make a turn
turn_angle = 45
# line shift to make an adjustment
shift_max = 20
# turning time of shift adjustment
shift_step = 0.125
# turning time of turn
turn_step = 0.25
# time of straight run
straight_run = 0.5
# attempts to find the line if lost
find_turn_attempts = 5
# turn step to find the line if lost
find_turn_step = 0.2
# max N of iterations of the whole tracking
max_steps = 100

Code

The source code is available on Github.

Links

PiTanq site

Story of PiTanq creation

PiTanq code on Github