Line following robot with OpenCV and contour-based approach
In my previous story, I told about PiTanq — a robot-tank I built. Then a big goal is to make it autonomous. I was inspired by Udacity course for self-driving cars and the first task of this course is to recognize lanes on roads.
As my robot is pretty far from hitting a road I put a white tape on the floor.
The idea was to implement tracking a line based on lanes detection approach from the Udacity course.
A general solution for this task would look like:
- Filter an image by color
- Find edges with Canny edge detector
- Crop irrelevant parts of the image
- Detect lines with Hough transform
Filter image by color
Udacity course task requires to detect both yellow and white lines. For that purpose, they use HSV or HLS conversion. I need to detect only a white line so I decided to use only a grayscale filter.
Next step I picked a threshold to have a binary image:
Detect the line
Then, applying Canny edge detector, I got that picture (inside a region of interest):
Using Hough line detector I found some odd lines:
Making detection rules more restrictive:
Not good in terms of relevance (and reliability as well).
Spent some time with these experiments I was not able to get satisfying results. Then I decided to try another approach. Instead of lines detection, I used contours detection. Assuming, the biggest contour is the line and taking a centerline of the bounding box, I got a plausible direction.
The next problem I encountered with, was different light conditions on the line. One side of the line turned out in the shadow of a couch and it was impossible to find a grayscale threshold working along the whole line loop.
The solution was to adjust the threshold individually for each image depending on a ratio of white pixels to the whole area.
ret = None
direction = 0
for i in range(0, tconf.th_iterations):
rc, gray = cv.threshold(image, T, 255, 0)
crop = Roi.crop_roi(gray)
nwh = cv.countNonZero(crop)
perc = int(100 * nwh / Roi.get_area())
if perc > tconf.white_max:
if T > tconf.threshold_max:
if direction == -1:
ret = crop
T += 10
direction = 1
elif perc < tconf.white_min:
if T < tconf.threshold_min:
if direction == 1:
ret = crop
T -= 10
direction = -1
ret = crop
Make driving decisions
Based on computer vision technics we got a direction to move. The real decisions were made depending on the angle of this vector and its shift from the image middle point.
Determine turn actions (if required):
def check_shift_turn(angle, shift):
turn_state = 0
if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle:
turn_state = np.sign(90 - angle)
shift_state = 0
if abs(shift) > tconf.shift_max:
shift_state = np.sign(shift)
return turn_state, shift_statedef get_turn(turn_state, shift_state):
turn_dir = 0
turn_val = 0
if shift_state != 0:
turn_dir = shift_state
turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step
elif turn_state != 0:
turn_dir = turn_state
turn_val = tconf.turn_step
return turn_dir, turn_val
a, shift = get_vector()
if a is None:
# there is some code omitted related to line finding
turn_state, shift_state = check_shift_turn(a, shift)
turn_dir, turn_val = get_turn(turn_state, shift_state)
if turn_dir != 0:
There is a debug visual info:
## Picture settings# initial grayscale threshold
threshold = 120# max grayscale threshold
threshold_max = 180#min grayscale threshold
threshold_min = 40# iterations to find balanced threshold
th_iterations = 10# min % of white in roi
white_min=3# max % of white in roi
white_max=12## Driving settings# line angle to make a turn
turn_angle = 45# line shift to make an adjustment
shift_max = 20# turning time of shift adjustment
shift_step = 0.125# turning time of turn
turn_step = 0.25# time of straight run
straight_run = 0.5# attempts to find the line if lost
find_turn_attempts = 5# turn step to find the line if lost
find_turn_step = 0.2# max N of iterations of the whole tracking
max_steps = 100
The source code is available on Github.