Railway detection with noisy input

I’m working on a railway detection algorithm with the following steps :

  • Get the frame and turn it into grayscale.
  • Apply Gaussian blur.
  • Apply Sobel operator on x-axis.
  • Apply Canny edge detector.
  • Apply Hough-transform to detect the lines.

This works fairly well in some scenarios like in the example below :

But for some scenarios the output it doesn’t work properly like the one below. In this case I want to detect the middle railway .

As you can see in this case I got too much noise and hough lines doesn’t work properly.

Is there a different approach I can use to detect the lines with different types of lighting ?

if you’re asking for alternative approaches, I would suggest showing the original pictures.

Input 1
Input 2

hate to say it but the best results can be obtained with Deep Learning, which means you’ll have to go to some effort to come up with enough training data.

if you want to solve it with low level image processing, these two pictures might need different approaches.

rails can appear darker or lighter than the surrounding area. they might even appear equally light, and only color or texture can distinguish them.

for better advice you’d have to explain what the ultimate goal here is.

I need to track the rails to my path following algorithm. My goal is to follow the rails with a drone and my controller needs a point in the center of the railway in order to control the drone. Here is an example :

Case 1

In this case the red dots were calculated correctly and I can set a referencial to control my drone.

I tested the algorithm for another railway video and while the red dots were calculated correcly in some frames but for some frames I got bad results like the one below :

Case 2

These kind of inconsistency can lead my system to instability, so I’m searching for different approaches on image processing to improve my detection.