Finding the required color interval for segmentation

I am trying to get the plant as it is by segmenting everything else in the region, but I am unable to pinpoint the boundaries of the colors forming the plant:

It’s a straightforward task that I am using the following code to achieve:

import cv2
import numpy as np

# specify the path to the video
vidcap = cv2.VideoCapture('/home/bla/Desktop/3d_reconst/data/2018/02/22/1/MVI_0590.MOV')

# read the first frame
success,image = vidcap.read()

# so that we know the number of frames
count = 0

# do it for the entire stream, as long as the video plays, success flag is for this purpose
while success:

  # switch to HSV color space
  hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
  
  #specify the lower and upper boundaries for the color segmentation
  low_brown = np.array([10, 100, 20])
  high_brown = np.array([20, 255, 200])

  # take the foreground with the specified boundaries
  mask = cv2.inRange(hsv, low_brown, high_brown)
  
  # apply it on the original frame and get its original form
  res = cv2.bitwise_and(image,image, mask= mask)

  # save frame as JPEG file  
  cv2.imwrite("frame%d.jpg" % count, res)     
  success,image = vidcap.read()
  print('Frame #:', count)
  count += 1

What I fail to find though is the lower and upper boundaries for the plant (assumingly it is brownish tones) I am trying to segment. Is there a convenient way to do that? At the moment the intervals are not doing it right, and I get an output where the plant is not segmented at all.

I am also not sure if I should use blurring before I go for segmentation, my input frames are coming from a .MOV file (uses MPEG4 compression) so there is already blur involved in my images.

Any recommendation is highly appreciated.