'select timeout' for opencv application while running on yocto linux environment

We are trying to port opencv application on an single board computer which runs on Yocto Linux and while doing that we have had many issues. Issues related to drivers, which are solved by changing the configuration file of the kernel and all. We were able to solve all those issues and finally, we are hitting the ‘select timeout’ error during below highlight line in the snapshot code.

#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/core.hpp>
#include <opencv2/core/core.hpp>

using namespace std;

int main(int, char**) {
  cv::VideoCapture camera(0);
   if (!camera.isOpened()) {
     return 1;
   cv::Mat frame;
   while (camera.isOpened()) {
     ***camera >> frame;***
  return 0;

This code does work on my local machine, but not on the SBC. Are there any leads for this issue to solve? I would be happy to share more details if required.

P.S. Camera is well detected and thus not throwing any error at !camera.isOpened() code. It gives the error ‘select timeout’ error at camera >> frame line.