override fun onCameraFrame(inputFrame: CameraBridgeViewBase.CvCameraViewFrame?): Mat {
val rgba = inputFrame!!.rgba()
val gray = Mat()
Imgproc.cvtColor(rgba, gray, Imgproc.COLOR_RGBA2GRAY)
val blurred = Mat()
Imgproc.GaussianBlur(gray, blurred, Size(5.0, 5.0), 0.0)
val edges = Mat()
Imgproc.Canny(blurred, edges, 50.0, 150.0)
// Find contours
val contours = mutableListOf<MatOfPoint>()
val hierarchy = Mat()
Imgproc.findContours(
edges,
contours,
hierarchy,
Imgproc.RETR_LIST,
Imgproc.CHAIN_APPROX_NONE
)
val minArea = 100.0
// Draw contours and put red dots on corners
for (contour in contours) {
// Draw the contour
val maxAreaContour = contours.maxByOrNull { Imgproc.contourArea(it) }
val boundingRect = Imgproc.boundingRect(maxAreaContour)
Imgproc.rectangle(
rgba,
Point(boundingRect.x.toDouble(), boundingRect.y.toDouble()),
Point(
boundingRect.x + boundingRect.width.toDouble(),
boundingRect.y + boundingRect.height.toDouble()
),
Scalar(0.0, 255.0, 0.0),
3
)
// set corners
val corners = listOf(
Point(boundingRect.x.toDouble(), boundingRect.y.toDouble()), // Top-left
Point(
boundingRect.x + boundingRect.width.toDouble(),
boundingRect.y.toDouble()
), // Top-right
Point(
boundingRect.x.toDouble(),
boundingRect.y + boundingRect.height.toDouble()
), // Bottom-left
Point(
boundingRect.x + boundingRect.width.toDouble(),
boundingRect.y + boundingRect.height.toDouble()
) // Bottom-right
)
/*for (corner in corners) {
Imgproc.circle(rgba, corner, 5, Scalar(255.0, 0.0, 0.0), -1) // Draw red dot at each corner
}*/
}
checkImgCapture(rgba)
return rgba
}
I see Canny. that’s a newbie trap. discard it in this instance and avoid it in general. there is no solution in that direction.
this looks like a picking task for a robot with suction cups.
if you want industrial performance, you’ll need to deploy ML/DL/AI. nothing less will do. those things are hard to make out with just image processing or anything that doesn’t have some amount of ML in it.
get some neural network to learn to point out the boards. best results if modeled as instance segmentation.
that task should be fairly easy to generate abundant synthetic training data for. boards are just textured boxes laying on top of each other. get someone to poke a 3D engine (unreal or unity or blender) to generate convincingly lit scenes with those things “strewn” around.
if you wanted to cheat, your first step would be to instruct workers to slap those labels in the middle of each board, and then to suck them up on their labels. those labels are probably easier to detect and locate. I see barcodes, and the form has a fairly rigid appearance.
another approach I’ve seen uses a laser scanner to obtain 3D surface information. then, planes are extracted from that point cloud (likely with traditional algorithms, not ML). you would hope that each plane is one board… but if boards butt up against each other, multiple boards might be seen as a single board. that would result in a picking center straddling two boards.
a ML/DL/AI approach should benefit from 3D information over just visual information. you should never rely on pure geometry (point clouds).
So do you mean it is not possible using OpenCV. I need to implement AI/ML tools?
it is possible, using opencv, but you need a more advanced strategy
Can you please guide.
you need about half a graduate level education on this stuff.
I hope you understand why that’s impossible for us to provide to you.
Yes, I implement the solution but now I only facing to work with dynamic height or width. Like what to do if the camera distance is different…!?
I detect this much for this specific image. Now I need to do it dynamic
Is there any way to train openCV to detect specific images. Please share some document on this.