I am trying to make a perspective rectangle so i can use perspective transform, the problem is that i have to use dynamic point specification, not some kind of manual one.
And then simply use getPerspectiveTransform and warpPerspective to a rectangular shape, but i don’t know how to calculate the perspective points so they fit all pixels.
RotatedRectangle doesn’t really do the job, because it’s still a rectangle, not “best match” polygon, i tried to make a hull and then use approxPolyDP, but the results are horrible, it basically loses details rather than reduce the hull to make a square, but still fit all pixels inside… I also tried to look it up, but the only thing i can find is either manual point specification or they do some processing but on a square shape already, like paper, which doesn’t need anything like i do.
Any ideas, how to do it? It shouldn’t be hard, but from the research i did, no luck.
that might be tricky, yes. I don’t think OpenCV has anything ready-made for your situation.
you might have to come up with an algorithm to fit a parallelogram around your contour.
I could think of something fairly ad-hoc but viable…
then either:
- come up with an algorithm that splits contours into (mostly) straight runs and curved runs (OpenCV needs that still, halcon has it already)
- see if the hull contour has four straight runs that pair up to be somewhat parallel
or:
- turn it dense and equally spaced
- try to fit a pair of lines onto the contour for various angles (hypotheses), allowing for outliers (those sides of the parallelogram that aren’t currently considered, and see how many points are inliers. find the best hypothesis lines.
for the line fitting algorithm, that could be done by projecting the points onto an axis (for a given angle) and seeing where the points form clusters.
for the rectification of scans, also document layout analysis, something like that is a common step to rotate the picture to be axis-aligned.
Are you always able to get the rotated rect to work as well as it did in the first picture in your your example? What I mean by that is that the rotated rect seems to enclose the desired area / doesn’t clip anything. It also seems that two of the edges are pretty close to coinciding with edges from the ideal quad, which might be helpful but maybe not necessary for what I’m thinking:
Can you start with an enclosing quad (rotated rectangle is fine, maybe a plain old rect is fine) and adjust the corners in a way that ends up with the quad you want? At least for your example image, the minimum enclosing quad (what I think you were going for with approxPolyDP?) would give you a pretty good transform. If so maybe something like:
- Find enclosing rect.
- Iterate Over the vertices and try to reduce the length of the edges with the constraint that the resulting quad still encloses all the pixels you care about.
- Repeat until you can’t reduce any edge lengths.
This is just off the cuff - I’m not sure if it will work at all, much less in the general case, but it might have legs?
To clarify, I’m thinking you’d start with vertex V0 and try to move it toward V3 (the last vertex in the quad) until doing so would cause the edge (V0,V1) to clip your image. Then you would try you move V0 toward V1 until edge (V3,V0) clips your image. Then you would consider V1, first moving it toward V0, then moving it toward V2, and so on.
I’m sure it’s more complicated than this and there are corner cases I’m not seeing, but maybe this would at least give you a path to try?
Good luck
OK, now that I have had a chance to look at this with paper and pencil, that approach clearly won’t work as stated for the general case (starting with any enclosing quad) , but it might work if the starting quad is constrained somehow. I hold out some hope that a minimum area rotated rect might work, or at least be a better starting point. My intuition is based on the belief that a rotated rect will at have (at least) one side parallel/coincident with one side of the desired quad. It’s just a hunch, but that’s where I’d start if I were working on this problem.
I didn’t do many tests, but the rotated rect as it is, it’s not acceptable, since the only distortion i can remove with it, it’s the rotation.
What i will do instead, is make a reduced hull, to be a 4 sided polygon, and then i will scale it until it will contain all the contours, i said it was horrible, because it was missing some parts and also i forgot that i can edit it after, so in fact it produces good enough results.
As in the picture, it’s missing some parts, so the best i can do is scale it, to match everything, the background should always be black, so it doesn’t matter, then i can use the same scale for the target transformation, and then resize it after, so the result should be like if it was matched at the beginning perfectly.
