DownhillSolver::minimize(x) assumes max(step.cols, step.rows) >= 2

Probably you are right, but then a short notice about N >= 2 in the documentation would be very helpful.
I solved the problem by setting the values of the second dimension to 0. And it works well.