Regarding specifications of shape and orientation. They should be the shape of the arm at an instance of time (drawn as a line (or even a cylinder) in a 3d plan , and its geographical location in a 3d plan.
We actually do. But we must abstract physical equipment right now, focusing in building an AI model using machine learning. That’s all. and it’s already enough! We have 3 cameras registering arm movements. I was intended to use OpenCV, but I don’t know it’s feasible. I believe OpenCV can be applied to only part of the project. I wish my assumptions are wrong.
idk, you can only build a model for a specific architecture, this:
would require measuring angles, while the ‘tentacle’ in the video will have totally different requirements
also, real world robot arms will have a way to read angles from stepmotor positions, etc (imo, using cameras for this is a bad idea)
however, the good news is:
once, you can formularize the forward kinematic
(it’s just 2 coupled affine transformations for the 2dof robot arm above !),
you can use that to generate synthetic data for machine learning,
e.g for the system above, generate 2 random angles, calculate the resulting position, then train the ann to predict the angles for a desired position
the bad news is:
you cannot use opencv’s ANN_MLP network,
(it can’t handle regression tasks)
however again, you could train e.g. a pytorch network,
(like the toy example here: )
and use that later with opencv’s dnn for prediction !
I’ve asked for hardware specs to robotic’s team.
I agree that cameras are not the best option. There’ll be a few blind spots for calculation where camera is unable to frame it. Specially for example, when the “tentacle” (i.e. soft arm manipulator) overlaps itself.
i. What if we use 3 cameras filming simultaneously?
ii. What about image reconstruction?
Speaking about processing power and real time feedback requirements, Would it be a very bad idea to implement i and ii?
Regarding AI model, and stepmotor data, I’d mentioned the output before
Right now, I’m focused in the formula for forward knematic.
Let me know once you allow my access to the notebook.
Unfortunately, we can’t use OpenCV for ANN_MLP. That’s the link where I grab the ideas for Regression tasks.
Let me know once you allow my access to the notebook. I’m goin g to review openCV’s dnn for prediction.