Hi!
Is there any good alternative to OpenCV’s Viz3D module for Python/C++ ? It’s broken since latest VTK update. I’ve reported it on Github but I don’t think it’s going to be fixed soon. What I look for is a simple 3D visualization library, that I could use to place cameras and update their positions in real-time.
What is broken ?
I can use viz with vtk :
General configuration for OpenCV 4.5.5-dev =====================================
Version control: 4.5.2-1068-g80c2fefc43
GUI: WIN32UI
Win32 UI: YES
OpenGL support: YES (opengl32 glu32)
VTK support: YES (ver 9.0.20210122)
I am using the latest version of both OpenCV and VTK:
General configuration for OpenCV 4.5.5 =====================================
Version control: unknown
Extra modules:
Location (extra): /build/opencv/src/opencv_contrib-4.5.5/modules
Version control (extra): unknown
Platform:
Timestamp: 2021-12-25T15:19:35Z
Host: Linux 5.15.6-arch2-1 x86_64
CMake: 3.22.1
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: Release
GUI: QT5
QT: YES (ver 5.15.2 )
QT OpenGL support: YES (Qt5::OpenGL 5.15.2)
GTK+: NO
OpenGL support: YES (/lib64/libGL.so /lib64/libGLU.so)
VTK support: YES (ver 9.1.0)
With VTK 9.1.0, viz.spinOnce method is broken, so I am unable to preview animations.
What do you mean by broken ? can you post full error message as text?
I 'm trying to update my opencv with vtk 9.1.0
There is no actual error message. The preview window pops up, but it’s impossible to rotate the scene or have an animation. I already posted an issue on github, there are steps to reproduce the bug and the error message I sometimes get.
I think that spinonce called vtkRenderWindowInteractor and in your configuration QVTKInteractor
Can you check this sample?
can you check this other sample
This works perfectly.
The viz window opens and crashes immediately. I still have the other OpenCV window open and it looks like this:
Also, this error appears in the terminal:
[ WARN:0@0.293] global /build/opencv/src/opencv-4.5.5/modules/highgui/src/window.cpp (697) createTrackbar UI/Trackbar(threshold@Image): Using 'value' pointer is unsafe and deprecated. Use NULL as value pointer. To fetch trackbar value setup callback.
But it doesn’t seem to be related to the VTK problem.
I have not any problem on windows without qt for viz example using VTK 9.1.
Did you build opencv with test? If yes you can run opencv_test_viz and post result?
Can you build opencv without qt and check viz?
No, I installed it using pacman on Arch Linux, it comes with pretty much everything but no test. I’ll try to build the viz test separately.
I am setting up a Docker container to test this. But VTK relies on Qt so wouldn’t that be a problem?
I built OpenCV 4.5.5 with VTK 9.0.3 and VTK 9.1.0 without Qt. VTK 9.0.3 works as expected, VTK 9.1.0 doesn’t, it still crashes when spinOnce is used.
Here is the Dockerfile I used, that contains all building information. I can provide the built Docker images, it takes quite a long time to build.
FROM ubuntu:21.04
WORKDIR /opt/build
ENV OPENCV_VERSION="4.5.5"
ENV VTK_VERSION="9.1.0"
RUN apt-get update && \
apt-get upgrade -y &&\
apt-get dist-upgrade -y &&\
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
wget \
xterm \
curl \
build-essential \
libglu1-mesa-dev \
freeglut3-dev\
libgtkglext1-dev \
libgl-dev \
g++ \
libgtk-3-dev \
gcc \
python3-dev \
python3-pip \
python3-numpy \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libgstreamer-plugins-base1.0-dev \
libgstreamer1.0-dev \
unzip \
cmake \
ninja-build \
mesa-common-dev\
mesa-utils \
git \
pkg-config \
ca-certificates \
vim
RUN apt-get clean -y
RUN wget -O vtk.tar.gz https://www.vtk.org/files/release/9.1/VTK-${VTK_VERSION}.tar.gz
RUN tar xvzf vtk.tar.gz
RUN cd VTK-${VTK_VERSION} && \
mkdir -p build && cd build && \
cmake -GNinja -DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_EXAMPLES=ON \
-DVTK_PYTHON_VERSION=3 \
-DVTK_WRAP_PYTHON=ON \
.. && \
ninja -j &&\
ninja install
RUN wget -O opencv.zip https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip
RUN wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip
RUN wget -O opencv_extra.zip https://github.com/opencv/opencv_extra/archive/${OPENCV_VERSION}.zip
RUN unzip opencv.zip && unzip opencv_contrib.zip && unzip opencv_extra.zip
RUN cd opencv-${OPENCV_VERSION} && mkdir -p build && cd build && \
cmake -GNinja -DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DWITH_VTK=ON \
-DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-${OPENCV_VERSION}/modules \
-DOPENCV_TEST_DATA_PATH=../../opencv_extra-${OPENCV_VERSION}/testdata \
-DWITH_QT=OFF \
#-DWITH_OPENGL=ON \
-DWITH_FFMPEG=ON \
-DOPENCV_ENABLE_NONFREE=ON \
-DWITH_TBB=ON \
-DBUILD_TESTS=ON \
-DBUILD_EXAMPLES=ON \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_opencv_python3=ON \
-DOPENCV_GENERATE_PKGCONFIG=ON \
.. && \
ninja -j && ninja install
RUN cp /usr/local/lib/python3.9/site-packages/cv2/python-3.9/*.so /usr/local/lib/python3.9/dist-packages/
RUN ldconfig
CMD ["bash"]
What’s about opencv_test_viz results?
opencv_test_viz is OK, here is the output:
CTEST_FULL_OUTPUT
OpenCV version: 4.5.5
OpenCV VCS version: unknown
Build type: RELEASE
WARNING: build value differs from runtime: Release
Compiler: /usr/bin/c++ (ver 10.3.0)
Parallel framework: pthreads (nthreads=12)
CPU features: SSE SSE2 SSE3 *SSE4.1 *SSE4.2 *FP16 *AVX *AVX2 *AVX512-SKX?
Intel(R) IPP version: ippIP AVX2 (l9) 2020.0.0 Gold (-) Oct 19 2019
Intel(R) IPP features code: 0x8000
OpenCL is disabled
TEST: Skip tests with tags: 'mem_6gb', 'verylong'
[==========] Running 22 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 22 tests from Viz
[ RUN ] Viz.tutorial3_global_view
[ OK ] Viz.tutorial3_global_view (689 ms)
[ RUN ] Viz.tutorial3_camera_view
[ OK ] Viz.tutorial3_camera_view (746 ms)
[ RUN ] Viz.show_cloud_bluberry
[ OK ] Viz.show_cloud_bluberry (722 ms)
[ RUN ] Viz.show_cloud_random_color
[ OK ] Viz.show_cloud_random_color (728 ms)
[ RUN ] Viz.show_cloud_masked
[ OK ] Viz.show_cloud_masked (720 ms)
[ RUN ] Viz.show_cloud_collection
[ OK ] Viz.show_cloud_collection (747 ms)
[ RUN ] Viz.show_painted_clouds
[ OK ] Viz.show_painted_clouds (736 ms)
[ RUN ] Viz.show_mesh
[ OK ] Viz.show_mesh (749 ms)
[ RUN ] Viz.show_mesh_random_colors
[ OK ] Viz.show_mesh_random_colors (825 ms)
[ RUN ] Viz.show_widget_merger
[ OK ] Viz.show_widget_merger (729 ms)
[ RUN ] Viz.show_textured_mesh
[ OK ] Viz.show_textured_mesh (679 ms)
[ RUN ] Viz.show_polyline
[ OK ] Viz.show_polyline (666 ms)
[ RUN ] Viz.show_sampled_normals
[ OK ] Viz.show_sampled_normals (840 ms)
[ RUN ] Viz.show_cloud_shaded_by_normals
[ OK ] Viz.show_cloud_shaded_by_normals (829 ms)
[ RUN ] Viz.show_trajectories
[ OK ] Viz.show_trajectories (1116 ms)
[ RUN ] Viz.show_trajectory_reposition
[ OK ] Viz.show_trajectory_reposition (617 ms)
[ RUN ] Viz.show_camera_positions
[ OK ] Viz.show_camera_positions (704 ms)
[ RUN ] Viz.show_overlay_image
[ OK ] Viz.show_overlay_image (1197 ms)
[ RUN ] Viz.show_image_method
[ OK ] Viz.show_image_method (2261 ms)
[ RUN ] Viz.show_image_3d
[ OK ] Viz.show_image_3d (1123 ms)
[ RUN ] Viz.show_simple_widgets
[ OK ] Viz.show_simple_widgets (731 ms)
[ RUN ] Viz.show_follower
[ OK ] Viz.show_follower (698 ms)
[----------] 22 tests from Viz (18854 ms total)
[----------] Global test environment tear-down
[==========] 22 tests from 1 test case ran. (18854 ms total)
[ PASSED ] 22 tests.
YOU HAVE 2 DISABLED TESTS
But in the source code of the tests I only see spinOnce being used outside a while loop. When using it like in the python tutorials, it crashes.
I dig in the code using debugger for windows stack trace is
|>|vtkRenderingUI-9.1d.dll!vtkWin32RenderWindowInteractor::StartEventLoop() Line 154|C++|
|---|---|---|
| |vtkRenderingCore-9.1d.dll!vtkRenderWindowInteractor::Start() Line 231|C++|
| |opencv_viz455d.dll!cv::viz::Viz3d::VizImpl::spinOnce(int time, bool force_redraw) Line 198|C++|
| |opencv_viz455d.dll!cv::viz::Viz3d::spinOnce(int time, bool force_redraw) Line 102|C++|
| |example_viz_widget_pose.exe!main() Line 95|C++|
I think(may be I’m wrong) problem is in VTK.
Can you set a breakpoint at line 230 in vtk\Rendering\Core\vtkRenderWindowInteractor.cxx and tell which function is called in your configuration?
Not sure if I did it correctly. I used gdb and set a breakpoint at function cv::viz::Viz3d::spinOnce, then run until it and set a breakpoint for vtkRenderWindowInteractor::Start and did next a couple of times:
(gdb) break cv::viz::Viz3d::spinOnce(int, bool)
Breakpoint 1 at 0x3070
(gdb) run
Starting program: /home/dshem/test_c++/build/test
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffd9aa7640 (LWP 313757)]
[New Thread 0x7fffd16a6640 (LWP 313758)]
[New Thread 0x7fffd96a6640 (LWP 313759)]
[New Thread 0x7fffd92a5640 (LWP 313760)]
[New Thread 0x7fffd8ea4640 (LWP 313761)]
[New Thread 0x7fffd8aa3640 (LWP 313762)]
[New Thread 0x7fffd86a2640 (LWP 313763)]
[New Thread 0x7fffd3fff640 (LWP 313764)]
[New Thread 0x7fffd37fd640 (LWP 313766)]
[New Thread 0x7fffd3bfe640 (LWP 313765)]
Thread 1 "test" hit Breakpoint 1, 0x00007ffff72a0920 in cv::viz::Viz3d::spinOnce(int, bool) () from /usr/lib/libopencv_viz.so.405
(gdb) break vtkRenderWindowInteractor::Start()
Breakpoint 2 at 0x7ffff0595e70
(gdb) next
Single stepping until exit from function _ZN2cv3viz5Viz3d8spinOnceEib,
which has no line number information.
[New Thread 0x7fffd31fc640 (LWP 313950)]
0x00007ffff72a9380 in cv::viz::Viz3d::VizImpl::spinOnce(int, bool) () from /usr/lib/libopencv_viz.so.405
(gdb) next
Single stepping until exit from function _ZN2cv3viz5Viz3d7VizImpl8spinOnceEib,
which has no line number information.
[New Thread 0x7fffd2ad7640 (LWP 313986)]
[New Thread 0x7fffd22d6640 (LWP 313987)]
[New Thread 0x7fffd12a5640 (LWP 313988)]
[New Thread 0x7fffd0aa4640 (LWP 313989)]
[New Thread 0x7fffc2333640 (LWP 313990)]
[New Thread 0x7fffc1b32640 (LWP 313991)]
[New Thread 0x7fffc1331640 (LWP 313992)]
[New Thread 0x7fffc0b30640 (LWP 313993)]
[New Thread 0x7fffb57d1640 (LWP 313994)]
[New Thread 0x7fffb4fd0640 (LWP 313995)]
[New Thread 0x7fff8ffff640 (LWP 313996)]
Thread 1 "test" hit Breakpoint 2, 0x00007ffff0595e70 in vtkRenderWindowInteractor::Start() () from /usr/lib/libvtkRenderingCore.so.1
(gdb) next
Single stepping until exit from function _ZN25vtkRenderWindowInteractor5StartEv,
which has no line number information.
0x00007fffea4a3ff0 in vtkXRenderWindowInteractor::StartEventLoop() () from /usr/lib/libvtkRenderingUI.so.1
(gdb) next
Single stepping until exit from function _ZN26vtkXRenderWindowInteractor14StartEventLoopEv,
which has no line number information.
[Thread 0x7fff8ffff640 (LWP 313996) exited]
[New Thread 0x7fff8ffff640 (LWP 314053)]
I’m not very good with debugging tools, sorry if this isn’t what you were asking for.
May be you can try to copy this file in your vtk repo and rebuild vtk If problem disappear you can post an issue on vtk
Tried it, didn’t work.