Update ROS interface etc. #49
Conversation
This makes it possible to import the source files without rospack.get_path(), and should also make it work from the catkin install space.
By using rospy.myargv(), it becomes possible to use ROS remapping arguments to remap the topic names.
This code path is never taken, and I'm not sure what will happen with an all-zero intrinsic matrix, but this commit still changes the line to what the original author intended.
This enables us to get rid of the global variables and to move the processing into the image callback in the following commits.
This gets rid of the g_draw global variable
This has the advantage that the processing is event-driven, i.e., an image is processed as soon as it is received instead of being throttled by the processing loop frequency. Additionally, it ensures that each input image is processed at most once, whereas the old code repeatedly processed the same input image until a new image arrived.
The output message should have the same frame_id and time stamp as the input message. This makes the "frame_id" parameter superfluous, so this commit removes it as well.
This makes DOPE work with any ROS camera driver, without having to specify the camera intrinsics in the config file.
Since DOPE is usually trained on synthetic NDSS data (which is always rectified), it is strongly suggested to also use a rectified input topic for inference. If this is not the case for whatever reason, this commit at least sets the distortion coefficients for solvePnP properly.
The existing camera node doesn't publish a camera_info topic, so it's not compatible with the previous commits. Also update the README to suggest existing ROS drivers as alternatives.
Very large input images eat up all the GPU memory and slow down inference. Also, DOPE seems to work best when the object size (in pixels) has appeared in the training data. For these reasons, downscaling large input images improves memory consumption, inference speed *and* recognition results.
This also has the effect of installing the new dependency vision_msgs.
This allows using any url format that resource_retriever understands, particularly `package://` URLs, so it becomes possible to load weight files from other packages than `dope`.
This is useful if the object mesh is rotated differently from the one the network was trained with.
Without copying the package to the image, the user had to run "catkin_make" once after the image was built. Also, now the proper setup.bash is sourced, so the user can directly type "rosrun dope dope" after building the Docker image.
Rephrase one sentence, fix whitespace
There was a problem hiding this comment.
We have been discussing about the fact of removing webcam.py and we are a little uncertain, because including the script like we did makes it easier for non ROS user to play with the code. Could I propose we update the webcam.py to also publish the camera intrinsic from a yaml file. What do you think?
Otherwise thank you so much for turning the DOPE repo into a proper ROS package :P
readme.md
Outdated
| @@ -94,7 +96,7 @@ This is the official DOPE ROS package for detection and 6-DoF pose estimation of | |||
|
|
|||
| * To debug in RViz, `rosrun rviz rviz`, then either | |||
| * `Add > Image` to view the raw RGB image or the image with cuboids overlaid | |||
| * `Add > Pose` to view the object coordinate frame in 3D. If you do not have a coordinate frame set up, you can run this static transformation: `rosrun tf static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world dope 10`. Make sure that in RViz's `Global Options`, the `Fixed Frame` is set to `world`. | |||
| * `Add > Pose` to view the object coordinate frame in 3D. If you do not have a coordinate frame set up, you can run this static transformation: `rosrun tf2_ros static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world <camera_frame_id>`, where `<camera_frame_id>` is the `frame_id` of your input camera messages. Make sure that in RViz's `Global Options`, the `Fixed Frame` is set to `world`. | |||
There was a problem hiding this comment.
If the user now is using a usb_cam we could simply mention to set the usb_cam as the fixed_frame.
There was a problem hiding this comment.
True. I'll update the README to mention that you can simply use <camera_frame_id> as the fixed_frame.
There was a problem hiding this comment.
For some reasons when I was testing the code this afternoon I could not put usb_cam or the dope_webcam as my fixed frame in rviz, I am not sure what is going on. After running a fixed frame transform I was able to visualize the poses. When I started rviz alone I do not see world for some reasons...
Here are the transform I had to run:
rosrun tf2_ros static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world dope_webcam
I was running the dope camera launch file.
There was a problem hiding this comment.
Did you solve this issue? I couldn't reproduce.
There was a problem hiding this comment.
I will test it out on a new install of ros and see what is going on there.
I've been pondering whether to remove If you prefer though, we could bring back the
It's really easy and not a lot of work. I just don't see how this provides any extra value over using an existing ROS node, but it's up to you to decide. Please also take the new tutorial into account, which also explains how to start the usb_cam node: e8750f5 |
This reverts commit 6c5040d.
|
Never mind, I've brought the camera node back to life and added support for camera_info, as outlined above. One thing that I like about DOPE is that it was so easy to set up and use, so we should keep it that way. :) |
readme.md
Outdated
| @@ -15,7 +15,6 @@ This is the official DOPE ROS package for detection and 6-DoF pose estimation of | |||
| We have tested on Ubuntu 16.04 with ROS Kinetic with an NVIDIA Titan X with python 2.7. The code may work on other systems. | |||
| If you do not have the full ROS install, you may need to install some packages, *e.g.*, | |||
| ``` | |||
| apt-get install ros-kinetic-tf2 | |||
| apt-get install ros-kinetic-cv-bridge | |||
There was a problem hiding this comment.
We also need to install ros-kinetic-camera-info-manager-py
There was a problem hiding this comment.
we also need to add this one: apt-get install ros-kinetic-vision-msgs
There was a problem hiding this comment.
All documented in the README now: 7f9e1fe
readme.md
Outdated
| @@ -94,7 +96,7 @@ This is the official DOPE ROS package for detection and 6-DoF pose estimation of | |||
|
|
|||
| * To debug in RViz, `rosrun rviz rviz`, then either | |||
| * `Add > Image` to view the raw RGB image or the image with cuboids overlaid | |||
| * `Add > Pose` to view the object coordinate frame in 3D. If you do not have a coordinate frame set up, you can run this static transformation: `rosrun tf static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world dope 10`. Make sure that in RViz's `Global Options`, the `Fixed Frame` is set to `world`. | |||
| * `Add > Pose` to view the object coordinate frame in 3D. If you do not have a coordinate frame set up, you can run this static transformation: `rosrun tf2_ros static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world <camera_frame_id>`, where `<camera_frame_id>` is the `frame_id` of your input camera messages. Make sure that in RViz's `Global Options`, the `Fixed Frame` is set to `world`. | |||
There was a problem hiding this comment.
For some reasons when I was testing the code this afternoon I could not put usb_cam or the dope_webcam as my fixed frame in rviz, I am not sure what is going on. After running a fixed frame transform I was able to visualize the poses. When I started rviz alone I do not see world for some reasons...
Here are the transform I had to run:
rosrun tf2_ros static_transform_publisher 0 0 0 0.7071 0 0 -0.7071 world dope_webcam
I was running the dope camera launch file.
|
Thank you for putting a new webcam node together. I quite like what you did and I agree with you about ROS not needing a new camera node. The webcam tutorial is quite extensive, thank you. Once we update the requirements in the readme and in the docker file I think we should be good to go for the merge. |
|
The requirements in the docker file are handled by the rosdep install line; they are specified in the package.xml. I will add the rosdep install line to the Readme on Monday. Thanks for testing it all! |
|
I've updated the installation instructions in the README: 7f9e1fe From my point, it's ready to merge now. |
|
Oh, I think I know what is going on with respect to the fixed frame. When there are no tf transforms, nothing appears in the drop down menu. But you can still enter 'dope_webcam' manually. This is normal. |
|
I tested on last time, thank you so much this is a huge update to DOPE. |
|
Thank you for taking the time to thoroughly test this PR and iterate on it, much appreciated! |
Update ROS interface etc.
This is a huge pull request (sorry for that!) that updates the ROS interface among other things. The git history is carefully curated, so for details check the commits contained here. I'm happy to explain in a bit more detail what I changed and why, if you want.