Complete a partial mesh and make it watetight

Question

I am capturing point clouds from a RealSense camera and converting them into meshes using the Trimesh library. The issue is that I only get a non-watertight mesh from this. How do I "finish" the mesh and make it watertight?

I tried

trimesh.repair.broken_faces(mesh, color=[255, 0, 0, 255]) 

but that didn't seem to fully help. I tried creating a convex hull:

and tried to perform a union between the two, but that died with:

ipdb> mesh.union(ch,engine='scad')                                                                                             
*** subprocess.CalledProcessError: Command '['/usr/bin/openscad', '/tmp/tmpqrdvbdd2', '-o', '/tmp/tmpvdzjmkgm.off']' returned non-zero exit status 1
ipdb> mesh.union(ch,engine='blender')                                                                                          
*** subprocess.CalledProcessError: Command '['/usr/bin/blender', '--background', '--python', '/tmp/tmp9_5phhhj']' returned non-zero exit status 127

and I would also lose the RGB information from the source mesh. How do I complete the mesh using a convex hull, yet retain all the known RGB values?

Edit: I moved the needle a little more. I looked at the face normals of the convex hull, and extracted all those that were pointed to the side and down. I created a new mesh with the old mesh vertices and the faces defined by the old faces + the new ones from the convex hull.

This gets me closer to my goal, but now I have a bunch of holes that neither trimesh.repair.fill_holes nor meshlab's filter is giving me good results.

Answer 1

I'd check into the pyvista example page, they've got a nice pyvista.wrap function to get you from trimesh to pyvista, then you can use pv_obj.points to set vertices and pv_obj.faces.reshape(-1, 4)[:, 1:] to get back into trimesh. Some operations will take you from PolyData into an UnstructedGrid class, but so long as they're triangles, those lines should work. Trimesh is very fast, and he's a rockstar at answering github issues, but the documentation is much better in pyvista. I have to jump between the two constantly for my project.

Here's the relevant example I'm thinking of: https://docs.pyvista.org/examples/01-filter/resample.html

Your hole filling can likely be handled by their side repo at https://github.com/pyvista/pymeshfix before pulling back into trimesh.

trimesh also lists Open3D as an optional dependency, which has much better point cloud reconstruction algorithms, but I haven't used it enough to push you towards resources for it.