Can you give me a little more info about how can I use the App_obj2sdf executable I am also facing similar issues while rendering

HI @danielpmorton did you get any solution for this did the use of obj2sdf work for you I am also trying to simulate a terrain like structure
which has little complex materials, so wondering did you figure this out?

Yeah, I figured this out. I wrote a bunch of info in a docs page for a still-unpublished repository. I'll paste what I wrote below, since it's not public yet. I made a few small changes to make it a bit more general, but there might still be some artifacts of my specific workflow.

Loading complex meshes into Pybullet

Loading complex meshes with lots of textures into pybullet is tricky. Ideally, we'd be able to just load a URDF and call it a day, but that just doesn't seem to be possible if you want all of the textures to appear. So, we have to resort to loading multiple OBJs.

If you load an OBJ file through createVisualShape(), this will look for an MTL file in the same directory and apply it to the OBJ. However, if the OBJ has multiple bodies each with different textures, pybullet will mess this up and apply only one of the textures to every body in the OBJ. To fix this problem, we have to separate the complex mesh into multiple OBJs, with each OBJ corresponding to a single texture. We can use Erwin's obj2sdf tool for this, even if we're not using an SDF.

For the collision side of things, if we just import a mesh into pybullet, the collision body will not be the mesh itself, but rather the convex hull of the entire body (which is not useful if you want to go inside the mesh). Running VHACD is the solution here - this will give us a decomposition of the mesh into multiple convex hulls and allow for it to be hollow.

Once we have the OBJs for each texture, and the VHACD OBJ for collision info, we can start loading these into pybullet. We now have some number n different obj files which represent the visual components of the mesh, but we only have 1 VHACD file for the mesh. So, we'll load the first visual component paired with the VHACD collision information, and for the remainder of the bodies, we'll load them strictly as visual elements by setting the collision ID in the multibody as -1. Note: it does not appear to be possible to create an "invisible" collision body, because if you set the visual ID in the multibody as -1, it will still visualize the collision mesh but in a bunch of random colors.

A full workflow of the steps required to go from meshes to a correct pybullet environment can be found below:

0. Folder structure

This is what the directory will look like at the end of the process:

meshes
└── obj
    ├── your_part_1
    │   ├── your_part_1.mtl # From Blender OBJ export
    │   ├── your_part_1.obj # From Blender OBJ export
    │   ├── decomp.mtl # From vhacd: Unused
    │   ├── decomp.obj # From vhacd: Collision body
    │   ├── decomp.stl # From vhacd: Unused
    │   ├── newsdf.sdf # From obj2sdf: Unused
    │   └── part*.obj # From obj2sdf: One obj per texture
    ├── your_part_2
    ├── your_part_3
    ├── ...
    └── textures
        └── *.png

1. Fix problematic texture images

Certain texture PNG images are unable to be loaded into pybullet. After some debugging, this appears to be an issue with the PNG filetype - some were exported as PNG8 whereas others are PNG16/24. Pybullet seems to need PNGs to be 8-bit, so these need to be converted.

To do so, import each problematic PNG into Gimp and then export/overwrite the image to the same location, with the pixelformat dropdown set to 8bpc RGBA

2. OBJ2SDF

(Ensure that the C++ version of Bullet is locally built first)

• For each part OBJ in its respective folder (i.e. obj/your_part_x/your_part_x.obj), run obj2sdf as follows:
• For example:

  cd obj/your_part_x/
  ./BULLET_PARENT_DIR/bullet3/build_cmake/Extras/obj2sdf/App_obj2sdf --fileName="cupola.obj"
  

This will add newsdf.sdf and multiple part*.obj files, one per texture, into the obj/your_part_x folder.

3. VHACD

• For each part OBJ in its respective folder (i.e. obj/your_part_x/your_part_x.obj), run VHACD as follows:
• For example:

  cd obj/your_part_x/
  ./VHACD_PARENT_DIR/v-hacd-4.1.0/app/build/TestVHACD cupola.obj
  

This will add decomp.obj, decomp.mtl, and decomp.stl to the obj/part_x folder.

4. Final adjustments

Refine VHACD

• The VHACD output will most likely not be perfect. There seems to be no exact combination of parameters that leads to an exact decomposition of the part (see the vhacd readme for more info on these). In particular, VHACD tends to poorly represent the thin walls and circular regions. So, to fix this, open the decomp.obj VHACD result for each part in Blender and update the file manually (as in, replace any "bad" convex hulls with simple objects in Blender, like rectangular prisms).

Update the paths

• The paths at the top of the OBJ/MTL files from this output (part0, part1, ...) will be absolute paths, but pybullet works best with these as relative paths. These will need to be manually modified with a quick directory search/replace. For MTL files especially, ensure that these are pointing to files inside the obj/textures directory

5. Importing into Pybullet

For each part,

• For the first OBJ file, load it as a visual shape. Pybullet will see the associated MTL file in the same directory and apply the texture. For the collision body, load the full VHACD result for the entire part.
• For each other OBJ file, load the visual shape as before, but set the collision body to -1, so that it doesn't load any collision info

I did this because I had an OBJ with multiple bodies and textures. You can open the OBJ in a text editor and see if this is the case for you, and if so, then yes these steps should help. But, if all you have is a single body and a single texture, then I don't know what the issue is.

I'm trying to install the obj2sdf executable. Can you tell me how to do that?

you don't install it you run the build_cmake_pybullet_double.sh and then it'll be in the build extrass