learnsyslab/meshfit: Place a generated mesh where the object actually is · GitHub

meshfit recovers the metric pose of a mesh in a real scene (position, per-axis scale and orientation) from RGB, object masks and a gravity-aligned point cloud.

It works with any mesh. Generated, scanned, or a CAD part; glb, obj, ply, stl, step, or anything else trimesh reads. meshfit uses only vertices, faces and, for appearance, the material. The one thing it must be told is which axis is up in the mesh's own frame, since that is what "upright" is measured against and no file format reliably records it.

Generated meshes are the motivating case, since they arrive wrong in proportion as well as pose, which is why scale is per-axis rather than a single number.

A pose-aware backend (SAM 3D, RecGen) passes its estimate as init and meshfit refines it. Without one (TRELLIS.2, TRELLIS, or a CAD part) meshfit finds the orientation by search. Same code path either way.

import meshfit

result = meshfit.fit(
    mesh,                       # trimesh, in the generator's canonical frame
    observation,                # views (RGB, mask, K, cam2world, pointmap) + object points
    canonical_up="+Y",          # glTF convention
    init=generator_pose,        # optional: SAM 3D / RecGen already had an opinion
)

result.pose.matrix()            # T_world_canonical (rigid)
result.pose.scale               # per-canonical-axis metric scale
result.confidence.ambiguous     # is this object's yaw determinable at all?

learnsyslab.github.io/meshfit: installation, user guide, examples, and API reference.

• Generator-agnostic. Nothing in the package branches on which model made the mesh. TRELLIS.2, SAM 3D and RecGen differ only in whether they pass init
• Metric. Scale and distance come from measured depth, not from appearance, which fixes only a viewing ray
• Upright when it should be. The constrained fit pins pitch and roll to zero; a free-rotation fit is adopted only when the lean is large and renders better
• Anisotropic scale. Generated meshes get proportions wrong, so scale is per-canonical-axis, solved jointly with pose
• Every stage must prove itself. A stage is kept only if it renders a better silhouette than its input
• Reports ambiguity. A symmetric object's yaw is flagged rather than committed to silently

pip install git+https://github.com/learnsyslab/meshfit.git

Not on PyPI yet, so it installs from the repository.

Developer install (pixi recommended):

git clone https://github.com/learnsyslab/meshfit.git
cd meshfit
pixi run install-dev
pixi run test

@software{meshfit2026,
  title  = {meshfit: Metric Pose Recovery for Generated Meshes},
  author = {Li, Jim Yun-Jin},
  year   = {2026},
  url    = {https://github.com/learnsyslab/meshfit},
}