A curated list of resources on deformable object simulation and manipulation (cloth, rope, soft bodies) for embodied intelligence — simulation environments, perception and control, policy learning, sim-to-real transfer, benchmarks, asset generation, and surveys.
Project website | English | 中文
- Simulation Environments & Platforms
- Deformable Object Manipulation Tasks
- Deformable Object Manipulation Methods
- Sim-to-Real Transfer
- Benchmarks & Evaluation
- 3D Asset Generation for Simulation
- Foundational Simulation Techniques
- Surveys & Reading Guides
- Legend
- RFUniverse: A Multiphysics Simulation Platform for Embodied AI - Multiphysics environment for household robot learning, including soft-body interactions, fluids, airflow, and heat transfer. 📄 🔧
- SAPIEN: A SimulAted Part-based Interactive ENvironment - Interactive simulation environment based on PhysX, excelling at articulated and deformable object simulation. Widely used in Embodied AI research. 📄 🔧 ⭐
- Genesis World - Multi-physics simulation platform covering rigid, deformable, and fluid bodies. 29k+ GitHub stars. 🔧 ⭐
- SAPIEN ManiSkill - Unified framework for manipulation skill learning built on SAPIEN, with rich deformable object tasks. 🔧 📊 ⭐
- Isaac Sim / Isaac Lab - NVIDIA's robotics simulation platform with GPU-accelerated deformable body support for large-scale Embodied AI training. 🔧 ⭐
- MuJoCo - Advanced physics engine with deformable body simulation support (since v3.x), widely adopted in robotics research. 🔧 ⭐
- Taichi - High-performance parallel computing language, popular for differentiable soft body simulation in Embodied AI research. 🔧
- Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware - Scales robot training data via a Real→Render→Real pipeline without dynamics simulation or physical robot hardware. 📄 ⭐
- DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy - Bimanual dexterous garment simulation with structural-correspondence demonstration generation and a hierarchical manipulation policy. 📄 📊
- DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics - Learns dexterous soft-object skills from human demonstrations and refines planned actions with differentiable physics. 📄
- GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks - Learns garment dynamics in simulation with graph networks and uses a residual model for real-world bimanual garment hanging. 📄
- GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation - Unified simulation and benchmark for garment manipulation tasks — folding, hanging, dressing. 📄 📊 ⭐
- ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environment - Large-scale 3D garment dataset with rich annotations for manipulation research. 📄 📊
- RAPID: Rapid Adaptation of Particle Dynamics for Generalized Deformable Object Mobile Manipulation - Rapid adaptation of particle dynamics for generalized deformable object mobile manipulation. 📄 ⭐
- Benchmarking the Sim-to-Real Gap in Cloth Manipulation - Systematic evaluation of the sim-to-real gap in cloth manipulation across different simulators. 📄 📊
- SoftMimicGen: A Data Generation System for Scalable Robot Learning in Deformable Object Manipulation - Extends MimicGen to deformable object manipulation, enabling scalable robot demonstration data generation. 📄 ⭐
- DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning - Dynamics-based topology augmentation for deformable object manipulation policy learning, built on PhysTwin and Real2Render2Real. 📄
- Cloth Region Segmentation for Robust Grasp Selection - Perception and grasping: segments cloth edges and corners from depth images and selects grasp poses, with real-robot grasp evaluation. 📄
- Center Direction Network for Grasping Point Localization on Cloths - Perception and grasping: CeDiRNet-3DoF localizes cloth grasp points and provides the ViCoS Towel Dataset; evaluates perception rather than a complete manipulation pipeline. 📄 📊
- Visuotactile Affordances for Cloth Manipulation with Local Control - Visuotactile control: grasps cloth edges using visual and tactile affordances, then slides along an edge to a corner with tactile feedback. 📄
- UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence - General garment manipulation: learns category-level dense correspondence to guide unfolding, folding, and hanging with one or few demonstrations. 📄
- CLASP: General-Purpose Clothes Manipulation with Semantic Keypoints - General garment manipulation: connects VLM task plans to keypoint-conditioned skills for folding, flattening, hanging, and placing, with dual-arm real-robot evaluation. 📄
- Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulation - Tidying and folding: combines dynamic flings and pick-and-place actions to canonicalize garment configurations before real-world ironing and folding. 📄
- FoldNet: Learning Generalizable Closed-Loop Policy for Garment Folding via Keypoint-Driven Asset and Demonstration Synthesis - Tidying and folding: synthesizes keypoint-annotated garment assets and demonstrations and uses KG-DAgger recovery data for closed-loop folding policies evaluated in simulation and reality. 📄
- GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulation - Cluttered garments: learns point-level affordances and adapts entangled piles to support garment retrieval, with simulation and real-world evaluation. 📄
- GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoning - Cluttered garments: combines vision-language reasoning, segmentation, affordances, and dual-arm coordination for language-guided garment retrieval. 📄
- DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignment - Policy learning: ranks diffusion-generated action candidates using a human-preference reward model for long-horizon real-world deformable manipulation. 📄
- DeMaVLA: A Vision-Language-Action Foundation Model for Generalizable Deformable Manipulation - Policy learning: combines real-world demonstration pretraining, flow-matching actions, and human-in-the-loop corrective data for garment folding across categories. 📄
- Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks - Multiple object types: uses image-goal-conditioned Transporter Networks for multi-step cable, fabric, and bag manipulation, evaluated in simulation and physical experiments. 📄
- DextAIRity: Deformable Manipulation Can be a Breeze - Airflow-assisted manipulation: combines grasping and closed-loop blowing actions for cloth unfolding and bag opening on a real three-arm system. 📄
- SimWeaver: Zero-Shot RGB Sim-to-Real for Deformable Manipulation - Combines measurement-backed cloth simulation, topology-aware trajectory synthesis, and photometric augmentation for RGB policy transfer. 📄
- SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds - Real-to-sim-to-real data generation with scene digitization, calibrated deformable dynamics, and filtered trajectory synthesis. 📄
- Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning - Uses stress-penalized rewards and a rigid-to-deformable curriculum to transfer gentle manipulation policies to fragile real objects. 📄
- Learning to Manipulate Deformable Objects in the Real World via Sim2Real - End-to-end sim2real pipeline for deformable object manipulation with domain randomization. 📄
- DaXBench: Benchmarking Deformable Object Manipulation with Differentiable Physics - JAX-based differentiable benchmark covering cloth, rope, and fluids for planning, imitation learning, and reinforcement learning. 📄 📊
- Real Garment Benchmark (RGBench): A Comprehensive Benchmark for Robotic Garment Manipulation featuring a High-Fidelity Scalable Simulator - Combines garment meshes, a cloth simulator, and real-motion measurements to evaluate physical simulation fidelity. 📄 📊
- MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects - Mobile manipulation task suite for deformable objects, with reinforcement and imitation learning baselines and real-robot transfer experiments. 📄 📊
- SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version) - Early-version visuo-tactile benchmark using FEM deformation states to distinguish task success from physically safe manipulation. 📄 📊
- Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions - Reconstructs soft-body digital twins from videos and evaluates robot policies using physics simulation and Gaussian splatting rendering. 📄
- PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Robotics - Real-world multimodal deformation data with meshes, RGB-D observations, and interaction forces for learning and validating deformable models. 📄 📊
- SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation - Benchmark suite for deformable object manipulation (cloth, rope, fluid) with standardized RL environments. 📄 📊 ⭐
- EMPM: Embodied MPM for Modeling and Simulation of Deformable Objects - Reconstructs deformable objects from RGB-D videos and identifies material parameters with differentiable MPM and online sensory feedback. 📄
- NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos - Reconstructs deformable digital twins using piecewise spring topology and neural spring fields for spatially varying physical properties. 📄
- Image2Garment: Simulation-ready Garment Generation from a Single Image - Predicts garment geometry and fabric physics from a single image using material attributes and measured material-to-physics mappings. 📄
- PhysX-3D: Physical-Grounded 3D Asset Generation - End-to-end paradigm for physical-grounded 3D asset generation with PhysXNet dataset. 📄 ⭐
- PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image - Generate simulation-ready 3D assets with physical properties from a single image. 📄
- PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects - First to include deformable objects in PhysX generation scope, unifying rigid/deformable/articulated. 📄 ⭐
- DiffGI: Differentiable Geometry Images - 3D garment generation via differentiable geometry images (no integrated physics yet). 📄
- PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos - Physics-informed reconstruction and simulation of deformable objects from video; a precursor to Real2Render2Real and DeformGen. 📄 ⭐
- Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video - Reconstructs simulation-ready, photorealistic standalone garment assets from multi-view video. 📄
- DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact - Differentiable cloth simulation with dry frictional contact for parameter identification, assisted dressing, and control. 📄 🔧
- AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation - Material-conditioned graph dynamics with online physical-property estimation for adapting manipulation to unfamiliar deformable materials. 📄
- Learning Mesh-Based Simulation with Graph Networks - MeshGraphNets learns mesh-based dynamics, including cloth, with adaptive discretization and mesh-space and world-space interactions. 📄
- XRTailor (OpenXRLab) - GPU-accelerated cloth simulation engine for large-scale data generation. 🔧
- Position Based Dynamics (PBD) - Classic introductory resource for Position Based Dynamics simulation. 🔧
- ThinShellLab: Thin-Shell Object Manipulations With Differentiable Physics Simulations - Fully differentiable simulation platform for thin-shell materials (paper, cloth) with varying bending stiffness. 📄 🔧
- Second-Order FEM for Deformable Surfaces - High-order finite element method for improving accuracy in cloth / thin-shell simulation. 📄
- A Survey on Robotic Manipulation of Deformable Objects: Recent Advances, Open Challenges and New Frontiers - Survey: reviews deformable-object perception, modeling, and manipulation, emphasizing data-driven approaches and open research challenges. 📄
- Unfolding the Literature: A Review of Robotic Cloth Manipulation - Survey: reviews how textile variation is addressed in modeling, perception, benchmarking, and manipulation, and identifies generalization challenges. 📄
| Tag | Meaning |
|---|---|
| 📄 | Paper |
| 🔧 | Tool / Framework / Engine |
| 📊 | Benchmark / Dataset |
| ⭐ | Recommended / Important |
Contributions welcome! Please read the contribution guidelines first.
Quick way to add a paper:
- Edit
papers.yml— add your entry under the appropriate section - Run
python generate.py - Submit a Pull Request