Deep Learning Framework for Cloth Animation

Deep Learning Framework for Physics-based Cloth Simulation

Physics-embedded NN structure for machine learning application in Computer Graphics cloth animation. Direct PBS features are encoded into the model, with functional extensions to integrate extra visual improvements.

Project Link on GitHub

Paper available at:

• ArXiv

Table of Contents

• Features
• Installation
• Usage
• Demo
• To-do

Demo

Blown-up airbag

This NN Framework

PBS result

Hanging cloth with wind

This NN Framework

PBS result

Fallen cloth folded on ball

This NN Framework

PBS result

Cloth Neural Network

Framework Structure

 

Features

• Physics-based cloth simulation: mass-spring system

• Comprehensive force interaction

  Internal: elastic, damping, and bending

  External: gravity, pressure, friction, and air drag

• Collision handling and boundary constraints
• Deep Learning application for specific PDE system
• CNN representation of spatial correlations
• Conditional programming with GPU-parallelized boolean tensor
• ML acceleration for real-time animation and rendering
• Integrable framework for prevailing AI techniques on folds and wrinkle enhancement

Installation

Requirements

• python ≈ 3.10

• taichi ≈ 1.4

• pytorch ≈ 2.1.1

Platforms

Cuda or CPU backends for simulation and learning; Vulkan available for rendering

Usage

PBS

Blown-up airbag

python cloth_press.py

Hanging cloth with wind

python cloth_hang.py

Fallen cloth folded on ball

python cloth_ball.py

Pre-Process

Training-set preparation

python groundTruth_press.py [DATA_SAVE_PATH]

NN

Train

python nn_cloth_train.py [training_data_set.npz]  [MODEL_SAVE_PATH]  [starting_model.pt](Optional)

Infer

python nn_cloth_infer.py [initial_state.npz]  [evaluated_model.pt]  [infered_result_name.npz]

Check

python nn_cloth_check.py [checked_data.npz]

Post-Process

Plot loss curves

python plotloss.py [loss_log]

View model parameters

python viewmodel.py [model.pt]

Rendering for NN predictions

python cloth_view.py [NN_result.npz]

Comparison between PBS and DL

python compare.py [ground_truth.npz] [NN_result.npz]

-some trained models are provided

-logs are provided to check loss track and time consumption

To do with the Framework

• Integrate with additional forces by PBS (e.g., turbulent flow).
• Add self-collision detection and response: Bounding Volume Hierarchy & vertex-triangle, edge-edge detection.
• Incorporate sub-NN to refine cloth wrinkles under low-reso mesh.