NeuralFur Teaser DynHair is a novel method for learning hair dynamics of 3D strands from data. It leverages multi-view video input to learn a dynamic hair strand deformation model based on the velocity and acceleration of the human head.

Abstract

We present DynHair, a novel method for tracking and modeling dynamic hair for human head avatars. From video input, we reconstruct a dynamic head avatar with an explicit strand-based hair representation using structured 3D Gaussian Splatting. In contrast to the face region of human head avatars, which can be modeled with 3D Gaussians that are attached or generated with respect to some expressive 3D head model, hair is particularly challenging as it exhibits dynamic motion effects. Therefore, we present a novel method that models the dynamic deformations of the hair strands using a temporal network that is conditioned on angular velocity and acceleration of the head, as well as relative gravity. Specifically, an LSTM encodes the motion history and modulates per-point strand features via FiLM conditioning which further used by MLP to produce physically plausible displacements to canonical hairstyle. We jointly optimize this motion and appearance representation of the hair, with a 3DGS-based representation of the face-region, via differentiable Gaussian splatting with photometric, geometric, and physics-based supervision. As a result of our method, we retrieve hair tracking of the training video data and an animatable head avatar with controllable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency, and generalization across subjects.

Video Presentation

Main idea

Given multi-view video input, we first initialize a canonical hybrid model for the upper body and hair. Hair strands are initialized using Im2Haircut prior model, and the upper body is represented with unstructured 3D Gaussians. We then jointly optimize hair and head deformations with two separate motion models. Hair deformation is conditioned on BFM motion history with an additional rigid transformation, while head deformation is conditioned on facial expressions and pose parameters. Finally, the hair and upper body Gaussians are composited and rendered via differentiable splatting to produce the output images.

Results: Self-Reenactment

We evaluate our method on self-reenactment, where the reconstructed avatar is driven by the same subject's captured motion. Compared to Gaussian Head Avatar, our approach produces sharper, more physically plausible hair motion rather than blurry, rigid strands.

Reference
Gaussian Head Avatar
Ours

Results: Cross-Reenactment

Our method also supports cross-reenactment, transferring the relative head and hair motion from one subject to drive the avatar of another. This demonstrates that the learned hair dynamics generalize beyond the motion seen during training of a specific subject. Since we do not track the shoulders, this can introduce minor blurring artifacts near the shoulder region — a limitation of the current approach.

New Captures

To further validate our method, we captured new scenes using 15 synchronized cameras at 72 fps, including 10 hair-motion sequences and 10 expression sequences, along with 360° turntable captures both with and without a hair net. We show qualitative results on these new captures alongside the corresponding reference motion.

Reference Motion
Results

Applications: Editing and Color Change

Because our representation is strand-based, it supports direct hairstyle editing — trimming strand length or changing hair color — while preserving the learned dynamics.

Ablations

Deformation Network

The deformation network significantly improves results and motion realism. Removing it leads to noticeably degraded hair motion.

w/o Hair Deformer
w/ Hair Deformer

Number of Views

Our approach also works in a monocular setup when head tracking is available. Here we compare results with a fixed tracker and static hairstyle while learning deformation from either four views or just a single view.

Conditioning Signal

The conditioning signal is crucial. Using absolute pose or removing acceleration reduces motion smoothness, and omitting relative gravity causes temporal instabilities.

Architecture and Loss

Finally, we ablate FiLM layers, replace the LSTM head-pose conditioning with an MLP, and analyze the elastic physics-based loss, showing that each component contributes to performance.

Acknowledgements and Disclosure

Vanessa Sklyarova is supported by the Max Planck ETH Center for Learning Systems. Justus Thies is supported by the ERC Starting Grant 101162081 ''LeMo'' and the DFG Excellence Strategy - EXC-3057. This work is supported by the ERC Consolidator Grant Gen3D (101171131). Berna Kabadayi is supported by the IMPRS-IS. Gerard Pons-Moll is a member of the ML Cluster of Excellence, EXC-2064/1 – Project No. 390727645, and is endowed by the Carl Zeiss Foundation. This work is supported by the BMBF: Tübingen AI Center, FKZ: 01IS18039A, and by the DFG – 409792180. This work is supported by the Swiss National Science Foundation Advanced Grant 216260: ''Beyond Frozen Worlds: Capturing Functional 3D Digital Twins from the Real World.''

The authors would like to thank Zhanfeng Liao for providing data from HHAvatar, Egor Zakharov for supervising Haonan during his time at ETH, and Benjamin Pellkofer for IT support.

While MJB is employed by Epic Games, this work was performed solely at, and funded solely by, the Max Planck Society.