Locally Stylized Neural Radiance Fields

July 31, 2023

In recent years, there has been increasing interest in applying stylization on 3D scenes from a reference style image, in particular onto neural radiance fields (NeRF). While performing stylization directly on NeRF guarantees appearance consistency over arbitrary novel views, it is a challenging problem to guide the transfer of patterns from the style image onto different parts of the NeRF scene. In this work, we propose a stylization framework for NeRF based on local style transfer. In particular, we use a hash-grid encoding to learn the embedding of the appearance and geometry components and show that the mapping defined by the hash table allows us to control the stylization to a certain extent. Stylization is then achieved by optimizing the appearance branch while keeping the geometry branch fixed. To support local style transfer, we propose a new loss function that utilizes a segmentation network and bipartite matching to establish region correspondences between the style image and the content images obtained from volume rendering. Our experiments show that our method yields plausible stylization results with novel view synthesis while having flexible controllability via manipulating and customizing the region correspondences.


< 1 minute

Hong-Wing Pang, Son Hua, Sai-Kit Yeung

ICCV 2023

Share Article

Related publications

ICML Top Tier
May 14, 2024

Bao Nguyen, Binh Nguyen, Hieu Nguyen, Viet Anh Nguyen

NAACL Top Tier
April 4, 2024

Thang Le, Tuan Luu

CVPR Top Tier
March 4, 2024

Thuan Hoang Nguyen, Anh Tran

ICLR – Tiny Papers Track
February 14, 2024

Thanh-Thien Le, Linh The Nguyen, Dat Quoc Nguyen