Summary of Our Attempt to Implement the Paper (~15 Mins)


This project is motivated by HuggingFace X GoogleAI (JAX) Community Week Event 2021 and we are pleased to announce that our project received the 2nd place in the event! This report aims to explain the model that we implemented and summarise how we trained it and what we found during the process.

In this project, we attempted to implement DietNeRF, a variant of NeRF recently published in CVPR (Computer Vision and Pattern Recognition) 2021 conference. It is designed for few-shot learning of 3D scene reconstruction using 2D views. Below are some of the 3D scenes that our model managed to reconstruct (converted as an animated 2D rendered views):

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/b464d464-93bd-4367-806a-10bfea9f4da1/8_shot_mic_diet_nerf.gif

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f4a16e16-f72c-4126-b196-4a264ed65e65/8_shot_drums_diet_nerf.gif

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/69b195dd-416a-4faa-bacb-7fae736b9884/8_shot_chair_diet_nerf.gif

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/8f7ce958-a9c7-4ddb-8830-3ebd5cfed26b/8_shot_hotdog_diet_nerf.gif

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/29026e1b-7af7-4677-ba14-fb67cca5ee07/8_shot_lego_diet_nerf.gif

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/1ff02702-3b79-4feb-bf80-4ffef444231b/8_shot_ship_diet_nerf.gif

We will break down our report into the following:

Additionally, you can view our project on below: