3D Shape generation from a short text description

Abstract

Today, the world witnesses a huge advancement in technology, specially in the domain of virtual (VR) and augmented reality (AR). A lot of the largest companies are interested in AR and VR, an imaginary world that you feel inside. That kind of project are based on visualization of 3d objects. It was always considered a difficult task for designers to build a full 3D environment. Even if it is possible to do it, it could not be achieved neither in a short time nor with less expensive software The objective of this work is to generate a 3d shape from a short text description with the help of the most interesting topic in IA neural, networks and specifically, the generative adversarial networks (GAN). We built and trained a conditional GAN (CGAN) having as input the Bert embeddings of text descriptions and as output their corresponding 3D shape embeddings. We trained an Autoencoder to learn the 3D shapes representations. To validate our model, we trained another CGAN having as output the 3D Shapes. We noticed that there isn’t a big loss in the obtained 3D forms. Keywords: Text to 3D shape, Text embeddings, Shape embeddings, CGAN, Bert, Autoencoder.