Style Share

Web-based 3D engine with neural stylization post processing.

This was created as a team project for UIUC’s CS 222. We tried to combine a mish-mash of ideas together and this was the result. I’ve always been interested in stylized rendering, so we tried to create a 3D editor to arrange a basic scene and then use a model to transfer style content on to it. I was mainly responsible for the 3D editor, which was done in React-three-fiber (I claim little experience with React and/or web-dev… this was just a bunch of hacking documentation around).

The 3D editor has some basic functionality, including loading an environment map, uploading OBJ files (with no material...), arranging objects around, and visualizing a grid. The idea was to keep it pretty simple and rely on style transfer to finish the job.

The current viewport could be saved as either the content image or the style image, although content makes much more sense. We then used an optimized TensorFlow style transfer model as a post-processing step on the content image.

Results are... mixed. The model generally seems to "wash" the entire image with a certain color that's usually quite pale.

If you don’t want to manually create a scene, you also have the option of uploading a pre-created content image. There’s also a live demo to try it out, with a gallery of previously stylized photos.

It generally seems pretty hard to reproduce the quality of the results from the TensorFlow post, but it might also be the set of content images I’ve tested with. I’d love to play around with this more in the future though :)