Poetry Based, 3D Printed Sculpture

2019

Andrei Budescu, Diana Drăgan-Chirilă, Oana Gui – University of Art and Design, Cluj-Napoca, Romania
Corneliu Dascălu, Miklos Uszkai, Mihai Simu, Radu Lohan – Zenitech Ltd. Cluj-Napoca, Romania

The paper discusses materialization of verses into abstract printed artwork. It is an experimental project involving artists and people from IT, where poetry is reinterpreted through the relationship between man and machine. At one end, the artist inputs one verse using his voice and collects a physical form of it at the other end. Between the input and output, the machine analyses the information and creates material evidence of the voice (a 3D printed layer). The final step is made by the artist who assembles all layers into a sculpture.

Artistic Statement:
It is the idea of a path in the opposite direction, the one in which words express images, it is about the semantic, morphologic and syntactic capacity of words to function as a shape or figure against a background.

3D Printed Sculpture – Sculptural element, one verse (left) 8x7x0,1 cm. 
Proposed final result from multiple elements assembled by the artist (right) 15x14x16 cm

Technical implementation
Emotion detection and visualization

At the technical level, the project works like a pipeline. It uses parts of an audio recording (duration imposed by limitations of technology), applies several operations on it and generates a drawing. The drawing is composed by chaining together basic symbols that correlate one-to-one to vowels and consonants or words and phonemes, and applying than various geometrical transformations based on the detected emotion of the audio clip

To detect emotions from a poem we used machine learning technology, in which a model can be created that can be trained to recognize common features in different audio files.

Research:
A verse of a poem can be classified as expressing one of the basic emotions: “disgust”, “anger”, “fear”, “sadness”, “surprise”, “happy”, “neutral”.
Convolutional neural networks are most suitable for classification problems.
Train a model which can detect emotions based on specific features extracted from an audio file.

A lot of data is needed to train a neural network.
We used free and open datasets, recorded by actors.
Each audio file, from the datasets, expresses a different emotion.

To visually represent a verse of a poem we associated different shapes to each letter, which can be influenced by
emotion.
Scalable vector graphics are easy to manipulate, animate and are scalable.
Letters of the alphabet are assigned shapes, which are different based on one of the basic emotions

An emotion is detected from an audio file (representing one verse of a poem).

The emotion is used for compiling the resulting shape.

Happiness
“Romeo, Romeo! wherefore art thou Romeo?”


3D Printing Process:

Final Result: