Inspired by the concept of synesthesia, Synesketch is an attempt to create an artificial synesthete, a computer program that maps text to visuals via feelings. The project started in the pre-post-truth, pre-crisis year of 2007, when Internet was still young, with a lot of creative makers untainted by questions of digital surveillance, algorithmic commodification, and social consequences of AI. One may say it was the age of innocence for digital art. Synesketch presents an abstract portrait of that age.
Technologically speaking, Synesketch algorithms analyse the emotional content of text sentences in terms of emotional types (happiness, sadness, anger, fear, disgust, and surprise), weights (how intense the emotion is), and a valence (is it positive or negative). The recognition technique is grounded on a refined keyword spotting method which employs a set of heuristic rules, a WordNet-based word lexicon, and a lexicon of emoticons and common abbreviations.
The real-time generative animations are partially based on Jared Tarbell’s Bubble Chamber, itself inspired by physics graphics of colliding particles.
Research papers about Synesketch were published in the IEEE Transactions on Human-Machine Systems and the IEEE Transactions on Affective Computing.
Synesketch was awarded by the International Digital Media and Arts Association, Canada, and the Belgrade Chamber of Commerce, Serbia. It was reviewed by Creative Review and many other digital art and visualization publications, shown on festivals and conferences in Europe, Canada, and China, and used by designers, artists, engineers, and researchers world-wide.
Synesketch is open and free to download.
The software is free and open for further development by Java and Processing communities. It is published under GNU General Public License (version 2 or later). When referencing Synesketch in your work, please use one of the citations from the Publications sections of this page. ■
— Creative Review, vol. 28 num. 11, 2008
