sound/tracks:Real-Time Synaesthetic Sonification and Visualisation of Passing Landscapes

sound/tracks:Real-Time Synaesthetic Sonification and Visualisation of Passing Landscapes

View more presentations from Chihyun.

When travelling on a train, many people enjoy looking outof the window at the landscape passing by. We presentsound/tracks, an application that translates the perceivedmovement of the scenery and other visual impressions, suchas passing trains, into music. The continuously changingview outside the window is captured with a camera andtranslated intoMIDI events that are replayed instantaneously.This allows for a reflection of the visual impression, addinga sound dimension to the visual experience and deepeningthe state of contemplation. The application is intended tobe run on both mobile phones (with built-in camera) andon laptops (with a connected Web-cam). We propose anddiscuss different approaches to translating the video signalinto an audio stream, present different application scenarios,and introduce a method to visualise the dynamics of completetrain journeys by “re-transcribing” the captured videoframes used to generate the music.

bonus

Chemical Brothers - Star Guitar (High Quality Ver.)

Lauri Gr¨ohn. Sound of Paintings.

Wikipedia. Alexander Scriabin.