Kaleidosmusic is a music-artistic project for creating psychedelic-looking videos, but it is structured according to the logic of mapping the sound spectrum onto the visual spectrum. The goal is not to replicate already widely used models but to give users a perceptual experience based on mutual synesthesia between music and images (and vice versa). In other words, the goal (perhaps a bit ambitious) is to make people “see” sounds and “hear” colors through a combined but still coherent perception.
What Kaleidosmusic is not
Before introducing some elements related to the functioning of the algorithm, I would like to preface what Kaleidosmusic is not:
- A system for producing video with random color variations (e.g., continuous mutations of a plasma)
- A model for inducing hypnosis or altered states of consciousness
- A model for conveying subliminal messages of any kind
What are the fundamental principles of Kaleidosmusic
Without going into overly technical details, the Kaleidosmusic algorithm is based on some basic steps implemented in Python, Scipy, and Librosa. The first consists of spectral analysis of frequencies contained in fixed-duration stereo samples. Assuming we have a piece of music M, it is discretized into a sequence A containing Nai samples:
Each sample ai consists of a tuple (ai1,ai2) containing the dates of the two stereo channels. At this point, the two channels containing a time series are resampled and overlapped (from stereo, the series becomes mono) and prepared to undergo SDFT (Short-time Discrete Fourier Transform) through a windowing process:
The fragments obtained from each window are then subjected to FFT (Fast Fourier Transform) to derive the frequency spectrum F:
At this point, the algorithm maps the sound frequency peaks and the corresponding frequencies of the visible light spectrum. The sample peaks correspond to the dominant notes and the higher-order harmonics induced by them, as shown in the figure:
Through a series of filtering, the peaks are combined and weighted according to their relative importance, and the result is transformed into a frame of the final animation (which will perfectly sync with the audio).
Attention
The videos generated through the Kaleidosmusic algorithm contain very rapid chromatic variations covering the spectrum from red to violet. Therefore, the effect produced may be dangerous for anyone suffering from epilepsy, psychotic disorders, or any other neuropsychiatric disorder that may be exacerbated through violent visual stimulation. Consequently, in all such cases, it is not recommended!
Examples of Kaleidosmusic
All the videos are usually published on my YouTube channel and in the post section. Here are a few examples.
F. Chopin – Waltz in A min. B150 Posth.
If you like my work, you can always donate to support my activity! One coffee is enough! And don’t forget to subscribe to my weekly newsletter!
