Post 34- The Mathematics of Electronic Music

This concise, beginner-friendly essay by Peter Elsea was useful in helping me understand more of the technical aspects of music analysis. The below illustration included in it really was worth a thousand words in explaining sine waves and Fourier transforms, not the most artistically drawn piece of work but exceptional use of metaphor to elucidate an idea that seems esoteric when described only in words.

In addition, this page on the Adobe Soundbooth help section also explains well, some fundamentals of digital audio.

Post 33- Music Sampler

In undertaking this project, I initially had no idea how I was going to procure the data from music to create the symbols so I met with my friend and aspiring programmer and artist Louis Eguchi seeking advice on how to go about this. Before long, him and his brother Nat (also a programmer and artist, with whom Louis often collaborates) had started writing me a custom application called Music Sampler, which takes measurements of sound levels from .mp3 files at specified intervals and writes them to a spreadsheet. 

This neat app allowed me to be flexible in my experimentation with representing this data, as opposed to an automated generative system. 

View Nat and Louis's blog full of awesome projects (the music drawing app might steal a few minutes off you), random scribbles and good music here and follow them on twitter.

Post 32- Peter Saville

Though I hadn't consciously referenced the work of Peter Saville in creating the visual system, somebody observed that his influence is evident within it and retrospectively I can also see this. This is not surprising however as Saville's fame as a designer is founded on his imagery for various music artists' album covers; imagery which has shaped visual language about music.

My system of symbols is most apparently linked to the key to his coded alphabet adorning the sleeve of New Order's Blue Monday. Though this link is not profound (it is a key to a code, rather than the code in itself and describes the alphabet rather than music), you can draw some parallels between the circular composition, importance of colour and relation to language.

Saville's coded alphabet key and alphabet code implemented on the Blue Monday sleeve.

Post 31- More Final Developments

The rule for the final set of symbols was developed with thought into what the equivalent visual characteristics of the sonic properties I am using are. Instead of trying to exhaustively try to describe the sound with the visual (this can never really be achieved by anything other than the sound itself) the system is intended to be an indicator of its fundamental attributes. The properties I used for my data are: most frequent pitch, most frequent key, mean tempo, dynamic Db information (values taken from the waveform that visualises the song's electronic signal, these are taken at specified time increments and are useful, like the waveform at indicating patterns and rhythms) and song length.

My research into these aspects led me to link the following sonic and visual properties: 

Key (most frequent) = Hue. Hue can be equated to Key as both are known to strongly influence the mood of a piece. I defined 14 separate hue classes for each Key class (C,D,E,F,G,A,B) and their sharps and flats. The frequency of each key corresponds to the frequencies on the spectrum of visible light.

Pitch (most frequent) = Warm or cool shade added to the hue. The higher the frequency of the sound, the higher percentage of blue added to the initial hue. The lower the frequency, the more orange (mix of magenta-yellow) added to the initial hue. This corresponds to the the spectrum of visible light and acts in a similar way to sound as, just as different pitches exist within different key classes, different ranges of hue can contain an array of variations. 

Tempo (Mean BPM) = This is reflected in tint and influences the colour variable for Db values. Tempo generally determines whether a song sounds vivacious, natural, mellow or languid whilst the tint value of a hue can make it seem vibrant, normal, washed out or insipid. A high tempo will result in an untinted hue whilst a slower one will see a greater amount of white added.

Decibel Information = The sound's waveform, measured in decibels against time reveals the sound levels and is a very good indicator of patterns and structure. This is interpreted by my visual system in an opacity mask which takes a dark hue (determined precisely by the tempo; higher tempo songs will generate a more harmonious hue that will help to accentuate the general hue of the symbol, increasing its vibrancy while a slower song generates a more complementary hue that will make the overall colour more cloudy) and makes it opaque or transparent to a certain extent, depending on the Db value measured at that point. Higher values will be have a higher transparency, allowing the brighter hue underneath to come through whilst lower values will be more opaque. This will create a darkened fade out effect as the sound levels drop at the end of a track which will help to create an intuitive marker of the end (and beginning) of the song.    

Length = A patterned edge to the circle shape shows clearly the number of measurements taken during the song, the longer the song, the finer the pattern and more colour detail within the shape. I am considering whether to vary the general size of symbols based on song length in the final piece, I'm not sure how important it is as a factor.

Post 30- Final Developments

Having rejected the blobby, idiosyncratic shapes generated by previous rules, I sought to strip my generative symbols down to something that would allow the user to gauge each sound's proximity to the next. The most immediate way of doing this, used universally to differentiate visual information in contexts such as tube maps, encyclopaedias, filing systems and electronics, is colour coding. Rather than the shape creating the point of difference, I would use the more immediately comparable medium of colour. 

In each symbol, there are a number of segments which correspond to intervals in the song where measurements of sound values are taken, this value determines the colour for that segment. The top image is one I experimented with, whereby changes in tone and bpm over time dictated the change in hue, however I felt that this was too similar to a pie-chart as the hue changes were the most dominant aspect. Giving the symbol a single hue, and communicating the differences in sound values through variations within that hue creates a much more readable visual and, I think, a more pleasing aesthetic.

Post 29- Music Info Resources

As I mentioned in a previous post, I've never played an instrument or studied music so it was necessary to research some of the key terms used to analyse it so that I could understand them enough to consider how to translate them into visual language, below are some of the resources I used to do this:

Explanation of sharp and flat keys

Info on tempo and tempo classes

Clarification of pitch, key and accidentals

Further explanation of key, chords and harmony

Post 28- Robert Corish

An interesting installation by Robert Corish harnessing the physical vibrations of sound to create a projection of fluctuating patterns of coloured light. His site is full of other fascinating generative projects that bridge the gap between art and science.

Post 27- Key and Colour

Simple graphic showing the relationship between colour and key:

Post 26- Key Charter

Charting is a web app that uses data from the Echo Nest (a music analysis company which creates databases to power music applications) to create a simple a infographic that demonstrates the key most used by an artist. This is quite closely related to what I want my system to produce, in that it is very easily comparable. However I do want to represent a little more detail (I will include data from more aspects of the music) and think that the visuals should be more variable and contain some cues to suggest what information is evident in them.

Post 25- Tune Glue

Though it is driven by the algorithms that connect and recommend 'similar' artists that I have previously slated a bit on this blog as a way of finding new music, TuneGlue is well realised and entertaining as a way of presenting proximity between artists.  

Post 24- Musicovery

Although it's pretty impossible to quantify which music suits a given mood or emotion (because what someone will want to listen to whilst in a certain state of mind is entirely subjective), Musicovery is a site with a nice interface that allows you to listen to a host of radio stations matched to 4 distinct moods: Positive, Dark, Energetic and Calm (and the shades of grey in-between).

I think the idea is executed well and you can discern a quite general change in mood of the music when moving the cursor over the grid but, like anything that tries to categorise the mood of music, it is by no means perfect. A case in point is when i scrolled over the bit between 'positive' and 'energetic' and it played Jimi Hendrix's song Manic Depression.

The playful feel of it and ability to seamlessly flit through 117 tracks makes it possible to discover new music but mostly I just end up entertaining myself with it by mashing together death metal and jazz.

This interface is just a single feature of the site however and it does contain a Pandora style 'similar artist radio' but as mentioned before on this blog I don't think these algorithmic recommendation systems are brilliant tools for finding new music.

 

Post 23- Candle In The Dark- Song Length

Found this interesting study into song length that took in data from 70,000 songs and found a strong tendency for the length of songs to be very close to 4 mins in length. Read more at the blog A Candle in the Dark.

Post 22- Jason Bailey - Drawing from Music

I found Jason Bailey's blog when searching for projects that span both music and art and was intrigued by this project whereby he has broken down the digital music file of jazz artist John Coltrane's giant steps into ASCII characters and inputted to a drawing program that creates a series of points on an X/Y axis that are joined up by lines. I've contacted Jason as I'd love to know more about this process and am awaiting reply. 

Post 21- Edward Tufte

Though I am not creating 'infographics' in my system for visualising music, (read the post on Isotype charts) I followed up a reference in Erik Spiekermann's book review (mentioned in the Isotype post) to Edward Tufte.

Tufte is an influential thinker in the field of information design who advocates a selective approach to data-based design with the elimination of any elements that do not contribute to the audience's understanding of the implications of that data (this is summed up by his term, 'chartjunk').

One idea that he is an exponent of, summarised on his Wikipedia page, struck me as being salient in general and particularly relevant to this project:

Tufte also encourages the use of data-rich illustrations with all the available data presented. When examined closely, every data point has value; when seen overall, trends and patterns can be observed. Tufte suggests these macro/micro readings be presented in the space of an eyespan, in the high resolution format of the printed page, and at the unhurried pace of the viewer's leisure

These sound like a good set of guidelines for visualising music, as I think to be able to compare the sounds visually with one another, my symbols will need to be adept at showing overall trends and patterns.

Below is an example of what Tufte brands 'the best statistical graphic ever drawn', a diagram of Napoleon's Russian campaign and subsequent retreat that gives the viewer a laconic insight into the fortunes of the campaign with the width of the line representing the dwindling number of men in his army.

Post 20- Transforming data into human stories

Erik Spiekermann's review of the book The Transformer: The Principles of Making Isotype Charts in Eye magazine makes some pretty pertinent points about the process of encoding data in graphics. The subject of the book he reviews is Otto Neurath's work in developing a style of data presentation which maintained the connection between the actual data being shown and the visual. The principles of Neurath's Isotype charts advocate the tailoring of the visual to the data it is representing to create a piece which can be understood as intuitively as possible. 

Neurath developed the notion of ‘the transformer’, as Robin Kinross writes in his preface, ‘to describe the process of analysing, selecting, ordering, and then making visual some information, data, ideas, implications’.

A crucial difference between my visual system and infographics is that the symbols are not intended to be transcriptions of the absolute music data which you could read a key to understand, rather they are a visual language for comparison and observation to be understood in relation only to one another. 


The arbitrary nature of languages means that the symbols created by my system could take nearly any form, as long as they make differences apparent. 


However, to give the user of the system some clues as to what is being signified in specific differences I will firstly draw on existing visual language and, more importantly, give them some reference points for translation in the form of well-known songs visualised through the system in the contexts in which my symbols will appear.

Post 19- Andy Gilmore Music Art

Andy Gilmore's collection of geometry and music inspired graphics beautifully demonstrate the tacit relationship between the visual arts and music and that the inherent aesthetic arrangements of light or sound can be transferred from one medium to the other and retain its ability to captivate.

   

Post 18- Visual Systems of Difference

Typefaces, maps and flags are all examples of visual languages that make differences within an overall paradigm very apparent and easily fathomable and serve as good reference points for what I aim to create. However, I need to bear the subject- music- in mind when forming the visuals as the aforementioned languages are (mostly) purposely reductive and this is somewhat at odds with the rich, diverse art that is music. Within this dichotomy of richness vs reduction is a balance which needs to be observed in my symbols.

The above typeface is a nice redrawing of Futura I stumbled across on Hetttty's blog, a collection of graphics and illustration by Belfast-based design student Heather Browne. 

Post 17- Software

A key piece of music analysis software I have used to obtain music data in this project is Sonic Visualiser, a free program developed at the Queen Mary University of London. It was developed as a visual aid to help musicologists and signal processing researchers study sound recordings and has many features and plugins that were useful to me such as the waveform generator, key detection and tempo detection. Aside from being useful for this project, it is genuinely interesting to use and exploring the myriad ways in which it can output sound into visuals can easily swallow up an hour.    

Post 16- Visual Testing

If I was to create a system that showed proximity between songs, it would need to have a modular framework within which difference could be expressed. I started to experiment with mapping the data around a circle (which conveniently alludes to a staple of visual language about music- the disc format and simultaneously, a clock, which would allow me to represent values against time in an easily understandable way), using sets of lines radiating out from a centre point with varying lengths to show different values. I tried several variations on this theme and received positive feedback on some of the results. However the capacity for these symbols to show difference was questionable, each one was too blobby, too indistinct which would make difference between them hard to fathom without studying them. 

The system needed to be more immediate and intuitive than this, with the viewer hopefully being able to spot differences instantly.

Post 15- Robert Horvitz Drawings

Found these beautiful drawings of patterns and structures by Robert Horvitz. It was something close to these which I originally envisaged I would use music data to create, however, aesthetic though they are, I have to factor in that want I want to create is a rule that will generate a series of (hopefully) comparable, contrastable objects and for this purpose the intricate details of this type of visual are not appropriate. 

Post 14- Feltron Blog

The Feltron blog is a constant stream of visual inspiration for anyone with an interested in rules-based, generative or information design. It's helpful to see successful examples of other designers selecting, ordering and representing information. Below are some recent highlights posted to it.

Post 13- Testing Continued

Just a couple of quick tests with different arrangements of the waveform. I probably won't pursue this idea as it isn't really viable as these are massively simplified versions of the wave's curvature and do not say much about the sound, though I do think they have a pleasing aesthetic that could be developed.

Post 12- Visual Tests

The first visual tests I did were based on an idea that the song data should be able to generate a distinctive shape. I tried to think of other ways of doing this than just an X, Y coordinate system, where the value determined the placement of a series of points which would be joined to create this shape. I tried out some web-like patterns but as pretty and organic as they looked, they all ultimately resembled mesh-like patterns and would have been pretty hard to compare with one another as the data was visualised only in very fine detail. 

Next I tried a method of taking a shape and using data (which at this point was placeholder data of random numbers from the really useful and interesting site Random.org as I had yet to find out how to extract data from music) to determine the placement of it in relation to the previous shape (eg for a square, if x=3 then place the next square adjacent to side 3). This required some more complex rule-making but produced some interesting, clustered formations of shapes. However, this rule eventually failed when it became apparent that two songs with the same data except for 1 value could end up looking very different as each value had a big impact on altering the course of the shape development.  

My focus, after these first tests then shifted onto making a system that could show proximity, in which songs with like data would look alike and different songs would look different.

Post 11- Sound & Programming

I approached this project of converting music into visuals knowing little about the technical aspects of the former. Having never played an instrument, I've only ever enjoyed music as a listener and was therefore unfamiliar with some of the terminology I came across when reading certain articles about how sound can be interpreted visually. Other terminology I had heard, but did not wholly understand, for example 'decibels', a unit for measuring sound that is logarithmic (another term I needed to clarify). The Fourier transform is something I'm currently trying to get my head around as it is one way in which visuals to describe sounds are generated (such as Spectograms) and in the process, I have been led down a rabbit hole of other terms and concepts that are used to define it such as sine waves, oscillation and resonance.

It also became apparent pretty early on that, for my visual system to viably be able to generate visuals for any meaningful number of sounds, I would need to learn some kind of programming to create something that would help generate either the data for me to use to create the shapes in illustrator or ideally to generate the shapes themselves. Since I have as much expertise in this as I do in music, I sought advice and help from Louis Eguchi, a creative programming genius who is in the progress of creating some software that will make it possible for me to create a decent number of visuals. 

Post 10- Stephanie Prosavec

On a recent visit to the brilliant Pick Me Up exhibition of contemporary graphic art at Somerset House, Stephanie Prosavec's work really stood out as an important reference point for this project. In creating a visual system to represent music data my project shares a similar premise to some of her visual systems work, particularly that of the Literary Organism, a chapter of Jack Kerouac's On the Road broken down and translated into a sprawling structure that resembles a conglomerate of beautiful linear florae. 

Where my project differs is that, while Prosavec has created some great one-off pieces using her systems, I hope to create a system that can allow for comparison and observation of patterns that may help users get an intuitive idea of what the music may be like. Therefore, to help make the system able to fulfil this function effectively, I will need to be slightly selective with the data; too much information will make each symbol hard to compare to one another, too little and the ability to express difference will be lessened.

Post 9- Survey

The central problem that I am looking at addressing in this project is helping people, who like me, often find themselves bored with the music they usually listen to, to find new music. 

Before going all out trying to figure out how to create something to help people overcome this problem, I first needed to confirm that it was actually a problem that affected others and not just a pet peeve of mine. 

I devised a quick survey with the aim of answering this as well as shedding some light on how others go about seeking new music and how they interact with music.

Of the 31 people that responded:

-55% often found themselves stuck in a rut with the music they listen to.

-Most said their preferred way to find new music was through friends' recommendations, with 61% reporting this as a method they use. 

-94% said they considered their taste to be eclectic as opposed to centring around certain genres.

-Most listened to music in one of these ways (ranked in popularity)

1) on a computer through a media player such as iTunes 

2) through an mp3 player or iPod and 

3) through a streaming service such as Spotify.

It seems that, from looking at this, that getting bored with all your music is not something limited to me, that human intuition trumps algorithms when it comes to recommendations, eclecticism is emphatically preferred over single genres and that most of the music people listen to is that which they have chosen to download/buy and import into a media player. 

Also, whilst people have resoundingly given the thumbs up to eclecticism, half of them get bored with their music, is there a way that my system could facilitate the musical exploration that they would purportedly be open to?

Post 8- Soundcloud

The music streaming website Soundcloud features a visualisation of each song embedded in its player panel which allows a certain degree of visual comparison. 

For making a quick, intuitive comparison as required by my visual system, they do not work as well as the sheer amount of information distilled into a long timeline style shape creates a level of detail that makes the observation of general shapes and patterns more difficult than if it were simplified. 

The long format is definitely something I will try and avoid and will try to find a way of compressing a timeline into a shape that could fit well into a square grid as this is would be more suited for presenting elements of a system of difference.     

For Soundcloud's purposes however, they work well and are interesting to watch whilst listening to the music.