Internet Art

How I helped create a generative audiovisual app that runs in your browser

This past two months have been nerve-racking. But we’ve eventually created something to be proud of.

A group of four undergrad students. We’ve build a net artwork, or as programmers like to call it: a web app.

We named it avtest. In a nutshell, avtest is an interactive multimedia web app that gathers textual data via the YouTube API from either predefined or user-selected videos to create generative music and visuals. It’s still a work in progress*, but it’s near enough to completion to be shared to with everyone.

*It works in almost every aspect but its behaviour changes independently from platform to platform.

What does it do

On the surface, the artwork generates visuals and music when clicking on one of the 5 checkboxes or when inserting a valid link in the underlying text-box and checking the relative checkbox to its side. Each checkbox is “connected” to a YouTube video selected by our team. The text-box + checkbox option instead allows users to insert their own link to any youtube video; this has to be a special API link formatted through Google’s YouTube Data API platform. We imagined most people would not be able to retrieve such info, so we’ve added our own links to make the artwork function without much effort.

The 20 most-recent comments from a YouTube video are displayed in a random order below the group of checkboxes. In a similar way, the “waterfall” of letters is generated using random characters from a random comment among the list of 20. The sound composition that starts about 2–5 seconds in (performance upgrades are still on our to do list!) is also generated using text from the comments, first converted into ASCII values and then into notes playing a synthesisers I personally built using the Web Audio API and other Javascript tools (libraries).

Checking into the “About” box will take you to another page where some info about our team and the project are shown.

Details and code

I created and programmed the sound composition and designed the website (HTML, CSS, text, DOM elements, etc). The rest of the team has worked on setting up the data-retrieving system and designing the visuals.

The music composition is created using two javascript libraries: Tone.js and tonal. I have used Tone.js to build the sound of the synthesiser playing the notes. It’s a polyphonic synth with long decay and sustain. A chain of effects — delay, reverb, compressor, limiter — has been added also using the same library. I then created a pattern elements responsible for playing the sequence of notes. Such sequence is generated using of the YouTube comments as input and the tonal library to convert number to musical values.

The list of comments is first “translated” from an array of strings (text) into numeric ASCII values (numbers) and then scanned to find the most recurrent number — which happens to be a value between 0 and 127. Such number is converted first to MIDI and then to a note value (C, D, E, F …) through tonal. From here — using sorting tools, plenty of for loops and arrays, and a list of all possible scale modes, I build a system that automatically defines a custom root note and a scale mode for every input (API link). Such scale is eventually played by the pattern element mentioned above so that sound can be heard on the page.

An almost transparent ellipse shows up when the synth is finally loaded and is playing.

Elements of the visuals and the DOM design may not work/display properly at first try nor instantly. We are not all professional coders and our experience in front-end dev is limited — hence many improvements are still to be made. The project has not yet been tested on every current browser and device. The audio does not work when the site is open on Internet Explorer (IE doesn’t support the Web Audio API).

The entire javascript code, including sound and visuals, is accessible here: https://github.com/francescoimola/avtest/blob/master/machinemusic.js

and here is the main Github page: https://github.com/francescoimola/avtest

> The artwork is accessible here.

Contributors:

Francesco Imola : design + web audio 
Jameel Knight : API + visuals 
Anthony Luc : visuals 
Ryan Nguyen : visuals


Francesco Imola is a London-based musician, multimedia artist, and current Sound Design student at the University of Greenwich.

website| twitter | instagram | linkedin

A brief look into Form Art

Alexei Shulgin’s Form Art (1997)

Once web art at its peak, Form Art now seems like a forgotten achievement in the infinite world of new media.

Yet, the innovative thinking that came along with the short development of Form Art as part of the net.art “movement” has inspired a plethora of contemporary artists.

Alexei Shulgin was one of the prominent members of the movement. He “created” Form Art, but also collaborated with many net artists such as Olia Liliana, Natalie Bookchin, Heath Bunting, and Vuk Cosic in projects which marked a turning point for new media art in the 90s.

In his perfomance Cyberknowledge for Real People (1997), he handed out printed collections of critical texts, previously distributed only online, to shoppers in Vienna. This showed that net art was not medium specific at all, which until then was the predominant theory; it did not have to be experienced online.¹

Shulgin has also organised several software arts festivals, spent time in Moscow collaborating with the creative arts collective Electroboutique, and gained a reputation with his “386DX” performances.

Form Art was commisioned to Shulgin during his residency at C3 in Budapest in 1997. He affirms to have started developing what became his most popular project at that time out of a simple need to experiment with the formal interface of internet technology and reshape it something different.

“I had those buttons, test areas, checkboxes in my mind for a while. The initial idea was to use them not as they were supposed to be used — as input interfaces — but to focus on their shapes, their position on a page, and to try to animate them.” (Alexei Shulgin)

Rhizome describes Form Art as “an interactive, formalist art site navigated aimlessly by clicking through blank boxes and links”².

The power of Form Art comes from its “misuse” of the browser aesthetic and HTML conventions imposed to users interacting with the web. Unknown behaviours, glitchy checkboxes, and patterns of textboxes are characteristic of Form Art.

By playing with combinations of these “forms”, Shulgin manages to create an abstract work of contemporary art which updates itself over time, in tandem with software’s constant evolution. In fact, the work’s appearance relies largely on whichever operating system [and browser] the viewer is using to access it.²

My extremely minimalistic attempt at Form Art

Giving And Taking, Francesco Imola, 2018

Giving and Taking is a short project I realised in p5js (a JavaScript library for creative purposes). I took inspiration from Michael Samyn’s “1001 checkboxes”, an artwork which won the Form Art competition organised by C3 (Hungary’s Center for Culture & Communication Foundation) in 1997.

Not much needs to be explained about its concept and operation. “Giving and Taking” is an endless loop between discomfort and relief.

This project can be visited here (via GitHub).

This is a snippet of the Javascript sketch which makes the artwork run:

let checkbox = [];
let centerwidth = innerWidth / 2;
let centerheight = innerHeight / 2;
let fr = 120;
let animY0 = 0;
let animY1 = innerHeight;
let isOn = false;

function setup() {
noCanvas();
frameRate(fr);
Checkboxes();
}

function Checkboxes() {
for (var i = 0; i <= 1 ; i++) {
checkbox[i] = createCheckbox();
}
}

function draw() {
let velocity = 2;
let speed0 = velocity;
let speed1 = velocity;

checkbox[0].position (centerwidth - 20, animY0);
checkbox[1].position (centerwidth + 20, animY1);

animY0 += speed0;
animY1 -= speed1;

if (animY0 > innerHeight) {
animY0 = 0;
}

if (animY1 < 0) {
animY1 = innerHeight;
}

if (animY0 == animY1) {
isOn = Check();
}
}

function Check() {
if (isOn) {
Uncheck();
}
else {
checkbox[0].checked(true);
checkbox[1].checked(true);
return true;
}
}

function Uncheck() {
checkbox[0].checked(false);
checkbox[1].checked(false);
return false;
}

You may encounter some issues — which I have not yet figured out — if trying to resize the browser window after loading the page. Try resizing the window first and then reloading the site.


References:

  1. Bosma, J. (2017). A Net Artist Named Google. [online] Rhizome. Available at: http://rhizome.org/editorial/2017/jan/12/a-net-artist-named-google-1/.
  2. Rhizome. (n.d.). Form Art. [online] Available at: http://rhizome.org/art/artbase/artwork/form-art/.

Francesco Imola is a London-based musician, weekend photographer, and current Sound Design student at the University of Greenwich.

w. http://francescoimola.com/

tw. https://twitter.com/francescoimola

Online drawing and first steps with p5.js

Take Processing and Javascript, put those in bowl, add some HTML5 and CSS, mix them, and here you have a shiny new tool for creating easy-to-interact shapes, text, video, sound, and a myriad of other online objects.

In an attempt to put onto use the very elementary functions of P5.js that I have learned in the past week, I sat down and built an half-ever-moving drawing formed of text and lines. I was well inspired by Picasso’s minimalist portraiture.

drawingsxdrawings, Francesco Imola

I made use of a Javascript array to initialise the words to be displayed and added 4 Random functions, each one receiving the array as input and outputting the words in a different order, following the frameRate fps to set the speed. The rest of the code consists of the actual drawing of lines and positioning of the text.

It can be viewed here https://francescoimola.github.io/drawingsxdrawings/

The P5.js script used to build all of drawingsxdrawings

Francesco Imola is a London-based musician, weekend photographer, and current Sound Design student at the University of Greenwich.

w. http://francescoimola.com/

tw. https://twitter.com/francescoimola

Internet art is more alive than ever

Digital galleries, online trends and the post-internet world.

Take a peek at Rafaël Rozendaal’s website and you’ll instantly get captured the myriad of micro internet artworks the artist has shared with the world. Today, his website attracts 50 million visits per year.

But, why have I never heard of net.art before, you may be wondering?

You have probably seen iterations of Net Art — a term coined by Sloveian artist Vuk Cosic in 1995 — at least once in your life, but you’ve never imagined it to be as valuable as a Dalí painting. Internet art had an hard time trying to be considered as much as an art form as that which you can find in traditional brick-and-mortar galleries.

Heath Bunting, Irational.org

Internet art was a naive, vaguely defined, art trend that started out in the second half of the 90s. It has its roots in what we now know as Networked Art — even before the advent of the internet, artists were experimenting with electronic interconnectivity (e.g telematics, kinetic art, video art, and conceptual art).

Net Art often evaded museums and galleries and appeared exclusively on the internet or though other web protocols. The Internet was still relatively uncluttered at the time and very few people associated with art institutions were making use of the world wide web. This is one of the reasons why internet art never really crossed the boundary from experimental to mainstream practice.

An important reading on the history of Net Art is Rachel Greene’s Web Work: A History of Internet Art. It includes insights into pioneering examples of Net works by artists such as Jodi, Olia Lialina, Mongrel, and many others.

Jodi — %GoodTimes´

I believe Internet art, after having been overlooked for too long because of its intrinsic portability, to be now more alive than ever. The boundaries between what can and can’t be in a gallery have faded and artists can finally embrace the power of interconnectivity to form their practice. The internet as we experience it today has become canvas to hundreds of non-traditional concepts and ideas. It’s the era of not-very-well self-defining post-internet art.

While Net Art of the late 1990s used the Internet primarily as a medium, post-internet practices build on those earlier experiments using both online and offline formats like social media, mobile technology, digital branding, and infrastructures.

Cory Arcangel — Photoshop CS: 1060 by 2744 centimeters, 10 DPC, RGB, square pixels, default gradient “Spectrum”, mousedown y=1800 x=6800, mouseup y=8800 x=20180. 2015

Francesco Imola is a London-based musician, weekend photographer, and current Sound Design student at the University of Greenwich.

w. http://francescoimola.com/

tw. https://twitter.com/francescoimola

Using JavaScript and HTML to build a collection of Sound Trademarks.

“(h)over.me” is an ongoing collection of sound logos and sonic trademarks built out of a simple idea and a willingness to learn more about coding with CSS and JavaScript. Currently counting just more than 30 elements, the page is designed with interaction at its core. Visitors are given no implicit instructions on how to navigate the work because of its practical immediacy: hovering over the text — or clicking on it when on mobile — will highlight the name of a brand/company/organisation and play the audio sample of its relative sound trademark.

Possibly, over the next few months more content will be added. Some other features currently under development are:

  • A shuffle function that randomises the order of the words each time the page is reloaded.
  • Cross-fades between sounds.
  • A search feature.

UPDATES: Due to server issues, this project is now discommended and cannot be accessed anymore.

Here you can take a look at the HTML behind (h)over.me — it includes JavaScript functions to play and stop audio, basic CSS styling, and HTML5 <audio> elements (thanks to instaudio for the free hosting):


Francesco Imola is a London-based musician, weekend photographer, and current Sound Design student at the University of Greenwich.

w. http://francescoimola.com/

tw. https://twitter.com/francescoimola