Twinkle Tartiflette – an Arduino driven interactive word/music artwork

LilyPad Arduino is a great platform for rapid prototyping, for my standalone interactive art projects and wearable artworks. It’s also a fun way to learn about electronics and programming.

Here’s how I created Twinkle Tartiflette, an interactive artwork, using various LilyPad modules connected with conductive thread.
My inspiration came from a Stylophone Beat Box that I recently purchased as a present and had a play with. I pondered how one would go about making an interactive artwork using LilyPad components.

I decided that I wanted to combine words, image and sound into an interactive experience, brought to life by touching the words with a stylus. I began to think about how I’d build this and firstly decided on re-using the frequencies for notes worked out for a favourite ditty, Twinkle Twinkle Little Star, that I’d used in another artwork. I would transfer the first two verses word for word onto felt stars, one star for each verse.

Sewing Twinkle Tartiflette

There are 6 notes in the 2 verses so I needed to map out a schema for the conductive thread to pass from the words to the Lilypad, joining each word to the right note pin on the LilyPad – being careful to select conveniently located pins.
First I cut out 2 star shapes and began sewing the words onto them, not being an experienced embroiderer this wasn’t easy or terribly pretty.

After about a week of evenings I had two stars with conductive thread sewn words in the right order. I was mindful to sew the words carefully so frayed thread did not touch and cause any shorts – fabric glue is good for sticking down frayed thread and keeping close knots apart.

With the word stars completed it was time to deploy the main sewing schema – I’d mapped notes to the words and then words back to pins on the LilyPad.

Twinkle Tartiflette schema

After an intensive couple of weeks of sewing evenings later, I thought I’d sewn all the words to the right notes and pins, also adding buzzer and battery modules. There were some interesting insulation/bridging issues to be solved between the various paths of conductive thread, but I was ready to extract code ideas from my brain to see if it would compile!

The code I have written uses the speaker module to produce simple musical notes from connecting to the words with a stylus. I originally used a chart to match frequencies to the different notes.

Sewing Twinkle Tartiflette with conductive thread

With code loaded to the Lilypad, via an FTDI breakout board, it’s time to test – annoyingly there was a problem! The buzzer was not playing notes correctly, after some thinking and testing with a multimeter, croc clips and a single resistor – a solution was concluded – I’d need to add some resistors.

Unconnected the circuit is connected to high, but when the stylus touches a word it creates a simple circuit through the resistor and pulls it to low, but I needed some resistivity in-between. Looking through a ton of resistors 10k ohm seemed like a good fit, but where and how to add them was another question! A small LilyPad protoboard I had was just the job to solder the resistors to. I have six notes, so the protoboard was just right – I only had 5 x 10k ohm resistors, but found another resistor that was near enough to work (reading up later I found out that 20K pull-up resistors are built into the Atmega chip that can be accessed from software, so I didn’t really need need to add the resistors if I’d known that – hey ho, lesson learnt for next time!).

Soldering resistors to the protoboard

After some soldering, I had some more complex routing of conductive thread to do for the resistors on the protoboard. When testing I discovered I’d fix0red one problem, but had found another to debug! Earlier, I said to be mindful of the pins – I had accidentally connected to pin 13 which is the LED pin and has it’s own resistor which is set too low for this project. This showed up in resistance testing with the multimeter.

The fix for the wrong pin incurred some more unpicking and re-routing of conductive thread. I used an analogue pin as it was nearer and the least hassle to route to, this pin change required to be reflected in the code. Finally I decided the best thing to use for a stylus is a crocodile clip – which worked a treat.

Testing resistance with a multimeter

After all that, yay Twinkle Tartiflette lives! All that remained to do is tidy up the sewing, ensuring there are no trailing bits of conductive thread to cause shorts and gluing down anything looking like it was going to stray or come undone with fabric glue. Lots of lessons learnt, but hurrah!

Twinkle Tartiflette finished

Twinkle Tartiflette & Rain

I’ve made two videos for your delectation below – the first (00:44 secs) is a quick demo of me playing Twinkle Tartiflette.

This second video is an in-depth (05:40 mins) explanation of how I made TT, plus examples of debugging along the way – hope you enjoy!

Here is my code – you can use it via a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license & I’d love to know if you do!

/*
* Rainycat’s LilyPad stylo style: sound used to power Twinkle Tartiflette
*
* Uses a LilyPad speaker module to produce simple musical notes from touching words to the song
* For a chart of the frequencies of different notes see:
* http://www.phy.mtu.edu/~suits/notefreqs.html
*/

int NotePinC6 = 0; // words connected to play note C6 analogue pin!
int NotePinG6 = 12; // words connected to play note G6
int NotePinA6 = 11; // words connected to play note A6
int NotePinF6 = 10; // words connected to play note F6
int NotePinE6 = 9; // words connected to play note E6
int NotePinD6 = 8; // words connected to play note D6
int speakerPin = 3; // speaker connected to digital pin 3

// A note in one octave is twice the frequency of the same note in the octave
// below. We define here the frequencies of the notes in octave 8. To get
// notes in lower octaves, we just divide by two however many times.

#define NOTE_C8 4186
#define NOTE_CSHARP8 4434
#define NOTE_D8 4698
#define NOTE_DSHARP8 4978
#define NOTE_E8 5274
#define NOTE_F8 5587
#define NOTE_FSHARP8 5919
#define NOTE_G8 6271
#define NOTE_GSHARP8 6644
#define NOTE_A8 7040
#define NOTE_ASHARP8 7458
#define NOTE_B8 7902

// This is an array of note frequencies. Index the array essentially by note
// letter multiplied by two (A = 0, B = 2, C = 4, etc.). Add one to index for
// “sharp” note. Where no sharp note exists, the natural note is just
// duplicated to make this indexing work. The play() function below does all
// of this for you :)

int octave_notes[14] = {
NOTE_A8, NOTE_ASHARP8,
NOTE_B8, NOTE_B8,
NOTE_C8, NOTE_CSHARP8,
NOTE_D8, NOTE_DSHARP8,
NOTE_E8, NOTE_E8,
NOTE_F8, NOTE_FSHARP8,
NOTE_G8, NOTE_GSHARP8,
};

// Arduino runs this bit of code first, then repeatedly calls loop() below. So
// all initialisation of variables and setting of initial pin modes (input or
// output) can be done here.

void setup() {
pinMode(13, INPUT); // make sure 13 is high impedance

//pinMode(NotePinC6, INPUT); — analogue pin automatically input
pinMode(NotePinG6, INPUT);
pinMode(NotePinA6, INPUT);
pinMode(NotePinF6, INPUT);
pinMode(NotePinE6, INPUT);
pinMode(NotePinD6, INPUT); // sets the ledPin to be an intput
pinMode(speakerPin, OUTPUT); // sets the speakerPin to be an output

}

// Arduino will run this over and over again once setup() is done.

void loop()
{

// special case hack for this pin:
if (analogRead(NotePinC6) < 256) {
play(speakerPin, "C6", 50);
}
if (digitalRead(NotePinG6) == LOW) {
play(speakerPin, "G6", 50);
}
if (digitalRead(NotePinA6) == LOW) {
play(speakerPin, "A6", 50);
}
if (digitalRead(NotePinF6) == LOW) {
play(speakerPin, "F6", 50);
}
if (digitalRead(NotePinE6) == LOW) {
play(speakerPin, "E6", 50);
}
if (digitalRead(NotePinD6) == LOW) {
play(speakerPin, "D6", 50);
}
}

// ————————————————————————-

// To produce a tone, this function toggles the speaker output pin at the
// desired frequency (in Hz). It calculates how many times to do this to
// produce a note of the desired length (in milliseconds).

void beep(unsigned char speakerPin, int frequency, long duration)
{

int i;
long delayAmount = (long)(1000000/frequency);
long loopTime = (long)((duration*1000)/(delayAmount*2));

//for (i = 0; i = ‘A’ && note[i] = ‘0’ && note[i] >’ operator is a useful shorthand that (for integers
// >= 0) basically translates to “divide by two this many
// times”, so we will use that:

frequency = frequency >> (8 – octave_number);

// Actually play the note!
beep(speakerPin, frequency, duration);
}
}

About these ads

3 responses to “Twinkle Tartiflette – an Arduino driven interactive word/music artwork

  1. Pingback: Twinkle Tartiflette – an Arduino driven interactive word/music artwork | She Bytes

  2. That is so awesome it deserves a comment! Nice work.

    (Discovered whilst googling “interactive word art”)

  3. Fanx, Adrian – that’s most kind of you :-D