For the past few weeks I’ve been working on the first stage of a project that involves hacking Eric Boyd’s excellent Heart Spark PCB pendant and combining it with a sensor module I’ve made from scratch.
The Heart Spark pulses 6 LEDs to my heartbeat via a signal from my Polar gym chest band transmitter (that usually transmits / displays my heart rate on my Polar wrist watch). The Heart Spark is open source and Eric has kindly made the schematics, code, plus lots of other information available on the Sensebridge.net website pages for the Heart Spark. http://sensebridge.net/projects/heart-spark/
I’m very interested in how sensing technology can display vital signs and how this can be monitored and combined with other tech to give us an insight into our moods, and how our body is reacting to certain situations.
For this proof of concept, here’s the back story / premise I used when considering what I wanted to build and present at Rewired State’s recent (H)activate hack weekend at the Guardian:
“Geeks are very particular about their personal space, so what could be better than a mobile, wearable device that notes their heart rate when someone comes close. Whether you’re cool, calm and collected or get palpitations when a certain someone enters your space, it’s interesting to note how a particular person affects your physicality, whether that be geek love or geek annoyance. It’s not quite your heart on your sleeve, but round your neck!”
For this, I’ve created a heart-shaped proximity detector module to link to the Heart Spark – I’ve named this ‘Flutter’.
To create the Flutter module, firstly I experimented with various combinations of components: an IR proximity sensor, resistor, potentiometer, LED and transistor on a breadboard. The 3.3V coin cell battery that powers the Heart Spark could not give enough output, even when combining with a step up component which boosted the voltage up to 5V to power both the Heart Spark and the proximity sensor.
So I began a period of trial and error with various combinations of batteries, resistors, transistors and twiddling the potentiometer. In the end I gave up fiddling and got my oscilloscope out to find out what was happening. I got some very curious signals from the various battery combinations and step up module I was using, but in the end the solution was to get more juice to the IR proximity sensor. This meant powering it separately and that more batteries were needed. So I settled on a 3 x AA battery pack, for which I’ve made a fabric heart shaped pouch dangling on a twisted twill rope, as it needs to be housed quite close and precisely to the sensor via battery pack connectors.
I have to say, I really feel that powering wearable technology is one of the more irksome things about trying to get this technology into wearables and I’m going to blog about this soon. On that note, the reason I added a 10k Ohm potentiometer to the heart-shaped IR sensor was so that small tweaks to the voltage can be made to the sensor.
To house all the components and with the help of a hacksaw, I made a heart-shaped substrate out of stripboard and coloured it black with a marker pen.
Hacking the Heart Spark – I have hacked Eric’s original code so that the top 3 LEDs on the Heart Spark light up when the IR proximity sensor detects someone in my space and the bottom 3 LEDs pulsate along to my heart beat. When there’s no-one in my space, the top 3 LEDs turn off again. I have some more interesting ideas for the code to work with the Heart Spark in the future, but in the first instance my goal was to get the Heart Spark to acknowledge the IR sensor and light up from its signal.
To upload code from my laptop to the Heart Spark, I fangled an FTDI board (which I usually use with LilyPad Arduinos) with some header pins – so I could connect it to the headers on the Heart Spark. I also soldered two pin headers to ground and positive / signal pins on the Heart Spark, which allows wire connection between the Heart Spark and the Flutter module.
Stage 2 and next steps…
For the next iteration, I’d like to add a tiny camera to this work, set to log periodic photos of who or what situation is in front of the wearer. Plus find a way of grabbing the photo, heartbeat and proximity sensor data from the devices and send to my laptop via a sewable a LilyPad Arduino x-Bee transmitter. Sewable, because I’d like it to become part of the heart-shaped battery pack pouch.
I want to graph the data from the two sensors to plot how many times during the day I get fluctuations in my heart rate when someone enters my space and would place the photos from the camera at appropriate points alongside, to see who had made my heart flutter. I’d use this information to work out my physiological state and reactions to certain situations, and people throughout the course of a day.
Other additions I’ve thought about are to include a temperature sensor, to record if someone becomes hot or flushed in certain situations or people and an accelerometer to assess body language by the user’s posture.
Mass participant uses? I’d love to hold a speed-dating event where all the participants wore the “You make my ❤ flutter” device to attempt to determine people’s reaction to each other based on their physical data signals.
Whether we like it or not, we presently live in an economic / political era where we’re constantly told there’s not enough money in the community coffers and so resources will become increasing hard to source. The current UK Government is pushing responsibility for many things back at us via the ethos of Big Society as a way to manage shortfalls in community care – which is pretty depressing.
It is very cool and rather convenient then, that in the not too distant future, wearable technology may give us some help with our lifestyle needs and personal independence. We’ve been talking about ubiquitous computing and the social of media for a while now, but how the streams cross and actually become part of us in a much more intimate and useful way interests me.
For me, this is where shrinking computing components, e-textiles and canny design comes together. We’ve had futuristic looking clothing, research and development departments tweaking devices for years, plus utopian ideas gracing sci-fi and all manner of future gazing documents and films, but in reality we’re only just on the cusp of having the right convergence of media, technology and ideas for this micro, wearable future to start becoming a reality.
I feel that wearable technology is on the precipice of an exciting leap into mainstream culture, right now in my humble opinion, it’s at about the same point of development as personal computing was in the 80s: raw, unrefined, without standards, but new, exciting and full of possibilities – as opposed to the clumsy, bulky and unfocused history that prevailed the wearable tech of the past.
I don’t think it’s a coincidence that rapid prototyping technology, 3D printing, the culture of Makers and Hackers, Hackspaces and events such as Maker Faire are also interwoven in the history of pushing experimentation with wearable technology forward.
Makers and Hackers event, 2009.
Empowerment via tinkering, due to technologies such as Processing.org, Arduino, the various flavours of ‘duino microcontrollers, mbed, and particularly sewable microcontrollers, actuators and sensors such as LilyPad Arduino are pushing experiments forward. This is influencing exciting and progressive ideas of what technology means to people and especially into the realms of what people want tailored for them – rather than being antiseptic, turgid or created for commercial gain. All this has made for a liberated approach to creating and thinking what wearable technology could be and can be used for in 2011 and beyond.
Dr Jan Zimmermann demonstrating new, embedded LED embroidery technology at Smart Fabrics Conference, April 2011.
Earlier this year, in April 2011, I attended the 7th annual Smart Fabrics Conference, hosted this year in London. It’s the world’s symposium for smart, wearable and e-textile technology, and draws presentations and attendees from the cream of the commercial, research and academic side of the community. I observed and learnt a lot from this 3-day experience and noted how diverse and yet still fledgling this growing community is. Comparing my three-day encounter I had with this community to that of the Maker community where I am usually to be found, I feel there is quite a difference between the two. Obviously, there is a commercial difference, so I wasn’t quite sure how much detail of the latest tech was being revealed, but it was very good to hear from researchers and academics and of course the snippets from the commercial side. Speakers represented universities, R&D laboratories and big business, and I noted major funding players such as Adidas, US Army, and Philips, to name a few. It was very interesting to note the difference between this and the Maker / Hacker communities in terms of where their priorities and interests lie, and crucially where the development funding is coming from!
So what’s the commercial potential that is attracting so much attention right now? Well, it’s predominantly flourishing in entertainment, sports, fashion, medical, lifestyle, specialist environmental, space exploration and military areas. To elaborate, here are some examples and a smattering of links to the awesome wearable tech that’s out there, with a few examples of my own work thrown in…
Entertainment and performance: well I could almost just say Lady Gaga, whom has been integral in wearing ever-more complex stage outfits which incorporate all sorts of technology, opening audiences eyes and getting them to accept technology as part of her personality and show. But, all sorts of artists and performers: from ballet dancers to, traveling shows and musicians have, for some time, also been incorporating technology into their outfits and performances.
An electroluminescent panel, part of my Ghost Ship Porthole dress.
The Dare Droid is a startling biomechanic cocktail-making outfit that uses medical and other hacked hardware to mix cocktails in exchange for a game of ‘Truth or Dare’. The performance’s Raison d’être is to explore human interaction in public spaces.
Fashion: from temperature and light sensitive inks, LEDs and electroluminescents, to phones, screens and sensing tech embedded in everyday wear, there is a huge potential for wearable technology and e-textiles to become integrated into fashion.
The N12 3D bikini by Continuum is a stunning example of what can be made by 3D printers. It’s made with Nylon 12 and snaps together so there is no need for stitching and the flexible, intricate design of thousands of connected plates is just amazing.
Amy Winters awesome designs feature prints that use thermo and hydrochromatic inks, that creatively use patterns with the properties of the inks to reveal themselves or disappear depending on light and temperature.
In terms of my own work, Neon-Victoriana Queen is an example of an electroluminescent costume, which I’ve exhibited at Kinetica Art Fair and Maker Faire this year. Its inspiration lies in Japanese Harajuku street fashion, of which I am a big fan and in turn has influenced me to create a my own technology based sub-genre: ‘Neon-Victoriana’. Another is the Ghost Ship Portal dress (below), which uses electroluminescent panels.
My Ghost Ship Portal dress, 2011.
Sports: athletes are finding it ever harder to improve human record times. Basically it comes down to the minute shaving off of milliseconds from performances and so monitoring athletes via their vital signs is significant to gaining those new human achievements. Also, from the point of view of broadcasters, they would like to give us ever more exclusive visuals and sounds from the athletes experiences, plus to have reliable communicative technology to give the viewer an athlete’s perspective live and authentically. With the 2012 Olympics coming up the wish list for sports tech is really hotting up!
Textronics have some intriguing sport performance, safety and health monitoring wearables and e-textiles. These range from clothing which integrates sensing fibres for heart monitoring to polymers, with variable resistance properties that can behave as strain gauges, switches and sensors.
Medical and lifestyle: smart textiles and wearables are becoming able to deliver medicines more exactly, hygienically and topically, and to also monitor patients from home or traveling so patients are not confined to long periods of hospitalization or being indoors.
Public groups such as the Quantified Self are devoted to ‘self-knowledge through numbers’ via monitoring themselves. For example, to improve their health, make changes to their lifestyle or monitor their moods by keeping tabs on their personal data. They do this via a plethora of means such as medical monitors, gym equipment, weighing machines, plus keeping spreadsheets, notes and graphs.
An example of medical technology is the non-invasive wearable cardioverter-defibrillator, which can be worn by patients at risk of sudden heart failure, whilst their doctors assess their health and decide what future healthcare plans need to be made.
“You make my ❤ flutter" sensing mood wearable, 2011.
Eric Boyd’sHeart Spark is an LED PCB pendant which pulses to one’s heartbeat via a Polar gym heartbeat transmitter chest strap. Eric has made the Heart Spark open source, which has allowed me to study the circuit diagrams and code, which I have in turn hacked its code and attached to a heart-shaped proximity sensing pendant that I have made. Combining the two pendants, I have created a mood device – the proximity sensor detects when someone has entered the wearer’s space and lights up three LEDs on the Heart Spark, which meanwhile pulsates 3 LEDs to the wearer’s heartbeat. My next step is to add a way of logging the data from these moments and visualise it on a graph, to see if there are fluctuations in heartbeat when someone enters the wearer’s space. I’d also like to add a simple camera device to log the data against images. I call this new hack / wearable tech: “You make my ❤ flutter”.
Another mood wearable of mine is “Yr in mah face” which uses a temperature sensor to detect fluctuations in heat from breath of someone entering one’s space or a fluctuation in the wearer’s body heat from mood or circumstance.
“Yr in mah face” temperature sensing mood t-shirt, 2011.
Extreme environmental and military clothing: for example those working in extreme or dangerous environments need monitors to tell them when they’re safe or in danger, for example fire fighters, arctic workers, astronauts or those working with chemicals. Similar to the extreme environmental clothing needs, the military want the latest sensing tech in their clothes such as adrenalin sensing fabrics, monitoring, GPS, moisture and heat absorbing clothing.
At last week’s London Quantified Self meet-up, we heard about Hidalgo’s Equivital vital signs monitoring wearable, which was developed and tested by the US military for physiological monitoring in a military environment. It gathers signs of different human states and brings the data together for analysis, for example: how close soldiers were to heat stress, fatigue or no longer being functional.
So what’s next? Well, things are changing very quickly and I’ll probably have a different take on things in another six months (see some of my observations from Smart Fabrics Conf). What I do keep saying is that there’s still no killer app for wearable technology, so in that respect it’s still all to play for! I’m currently researching, keeping long lists of links and ideas, and of course a beady eye on what’s going on out there: in the Maker and Hacker communities, in the research labs, in funding and academia and in business of what we might want, call, use, need, wear and manufacture in terms wearable technology!
Comments Off on Some thoughts on sensing, smart, wearable technology and e-textiles
Posted onJune 2, 2011|Comments Off on Thinking Digital 2011, Sage Gateshead, Day 2
Even though I was worn out by all the information and ideas that Thinking Digitalday one thrust into my head, I was up early (for me) and bounced over to The Sage for day two’s talks. Here are the potted highlights of my day via my rambling notes…
Firstly for me, Matthew Postgate, controller of BBC R&D, whose perspective I always find interesting, plus having worked for BBC R&D under Matthew’s leadership I’m always pleased to hear about the deptartment.
His talk, entitled ‘“Who needs telly when we’ve got each other”: how broadcasters will thrive in the information age’, took a look at how broadcasting is shifting from the industrial age to the information age and how this will challenge and change broadcasting as we know it.
He kicked off by saying about his role “It’s about helping the smart guys do the ideas”. Matthew talked about BBC’s mantra – inform educate and entertain in terms of broadcasting being live and having an immediacy, topicality, persistence and a level of quality.
He also showed a couple of demos, one was R&D’s Surround Video project which uses a fish-eye mirror to project video around a room beyond the television, which give an immersive experience of ‘being there’. The other was from a BBC nature programme (AutumnWatch, I think) and I recall Matthew quipping “I never thought I’d show a trout in a presentation!”
He said that new context creates challenges for broadcasters, in terms of:
New entrants
Creative competitiveness
Radically reducing costs
Adopting a global perspective
Re-imagining our relationship with the archive
Understanding a two0way relationship with audiences
– Guaranteeing access: a digital public space
Matthew acknowledged the democratisation and choice of technology and content, which is giving everyone the opportunity to be their own broadcast channel. Looking forward, Matthew commented “This is very much a future that is going to be created by collaboration and the BBC is very much committed to open innovation and open research.”
At the end there was time for a couple of questions, he was asked about
listening to audiences and answered “We always start with the audience in mind’ and went on to say “you used to be able to send in a SAE (stamped addressed envelope) but new media takes this to a new level. Journalists engage with twitter and the BBC is trying to come up with a new language – it’s much more about that interaction.”
Dr Vincent W. Li, co-founder of the Angiogenesis Foundation gave a heartfelt talk about angiogenesis, or new blood vessel growth. Dr Vincent informed us that Angiogenesis helps the body heal wounds and grow essential blood vessels during our lifetime, but during times of disease the growing mechanism can get out of balance and that’s when ‘antiangiogenesis’ can occur. One of the diseases is cancer and he told us most cancers are not discovered until they’re in an advanced stage and hard to treat, so finding ways to stop them growing so quickly and spreading is crucial. Dr Vincent ponders, if antiangiogenesis is the trigger, how do we cut off the blood supply?
The Angiogenesis Foundation is looking at ways to prevent and starve various diseases of by investigating foods that act as blood supply inhibitors to diseases in both humans and animal companions. There’s also another interesting side to this, in communities where access to expensive drugs is prohibitive, eating to prevent or slow down disease is a way of using this concept to help people help themselves.
A list of the good stuff to eat!
In the future Dr Vincent and his team are looking to get a number score system added to nutritional food facts to display a food’s ranking on the inhibitor list and a set of tasty recipes for those who want to follow the diet as some foods, such as tomatoes, work better cooked or combined with others.
I found Dr Vincent’s talk fascinating as although I do like my caeks, I try to follow a healthy vegan diet. I was lucky enough to catch-up with Dr Vincent briefly before he left to ask a question and hear a couple that friends had for him. I wonder if his foundation and The Quantified Self (as mentioned by Walter de Brouwer yesterday) could work together on the numbers system?
Jer Thorp is a digital artist from Vancouver, presently living in New York, he gave a talk on reflecting histories with mathematical design.
He did a live demo of The New York Times Cascade data visualization project that he’s been recently working on. It gives a detailed image of how information flows via social media, so one can track the history of a story or event.
He also talked about his work on the National September 11th Memorial at Ground Zero in New York. He’d been working on it last year when he appeared at Thinking Digital in 2010, but had been under NDA to not speak about it till only recently. Jer was asked to produce an algorithm that would lay out all the names of the people who were killed on September 11th including those at the Pentagon. The 2900 names were to be places around the pools of the site, 1400 of these had adjacency requests for example friends or workers who had died together or were family. The clusters of names requested to be together could get quite large, the biggest was a group of 70 whose families wanted them to be together. Jer had the task of trying to get the names to work with panels, corners that would wrap and panels that didn’t join. He also had to work with typography, plus the groups and departments of people to fit in too. What made it difficult was that there were to be no visible groupings or clusters or noticeable breaks – this proved very difficult. Jer solved the problem, part maths and part typography, with his bespoke algorithm. This kept the architect Michael Arad happy too! The memorial that will be made in bronze will open on September 11th this year.
He finished with a quick demo of Openpaths.cc from the NY Times R&D Lab, it’s a site to track travel history via the location data collected by iPhones. It tracks a path on a map and the user can look back to see where they’ve been on certain dates or show friends and family a location on a momentous occasion. It’s also a research project, so if a user happy to share your data it becomes part of a bigger research piece. Openpaths.cc allows one to revisit up to a year of travel data and people find they have an emotional response to replaying their personal narrative and history. This is quite a nice idea, but as I don’t have an iPhone this site doesn’t work for me – I wonder if there’s something similar for Android?
Tan Le, entrepreneur and co-founder of Emotiv neuroengineering company demonstrated the EPOC brainwave headset and software. Her introduction mentioned EEG as a non-evasive way at looking at how the brain is functioning and how the brain is constantly rewiring itself, learning and can rehabilitate itself – that evidence suggests our synapses are not hard-wired, but are changing all the time.
She talked about how up to now we’ve had to give machines commands to get them to do anything, whereas humans use body language also to convey information. Tan then showed data on her brainwave visualisation software and gave a live demo of it working with the help of a glamorous volunteer, Rob Colling of Internet Subtitling, who put on an spidery looking EPOC brainwave headset. The audience cheered as the headset software showed some activity in Rob’s brain in the form of regions lighting up in different colours on the back projected software, but the real fun started when Tan instructed Rob how to move a box up and down on-screen with his thoughts! Rob has written an excellent post about this experience.
To illustrate some of possibilities for usage of the Emotiv technology, Tan showed a video of someone driving a car with it (slightly scary but amusing) and an artist using it to create mood artworks.
I enjoyed the demo immensely and was amazed to hear from Tan that the headset is possibly affordable and even as she was speaking I was looking up the prices for the developer set and SDK, though hope the contacts on the headset would work with my big frizzy mess of hair – I have some ideas already – watch out!
To round off Thinking Digital, geek comedian Tom Scott performed a live experiment in social media privacy using Facebook, it was quite hilarious and caused a few people in the audience to check their privacy settings.
Afterwards, all that was left to do was skip over to the Baltic for the closing party and it was all the better for Ian Forrester being back this year after missing it due to #hisbrushwithdeath in 2010.
To sum up, I had a very enjoyable Thinking Digital 2012, it was ran like clockwork, it exhausted me as usual and I took some very interesting ideas home with me to think about. Plus I got to catch-up with great friends and made some new ones along the way – sadly there wasn’t any vegan caek – so next year I might bring my own ;-P
