The Bowes Museum Blog

My studio, TechxArtisan, is a creative technology studio in Guangzhou. In January of 2021, as we familiarised ourselves with the New Media Art industry, we collaborated with several artists who were using interesting technologies such as machine learning, computer vision and 3D printing in educational institutions in China.

It was still the first year of the pandemic then. Many exhibitions in China were affected and cancelled. Business was challenging but this gave us plenty of time to try out new things and to experiment with technologies like LED control and object detection. However, it wasn’t until I met Dave Bramston that I really appreciated the creative potential of these technologies. I saw a couple of Dave’s social media posts on the system most used in China, WeChat, about his lighting artworks made from waste plastic to promote a zero-waste lifestyle in Guangzhou.

Dave’s work turning waste into art fascinated me and I wanted to know more. So I got in touch and we talked on WeChat.

Dave Bramston’s artworks.

I knew before the pandemic that Dave was a super active British artist who had already delivered over 100 design workshops and creative lectures in China! His workplace, Bramston Studio, concentrates mainly on generating original thinking and fostering creative inspiration. This was exactly the artist I was looking to collaborate with.

One of several video meetings with Dave Bramston.

Although international travel was impossible during the pandemic, it didn’t stop us from brainstorming ideas for experimental creations. We held a video meeting to introduce each of our teams and started working on artworks applying interactive technologies. We came up with ‘Coloured Towers’, a work that detects the colours of people’s clothes and responds by changing the LED colours. This interactive light installation was an interesting concept for our two studios to collaborate in creating.

The draft of the concept design

Dave’s original idea was to enable light towers to interact with people’s hats. However, I modified the algorithm to detect and extract colours of clothes instead, by taking into account that people generally don’t wear hats these days. Or do they? Still in England? Like people in Downton Abbey do?

After a period of development, my studio built a set of equipment and LED strips and shipped them from China to Bramston Studio in the UK.

LED Controllers have our specially designed, UK-flag-shaped, 3d-printed cover case.

Dave used his magic with fabrics and turned these electric devices into beautiful light artworks.

Pictures of light stands in development.

Dave soon decided to scale up the project and present it in the exhibition he was curating at The Bowes Museum in the summer of 2022. At my studio, we were excited about the new project and the application of many LED strips on four tall pillars, each about 5 meters tall. We could imagine how great the completed work would be when fully lit up.

After a month of fabrication in China, we completed the testing stage and shipped all the equipment to the Museum in March 2022.

A final test at TECHxARTISAN studio, China, before the shipment to the UK.

In May in the UK, all the equipment was installed successfully. It was a delightful experience and a great collaboration between Dave’s team, the Museum’s team and TECHxARTISAN studio.

I met the teams of Bramston Studio and The Bowes Museum remotely and started working on installations for the exhibition.

This is the Museum team putting LED strips on all the pillars in the Museum’s temporary exhibition gallery.

Can you find me in this photo?

The exhibition, Journey in Colour, opened on 18 June and will close on 30 October.

My only disappointment is that I could not be there physically for the opening of the exhibition to see the final artwork with my own eyes.  But I do see the smiles of visitors in front of these Coloured Towers in the photos like the above. So, I express my thanks to the photographer who captured these moments at the Museum and shared them with us. I feel honoured to have been invited by Dave Bramston and to be part of this cool and interesting collaborative creation of the artwork for this exhibition at The Bowes Museum in the UK.

Even now, as I write this article, I still haven’t yet met Dave Bramston or the Museum team in person due to the pandemic.  But it won’t stop us from brainstorming creative ideas and working together on projects in both China and in the UK.

Technical Challenges and Experiments

To build the structure for this art installation, there were several technical challenges for our studio to overcome.  The essential hardware we applied in our project is a micro, GPU-backed, low-power computer, Nvidia’s Jetson Nano. The LED controller to manage an elegant communication between a host computer and LED light units and to run lighting animations by open-source FastLED library effectively is ESP32, a popular MCU control unit.

The basic procedures here are: 1) run AI algorithms in the microcomputer to detect

clothes people are wearing via a camera; 2) apply k-means to extract the dominant colour from clothes; 3) manipulate, blend and send RGB values to the LED controller, which controls LED strips attached to the artwork.

I don’t want to make this article too technical, but I would like to outline the challenges we encountered during the development of the works to realise the final art works. I have to say these are very interesting subjects to learn (for people like us, geeks, makers, electricians, engineers, developers, etc.).

How does Nvidia Jetson Nano detect humans’ clothes?

Jetson Nano allows us to apply an open-source AI algorithm and well-trained inference model for detecting human clothes in our project. The general procedure of detection by the machine is to grasp each image frame from a camera, to divide the frame into 10*16 matrix grids, to run the inference model to see whether these grids have a high confidence to fit into the feature of the pre-trained target object, in our case, clothes. Technically speaking, the model can detect any objects from this list of MHP Classes. It doesn’t have to be clothes.

Colours Travel Between Real and Digital World and RGB Problems

We humans can see the colours of things because of the light reflected from their surfaces and textures to our eyes. Machines “see” colours through a camera capturing frames from the real world which are transferred into the digital codes of colours. In this early step, the colours have already been mis-interpreted due to the condition of changing light environment, different parameters of focus, white balance and exposure of various cameras. This step is just a matter of capturing the colour data as close to the real world as possible and converting it into the machine world.

Using a colour checker board, widely applied in photography industry, to align digital colours with the real world as much as possible.

Once the machine is able to capture colours from frames and detect human clothes, the program applies k-means, a statistical trick, to extract the dominant colour from image blocks of clothes. The next is to average out all the RGB colours from these blocks into one dominant colour for the whole image frame.

Dave testing the program to capture the colour of his yellow shirt.

The way LED light delivers different colours to the human eye is through manipulating brightness and blending 3 colour channels: red, green and blue lights (RGB). The next problem is that LED light cannot really present the colours of grey or black if someone wears dark clothes. It just lowers its overall brightness. Moreover, the RGB LED strips that we used were ws281x, a specific series of LED strips, which tend to be a bit bluish in general. The blue channel requires adjustment in software levels.

My teammate testing the stability of the detection programme.

The last interesting fact related to RGB light performance in this project is colour iteration. When the machine detects the colour of clothes in the real world, it responds and projects a LED light colour on the visitor’s clothes. This strengthens the original colour of the visitor’s clothes with its projected lighting colour. Then the combined colour is re-fed back to the machine that presents another stronger new colour on its LED lights, and so on. Such a colour iterative feature makes interaction more fun for multi-participants whose clothes are different colours.

Electronic Engineering: How to Light Up That Many LED Strips?

This project uses many LED strips to cover the four tall pillars. It requires much caution, careful calculation and tests of electricity before powering up all the LED strips. It is a matter of safety in the use of electricity and is critically important. After all, blowing things up is the last thing we want to do. The following is a rough calculation of electronic capability for you to understand the general electrical scale.

For each pillar,

24 LED strips * 4 metres per strip = 96 metres

96m × 60 LEDs per metre = 5760 LEDs

Each strip can go up to 4 amps *

24 LED strips × 4 amps ≈ 96 amps for one set *

96 amps × 12V ≈ 1152W for one set *

* When all LEDs turn white (RGB channels all run in full power)

Powering up 4 pillars of LED strips at the same time in full power, requires up to 4600W power, at least in theory. The project itself, in fact, does not need to turn white when all RGB channels are in full power, for a long time. Moreover, most of the light animations and effects consume much less power than white in full power does. Yet its electricity is built in redundancy for safety.

For more technical details here’s my post here in the FastLED community of Reddit. Finally, I want to thank Quindor and Charles Goodwin. Upvotes from the community are also great supporters of our work. We would be happy to share our knowledge and experience of creating these artworks in collaboration with Dave Bramston and The Bowes Museum and to learn about other similar artworks.  So, do get in touch.

Thanks!

Author: Bolly Wang, TECHxARTISAN