Category: Interactive

Interaction design and development

Glimmer

Glimmer by Micro Audio Waves and Rui Horta

Glimmer is the new album by Micro Audio Waves, and just like in the past Zoetrope, they partnership with the coreographer Rui Horta to create an unique and immersive concert experience. Just like in Zoetrope, I was once again privileged in being inside this epic journey. 

My work consisted in creating visual digital contents, writing a custom software in Touchdesigner to agregate videos and interactive visual contents, LED strips control, and also to trigger syncronized events to Ableton Live and GrandMA Light Mixer.

For me this was a massive challenge, to be able to do everything from scratch on a tight timeline, easy to manage and update,, and also easy to operate live.

In this video I’m showing a little bit of the Sync/Trigger Program made in Cycling 74 MAX. This one sends a trigger to Touchdesigner and simultaneously to Ableton Live.
If there is a precount bit in ableton Live it delays the trigger to TD. This way the videos are always in sync with the audio.

The link between Max and TouchDesigner is through OSC.
The link between Max and Ableton Live is via RTPMidi.

There are two computers running the same TouchDesigner program, in case of failure, the other computer takes control. There’s an HDMI switch to commute from one output to the other.

There are also two computers running the same Ableton Live Session, syncronized with an iConnectivity interface. My Max program sends a Midi command to iConnectivity, and this command is sent to both Live instances, to TazLite plugin.

The following images show a little bit of my visual art for Glimmer.

Glimmer
Direção, desenho de luz e espaço cénico: Rui Horta
Coreografia: Rui Horta, Gaya de Medeiros
Música original: Micro Audio Waves (Claudia Efe, C. Morg, Flak e Francisco Rebelo)
Intérpretes: Claudia Efe, Gaya de Medeiros, C. Morg, Flak e Francisco Rebelo
Conteúdos digitais e desenvolvimento de software: Guilherme Martins
Realização vídeo: Stella Horta
Direção de fotografia: Tomás Vieira
Vídeo ‘Liquid Luck’ e consultoria LED: David Ventura
Video ‘Liquid Luck’ e design gráfico: Marco Madruga
Figurinos: Constança Entrudo
Figurinos do vídeo ‘Pacific Airways’: Sofia Braga
Direção de produção: Pedro Santos
Direção técnica: João Chicó
Som: Artur David
Roadie: Sérgio Barata
Assistente de iluminação e palco: Pedro Guimarães
Comunicação: this is ground control

Dreambox_pt

Dreambox

Read this post in English

Dreambox é uma instalação que me remete para um recanto mental, onde sonhos, ideias e memórias habitam. Estas entidades mentais têm uma actividade própria, podem estar mais ou menos activas, pulsam em sincronia ou em dessincronia, podem adormecer completamente ficando apenas uma activa, ou nem isso.

Esta instalação começou com a necessidade de intervir num dos espaços do “Canto Circular” no Largo de Residências. O “Canto Circular” engloba o “Armário Comunitário” onde se pode fazer troca de objectos, e o “Guarda-Roupa Comunitário” onde é possível trocar roupa, e é aqui que Dreambox se encontra.

Identificado o local, encontrei estas caixas de plástico no Blab, e vi logo o seu potencial estético e gráfico.
Depois de alguns ensaios com as luzes decidiu-se que tinhamos bom material para trabalhar.

Queria colocar as caixas na parede de uma maneira não óbvia, e meio desconstruida, como se as caixas tivessem vindo a cair com o tempo.
Fiz uma série de ensaios em Photoshop até encontrar um padrão que me agradasse. 

Os anéis de luz são sobras de tiras de Led Neon de outros projectos.

A caixa de controle com toda a electrónica feita em PVC.

Para cada luz estou a usar um Mosfet, estes têm um Led na placa o que permitiu simular todo o comportamento das luzes no laboratório.

Como queria que as luzes tivessem vários comportamentos, decidi usar um sensor de movimento (PIR), e como redundância acrescentei um sensor de Micro Ondas que detecta alterações nos campos electromagnéticos. Com estas detecções os diferentes estados vão-se alternando.

Diferentes estados/comportamentos das luzes:

Heartbeat:

Todas pulsam em uníssono, remetendo às várias ideias/sonhos/memórias que nos pulsam em simultâneo ora por estamos em multi-tasking, ora por estarmos desfocados, ou por ter várias preocupações, ou por sermos simplesmente multi-sonhadores.

Random Fade In/Out:

O mesmo do heartbeat mas assíncrono. Ideias/sonhos/memórias díspares, que competem por atenção. Mente desfocada ou actividade mental excessiva.

Actividade Individual

Uma única luz activa, como que se o nosso foco mental estivesse agudo, consciência activa, mente preenchida com um único pensamento. Mindfulness, Nirvana.

Esta foi a primeira instalação a solo, depois de mais de uma década a desenvolver projectos comerciais para clientes.

Deixo um especial agradecimento à Sónia Sousa (@Lady.Maker), pelo convite e desafio para participar no “Canto Circular”, e por ter acreditado e confiado em mim para a realização deste projecto. À crew do Blab Colectivo Maker (@blablapt) por toda a ajuda e disponibilidade. E ao Largo de Residências (@largoresidencias) por suportar e incentivar estas iniciativas.

É também uma honra estar ao lado de “masya,, é uma instalação” do @m41.m, e ver como no fim estas duas obras co-existem e comunicam tão bem entre elas.

Dreambox

Dreambox

Para ler este artigo em Português clica aqui.

Dreambox is an installation that takes me to a mental corner, where dreams, ideas, and memories reside. These mental entities have a life of their own, they can be more or less active, pulsate in synchrony or asynchrony, they can fall asleep completely leaving only one active, or even none.

This installation began with the need to decorate one of the spaces in the “Circular Corner” (Canto Circular) at Largo de Residências. The “Circular Corner” includes the “Community Cabinet” (Armário Comunitário) where you can exchange objects, and the “Community Cabinet” where you can exchange clothes, and this is where Dreambox is located.

Having identified the location to intervene, I found these plastic boxes on Blab, and immediately saw their graphic and aesthetic potential.
After some tests with the lights, it was decided that this was the way to go.

I wanted to place the boxes on the wall in a non-obvious and somewhat deconstructed way, as if the boxes had been falling over time. I did a series of tests in Photoshop until I found a pattern that I liked.

Regarding the lighting, it consists of leftovers from Neon LED strips.

The control box with all the electronics made with PVC.

For each light, I’m using a Mosfet. These Mosfets have an LED on the board, which allowed me to simulate all the light behaviors at the lab.

As I wanted the lights to have various behaviors, I decided to use a motion sensor (PIR), and as a redundancy, I added a microwave sensor that detects changes in electromagnetic fields. With these detections, the different states alternate.

Different states/behaviors of the lights:

Heartbeat:

All lights pulsate in unison, referring to the various ideas/dreams/memories that pulsate simultaneously in our minds, whether it’s due to multitasking, being unfocused, having multiple concerns, or simply being multi-dreamers.

Random Fade In/Out:

Similar to Heartbeat, but asynchronous. Disparate ideas/dreams/memories competing for attention. An unfocused mind or excessive mental activity.

Individual Activity:

A single active light, as if our mental focus were acute, with active consciousness and a mind filled with a single thought. Mindfulness, Nirvana.

This was my first solo installation after more than a decade of developing commercial projects for clients.

I would like to give a special thanks to Sónia Sousa (@Lady.Maker) for the invitation and challenge to participate in the “Canto Circular” and for believing in me to carry out this project. To the crew of Blab Colectivo Maker (@blablapt) for all the help and availability. And to Largo de Residências (@largoresidencias) for supporting and encouraging these initiatives.

It is also an honor to be next to “Masya, é uma instalação” by @m41.m, and to see how these two works coexist and communicate so well with each other in the end.

Versailles

Centenary of Versailles

Partnership with Marvilab, a highly talented and creative boutique in Marvila (Lisbon).

This project consisted in a set composed by kinetic sculptures, moving furniture and dynamic lights.
This moving set had to be activated on a timed based interval. 

This video shows some making off and the final result of this great project.

My work consisted in developing a set of microcontrollers to drive the motors and lights.
Then there was a Maestro microcontroller that switched on and off all the others microcontrollers. The Maestro had the capability of programming the time based interval, and also how long the activation persisted.

The following video shows my process of developing a set of homemade PCB’s and the Maestro controller.
All microcontrollers are ESP32, and the Maestro is an M5Stack Core 2.

 

Vimana Sequencer

Vimana Sequencer

Vimana is a personal interpretation of a Multi-track Midi Step Sequencer with an Open Philosophy regarding the User Interface

I have been working on this project during the last 4 years, and today, I’m presenting it to the world as it is right now.
Vimana is also my submission to the MIDI Awards 2022.

Below is a list of the main features that are implemented:

16 independent MIDI channels/tracks
Main clock, division/multiplication per track
Step parameters: pitch, velocity, gate, retrigger, repeat, chord, inversions, sustain
Play mode: forward, reverse, pendulum, random, drunk
Built-in Quantizer with many different scales, and also the possibility of setting a custom scale.
Euclidean Generator

Vimana stands upon an open philosophy regarding the physical user interface. As a developer and a musician, I wanted to be able to build multiple instruments according to the need of the project, or the moment. For this I have created a modular user interface from the physical point of view, but also from the software point of view. Let’s say that we need a step sequencer with 5 steps and 3 tracks only, or a gate sequencer for drums and rhythms with as many steps and pages and tracks, it is all possible with this system, it’s just a matter of changing variables and voila. CPU is the limit here, and since we have a Teensy platform as the brain, we have a lot of free ground to play with.

Credits:
Concept: Guilherme Martins
Electronics: Guilherme Martins, Tarquínio Mota
Software: Guilherme Martins, Tarquínio Mota
Beta Tester: Ricardo Webbens

In the video below I have an intrument ensemble in Ableton Live, and I’m using Vimana to play them all. 

Other instances and prototypes of Step Sequencers, testing different layouts and user interfaces.

The video below shows how it all started, the 1st prototype of a step sequencer. When I accomplish this concept, I knew I could create something way better.

Drone Engine

Drone Engine

I have an Axoloti core for a couple of years, I never gave good use to it. The time  finally came. I have been thinking in a Drone Machine, something that could be fun and simple to use, with a lot of ground to explore and play. On the other hand, I also want to build a generic physical setup, that allows me to change firmware and experiment with new sound objects and synth architectures.

The Axoloti Core is a bare bone pcb, built with a powerful DSP, and it is made to be hacked and customized. It has a great community, that has been creating a huge amount of amazing sonic objects since the beginning. I don’t think I have invented anything, I only connected ready made objects like the oscillators and the effects, it is that simple.

This is the physical setup, it is a Teensy LC with two groups of 16 pots that are connected to multiplexers. Teensy is sending MIDI CC messages to the Axo, and all the parameters are being controlled this way. The major drawback is the MIDI low resolution (127 steps), this is noticeable in the oscillators pitch. One thing to do in the future is to connect the pitch potentiomers directly to the analog input on the Axo.
The black PCB boards I’m using were developed during Artica’s days. Luckilly I still have a bunch of these boards with me. 

Since the Teensy is connected to the Axo via USB (Axo has USB host port), and I didn’t want to have the usb plug showing on the back panel for several reasons, the USB is soldered directly to the Axo’s PCB.

If I still need to program the Teensy, I can still use a modified USB cable that connects to this JST header.

The Axoloti program is very simple, basically there’s 2 oscillators based on Mutable Instruments Braids algorythms, and a third very simple oscillator. Each of the Oscillators has dedicated LFO’s to the filter, and the Mutable Oscillators has also LFO to modulate color and timbre.
Than all the voices are mixed and send to Delay and a Reverb.
If you have an Axoloti and want to experiment this synth, you can download the patch here.

With a MIDI keyboard this synth can be played normally. By the time of this video I had the 3 oscillators being affected by keys, but right now only one oscillator receives MIDI information, so I can still play around with the other 2 as drone voices. 

If you want to know more about how to make a custom MIDI controller check my tutorials about MIDI programming and generic IO processing.

Physical Computing Tutorials

Physical Computing Tutorials

This is my humble contribution, showing how to interface with physical stuff.
I’ve been developing physical interfaces for a couple of years, from robotics to electronic musical instruments and other physical devices to be used in interactive installations.
In this Youtube channel, I will be sharing experiments hoping they will be useful for someone.
Here’s a list of the tutorials I have made so far.  

Simple USB MIDI in 3 parts

Reading analog and digital inputs using Multiplexers in 3 parts

Reading analog and digital inputs using Multiplexers in 3 parts

Reading analog and digital inputs in 3 parts

Modular Physical Computing

Modular Physical Computing

I personally enjoy pressing buttons, turning knobs,  see and touch physical devices, and if they make any bleep or bloops it’s even better.

Building physical control systems is not always easy, speccially if we need them to be sturdy enough to throw on a backpack, or take them to an installation or stage performance. This project started in ArticaCC, during a research project named “Interact”. This enabled us to research ways to build interactive systems related to software and hardware. These modular systems are easy to solder and easy to interface with, and because they are totally generic they can be used with Arduino, Teensy, RaspberyPI or any other microcontroller of your choice.

All these modules have an open source hardware philosophy and have a dedicated repository on Github. Feel free to use them as you wish.

Spatial Sound Exploration

Spatial Sound Exploration

Exploring spatial soundscapes with body location.

This prototype is a proof of concept for location-based sound exploration.

Each participant/VIVE tracker has a sound attached, this sound follows the participant inside the room. If the participant moves to a corner the sound becomes more audible in that specific corner. This video shows a proof of concept in a small room, the same concept can be applied in any room size, or even in large venues.
Follow the video’s link for a full description.