Hardware Setup & Rundown
Filtering Audio and converting to MIDI Example
(Under the Video instead of right of the video as I stated)
THIS REQUIRED ADDITIONAL GATING AND MIDI EDITING, SEEING THE EXAMPLE SNARE RECORDING SUFFERED FROM LESS BLEED.
Because I used condensers on toms, I ended up only reamping the Snare and the Kick this way, and using the monitor for other elements.
Initially I had planned to use 360 footage in combination with VR goggles, in order to preserve the interactive ‘ambient’ peripheral vision (Gibson, 1979). Unfortunately, this didn’t work out seeing the smartphone-based headset I had experimented with didn’t allow for zero latency monitoring. Next to this it wasn’t possible to sync a VR headset player to a DAW using MTC. Therefore I experimented with projectors, seeking to create a hologram effect, which is actually closer to augmented reality in some ways. Upon trying this idea out, the distance my projector needed to create a large enough picture proved to be problematic, so I purchased three short throw projectors. To create the projections and holograms I experimented with different types of translucent plactic, so I could mount projection surfaces in the spots where the performer actually stood without blocking the view etc. Because professional projection plastic was expensive, I ended up using several layers of plastic attached to a frame that sat behind the drums. Because the Solenoid beaters functioned as arms, I only projected myself, and not the kit.
To cut out myself, the video and performances of the previous takes were all recorded to a green screen, so I could cut out the background and again preserve the hologram effect so the real room served as the background as if the person was actually there. Given the number of available projectors, screens, space, and instances of DAWS open, I was only able to project and reamp two actors (the guitarist and drummer).
Although reamping the guitar and bass was relatively straightforward, the drum reamping required more advanced editing. Because it was not possible to filter all signals sufficiently, only close mics were eventually reamped using the DADA machines 'Solenoid beaters'. To reamp the overheads, a monitor was used at the height of the cymbals. Although this didn’t provide a fully realistic representation of the kit in the space as when it was recorded, seeing the detail and type of energy expanded by the solenoid beater didn’t have human detail, it did sound very close to a real drummer when it comes to loudness and directionality. Next to this the visual element that the moving hammers incorporated also helped with synchronisation. Triggers and other sensors could make midi based drum reamping a lot easier, which can be explored in the future.