Mark Watt is the creator of Spacecraft Granular Synth , for iOS (iPhone & iPad standalone, AUv3 and IAA) and desktop (Mac and Windows VST, AU). I contacted Mark to find out how this very musical and creative vision of granular synthesis came to be. What follows is a conversation with Mark that covers his background, the design of the app and in-depth details about how the app works.
The inner workings
How did you choose granular synthesis?
It is an extremely powerful form of synthesis. It’s power lies in the fact that it can take on the essence of any sample you provided it with, so it can cover a huge range of instruments and textures. That’s where its power is. Even just exploring the landscape of a sample without any effects can provide a world of musical results. It’s something of the modern age, with the technology we have, it is the time for granular sampling to shine because we have more and more processing power to do it, even on mobile devices.
Everything you do in Spacecraft Granular sounds musical, yet granular synthesis has a lot of complexity
It can be very tempting to create a granular instrument with many controls that looks impressively complex and there is certainly a group of users who want to have complete control down to the deepest parameters. However, there is clearly also a niche for more streamlined, opinionated design for granular interface where you don’t allow the user to control all details but instead present the them with a limited set of controls that produce a hopefully musical result. The user doesn’t necessarily need to understand exactly what is going on behind the scenes, but they might still be inclined to jump in and explore. In the case of SC, I was aiming for an intuitive, inspiring, fun experience.
To make the simplified interface approach work, SC actually has to be quite complex behind the scenes. There are algorithms that aim to keep things musical as much as possible depending on the state of the controls. For example, when you move around the grain control in the top left, there are algorithms which automatically tweak the overlap, envelope and grain timings depending on the mode of operation. I spent a lot of time tuning the background stuff, it’s one of the challenging aspects of the design. One example of a control that’s hidden from the user is randomization of the timing of the grain firings. This parameter is often modifiable in granular synths, and if misused can result in horrible ringing artifacts. Since I was interested in focusing on the smoother style of granular, I decided to have this parameter tuned automatically in the background so that the user is not distracted by it.
How does the sequencer work?
The sequencer is probably one of the most unique features of spacecraft. The grains fire in tempo and follow the notes activated on the notes grid in an arpeggio sequence. The user can activate notes in the arpeggio either by tapping the grid or by playing MIDI or MPE. With this approach, you can play slower firing grains as an audible arpeggio, or if you play the grains faster with maximum overlap, in the top right of grain panel, the arpeggios blend together to become chords.
I wanted to make it very easy to achieve a smoothness sweet-spot that can be musically useful with granular, this happens when you move to the top right of the grain panel. I wanted to give focus to that rather than the stuttering, heavily randomized textures that are sometimes heard in granular. This arpeggiated grains approach evolved from the need to have the smoothest sounding synthesis with optimal grain overlap for a given polyphony. A downside to this is that playing many notes in a chord tends to result in a less smooth sounding synthesis for high frequency grains, but with the desktop version you can get around this by simply increasing the number of grains.
Having this arpeggio-based grain engine has some interesting consequences. You can get some pretty wild results with an arpeggio of very long duration grains in the bottom right of the grain panel. This mode of operation is not technically granular synthesis due to the long grain duration, I guess that’s an odd form of pitched sample layering which I suspect is quite unique to spacecraft.
What about the new VST version of Spacecraft Granular?
The VST is a bit more advanced than the iOS version, with some extra features: you can have a larger number of overlapped grains playing simultaneously, and at higher frequency. This means you can get a smoother sound when, for example, you are using high frequency grains many notes being played in a chord.
Any tips for using the app that may not be obvious?
The fact that you can play it with MIDI or MPE seems to be a features some users don’t explore. On iOS with an AUv3 host you can easily explore this just with the on-screen keyboard. When played with MIDI or MPE, it is sometimes good to enable the envelope on the settings panel of the grid, if you have the envelope on you can have the smooth attack and release of all grains associated with a given note. This works best when you are in the top right-hand corner of the grain panel.
Also there is a ‘Stereo Phase’ feature (currently desktop only) where you can get an impressive stereo separation with a mono sample. This is based on the Hass effect which delays the right and left channels with respect to one another. In normal sample playback this technique can sound terrible if misused, but when used in granular synthesis and applied independently for each grain, it sounds absolutely amazing, even with very large amounts of delay offset between left/right channels.
How do the envelope controls work?
The envelope feature is meant to be used when playing notes in a live performance sense, the X/Y panel controls the attack and release of any notes created either by tapping on the grid or via MIDI or MPE. This is a sort of global note-level envelope that will be applied to whichever grains are associated with a given note.
When you activate the envelope, two buttons appear above it, these represent functions that are more useful when performing via MIDI. The button above the envelope panel that looks like a dash kills all notes currently on. The other button to the right is linked to MIDI sustain cc64. If you hit a sustain pedal that button turns on. You can also change it manually via the screen if you don’t have a physical sustain pedal.
Background
You have a very interesting background, can you tell us a little bit about it?
I work in the space industry as a control systems engineer on interplanetary space science missions. My specialization is feedback control loops of spacecraft GNC (Guidance, Navigation and Control systems), this is where SpaceCraft Granular got its name from. GNC is a very challenging subsystem to work on but it’s absolutely fascinating. There is a lot of applied maths and physics involved in solving everyday problems which can sometimes be beautiful and mind-blowing.
Something I find very interesting is the overlap between control systems engineering and audio. You would be amazed at the overlap. Filters, frequency domain stuff, signal processing, Fourier analysis etc.. For example, a familiar second order low pass filter used in a controller might act on the measurement signal of the spacecraft’s physical motion, rather than an audio signal. Having this background makes it really enjoyable for me to explore audio synthesis as there is less of a painful learning curve to deal with and I can do a lot from first-principles.
I can’t imagine the scale of those projects, how long does the design phase last?
Mission development phases can last many years. For example, I worked on the Lisa Pathfinder mission for approximately 8 years which travelled to Lagrange Point 1 in 2016.
What aspects do you focus on?
Mainly the design and tuning of algorithms, modeling of a dynamics of complex flexible spacecraft, simulators, gyros and star trackers, sun sensors, thrusters, reaction wheels etc. This tends to involve a lot of coding, often for the purpose of designing and analyzing the control system.
Did you create any music apps before you started Spacecraft Granular?
No music apps before SpaceCraft, no. Nothing commercial. I messed around in Reaktor. It’s a playground for experimenting in audio. I’ve always been interested in experimental music. Using technology to generate sound, chaotic functions, feedback of information, I find it fascinating.
How did the idea for Spacecraft Granular begin?
I created a prototype of spacecraft whilst messing around in (Native Instruments) Reaktor. Reaktor is kind of a visual programming language for musical instruments, it’s immensely powerful.
My initial Reaktor experiment was never intended as a commercial product, it was just one of my nerdy interests I did in my spare time. From this, I could see the potential for a VST and also an iOS touch interface, which eventually led me to the c++ based JUCE framework. Then out of nowhere, I found myself prototyping the iOS version in c++ which then took on a life of its own.
Have you worked with C++ before?
I had very little experience in C++ before writing SC. I had done bits and pieces of C in the past, mainly manipulating other people’s code. However, I do have quite a bit of programming experience in various other languages, particularly Matlab, so I could get moving quite quickly. Writing SC was an opportunity to start to learn C++, a language that I absolutely love.
How did you arrive at such a great interface design?
I really had a blank canvas. It’s a load of fun to create an interface concept and experience its evolution over time.
I always had the intention to keep it as minimal as possible. I discarded many prototypes. My general approach for designing is to constantly rewrite from scratch. I will create a rough initial prototype, writing code as if it is not going to be the final code. Each iteration gives me insight as to how the final feature will behave. As I keep rewriting I find that I converge on what it should be. As you are writing it you are using it, as you are using it, it feeds back into what you want to modify. Eventually you converge on something that works. That’s how I prefer to do things when having the luxury of working to my own schedule.
Can you share any early iterations of the the interface?
You can see where it was headed. Two engines were there from the start.
The Experience
What was it like releasing your own app for the first time?
Wonderful, intense and kind of terrifying. The initial release was on iOS.
How did you market the app?
Initially it was just word of mouth. I actually didn’t tell anyone when I released it, the app just appeared in the App Store. That kicked off a thread on the Audiobus forum, I remember watching in amazement as the discussion grew. People then started posting videos on Instagram. Producers and artists started contacting me.
It felt pretty much viral at the start. One of the craziest things that happened was where the Indian producer A.R. Rahman used it in a documentary about traditional music vs modern technology. At one point, the SC GUI is shown on the big screen right the middle of this enormous orchestral finale, my jaw hit the floor. That was mad.
I initially intended SC predominantly as a standalone iOS app. I was only very vaguely familiar with the iOS AUv3 format, so I didn’t market SC as AUv3 compatible in the app store description. However, I made the mistake at the time of launch of enabling the option to use SC as AUv3 without having it properly AUv3 compatible at the time. I could see from discussions on the Audiobus Forum that there was a huge interest for a fully featured AUv3 version of SC where many users were focusing entirely on AUv3, using it in hosts like Audiobus, AUM and ApeMatrix etc.
Although the standalone version of SC was very solid at time of launch, the features needed for the AUv3 were missing at the time, in particular state saving. So, I was under some pressure to implement these missing AUv3 features throughout the following year.
What aspects of this experience have been most rewarding?
The whole experience has been massively rewarding in general, but the most important aspect has been in simply creating something and putting it out into the world, something that represents me, my art. Just doing that in itself is the most important thing. To be honest I had to pluck up a lot of courage to do that. You do feel quite vulnerable putting something like this out. I think the most worthwhile things to do in life also tend to be the scariest.
Having feedback from people that are using it is just great. Everything from casual use to actual producers. That’s really what you are trying to achieve at the end of the day, that people use your instrument.
It is great to be able to interact with users. I try to interact with every person that contacts me on a one-to-one basis. I only have a single product, so with the amount of users and the low frequency of contact, I can manage to respond to all emails etc.
Meeting other developers is also a great part of the journey. I’m an indie dev and do everything in my spare time and don’t attend all the conferences and trade shows, so I do see myself a little bit of an outsider in the industry, but that’s fine. I do get the odd chance to meet with other Audio devs and music industry folk and I can say that I’ve had entirely positive experiences, very cool, inspiring people indeed, positive energy.
How can your customers help you?
Users can be very helpful indeed, sometimes in unexpected ways. Especially at the start just after launch when I was working on new AUv3 features, users gave important feedback and were really patient and supportive during those early updates. Before I knew it, I had a team of voluntary beta testers
Something that is really helpful is for a user to leave a review on the App Store, that has a big impact on the perception of an app.
So much thought and design goes into making apps, I want to highlight that here so it can be appreciated.
Thanks! A huge amount of hard work, energy and passion goes into it.
Conclusion
Many popular iOS music apps are created by indie developers, like Mark, who work hard to create a great product. There will be more articles to come highlighting the person and the process behind the app. Check back here or subscribe to the newsletter if you would like to hear more Synth Talk.
Links:
Spacecraft Granular Manual
If you have suggestions or topics you want covered please contact me. 🙂