Most audio processing focuses on real-time applications: DAW plugins, MIDI instruments, analog devices, livestreaming, etc. While those applications are exciting and useful in many contexts, they have strong intrinsic limitations, which many have become so used to that it can be hard to imagine what could be achieved were those limitations removed.
One of most significant limitation of real-time audio is that the signal has to
be processed as it is received, with at best a small buffer to work with.
Anything more complicated will require delaying the output, which is often
considered very expensive. Besides, the amount of delay is itself a compromise,
usually about the lowest frequency that can be handled.
Even more significantly, the output cannot precede its corresponding input. This
might sound obvious and inevitable, but it rules out whole classes of effects,
including basic ones like reversing a sound. There is no way to play the end of
a sound before its beginning in real time.
With that being said, those limitations can be lifted by using a
simpletrick™: just process your audio off-line!
Obviously, this approach has drawbacks. First
of all, it requires a little more management: since the input has to be
determined before starting processing, it can't just be a steady stream.
Therefore, the user must specify a file (usually) to be processed, rather than
just playing something into the processor. In most environment, the output also
has to be explicitly specified: unless the processor is integrated in a
real-time environment e.g. as a sampler, an output file has to be specified.
Off-line processors are also rarely suitable for live previewing, otherwise they
might as well be real-time! This means that at best the feedback loop for the
user is a little looser, and at worst that live previewing will be glitchy,
albeit with a potentially tighter feedback loop.
Those usability limitations are inherent to off-line processing, but are also
accentuated by the fact that most off-line processors focus on technical aspects
rather than user-friendliness. A good example of that is
CDP, which is hardly friendly at all,
even when using one of its GUI frontends, but provides interesting routines that
could simply not be implemented in a VST plugin.
Once the limitations of real-time are lifted, many new possibilities can emerge.
First of all, off-line processing allows for batch processing. While the
feedback loop is looser, it can also be made much more powerful. Once can apply
a processing chain on hundreds of samples at a time and audition them
afterwards, or procedurally generate many effects chains and feed a single
sample into all of them. It is a great way of generating lots of new and
possibly unexpected material that can then be quickly audited.
Besides, algorithms can be made more accurate: any buffer can be tuned to the
input audio and the processing window is much more flexible than the fixed
buffer that has to be constantly fed to an audio interface.
I strongly encourage people who are interested in audio to give some attention
to off-line processing, may it be by exploring new processing algorithms or
focusing on the user-facing aspects. In particular, I believe that there is
great potential in integrating with real-time systems like DAWs and in improving
the feedback loop by providing quicker tweaking workflows and better
auditability.
I am personally currently investigating algorithms that can leverage the unique
advantages of off-line processing (especially pseudo-cycle distortions, which I
will cover in another article) in my screech
library.