AudioStretch Lite for iOS – Free!

I’m happy to announce a new addition to the AudioStretch family: AudioStretch Lite for iOS, available for free on the iOS App Store.  AudioStretch Lite is built from the same codebase as its non-free big brother, AudioStretch for iOS, and gives potential customers a risk-free way to experience first-hand AudioStretch, especially the amazing LiveScrub™ feature.  It has a few limitations – but they’re pretty generous compared to many other apps, and some users may find the Lite version is all they need.

FEATURES

• Simple uncluttered user interface
• LiveScrub™ : listen as you drag/hold the waveform
• Super-easy single-button looping feature
• Load any song in your music library*
• No ads or in-app purchases

LIMITATIONS OF LITE VERSION

• Max song duration 5 minutes
• Speed range 60% to 100%
• Pitch shift +/- 2 semitone

FULL VERSION ADVANTAGES

• Max song duration 60 minutes
• Speed range 0% to 250%
• Pitch shift +/- 12 semitones
• Keyboard/spectrum display
• Open Dropbox files
• Open email attachments

Pitch Detection in Flash

Was just doing some tidying up on my webserver and I came across an old demo I created quite a while ago, but never demoed publicly (as far as I can remember): a real-time pitch monophonic pitch detector, written in Flash.  It looks like this (click to run it!):

The pitch detector listens to the microphone input, displays the waveform, and shows the detected pitch as a red dot on a keyboard. It updates continuously, so as you sing, whistle, or play an instrument, you can see the red do move around.

I’m not planning on releasing the source code to this. I earn a living writing audio code other people, so if I give away all my secrets, I’d be putting myself out of business

But I can say a bit about how it works. Basically there are two main approaches to pitch detection: time-domain approaches, which typically use an autocorrelation; and spectral approaches, which typically use Fourier transforms and some simple pattern matching.  This demo uses a time-domain approach.

Time-domain approaches  are only useful for monophonic cases: that is, where there’s at most one pitched source at any given instant.  The idea behind autocorrelation is basically to see how well a signal lines up with a delayed version of itself for varying amounts of delay (or “lag”).  The very best alignment is for zero lag, but that’s not very interesting.  What is interesting is that you also get very good alignment (auto-correlation) at a lag that corresponds to the period of the waveform; the reciprocal of that is the fundamental frequency (i.e. the pitch or f0 ‘F naught’) of the sound.

That’s basically it, but as always with anything audio, there are loads of subtleties and tricks.  First off, the autocorrelation will be very strong not just for a lag of one period, but also two periods, three periods, etc. What that means is that a middle C (C4) can easily be ‘mis-heard’ as a note an one octave below (C3, whose period is 1/2 that of C4) or an octave-and-a-fifth below (F2, period  1/3 that of C4).  Depending on the signal and what sort of normalization you use, the auto-correlation peaks for multiples of the real period may be stronger than the peak at a lag of one period.

Next, there’s a problem of resolution: for high pitches, the period is really not very long.  For the highest note on a piano (C8, ~4186Hz), the period is less than a dozen samples (if the sample rate is 44.1 kHz) To get a musically accurate measurement of the pitch, you need to upsample (say, to 88.2 kHz), interpolate the peaks of the autocorrelation, or both.

There’s also a matter of CPU load.  Brute force auto-correlation is pretty expensive computationally. Fortunately autocorrelation can be performed more efficiently using Fourier transforms, as the autocorrelation of a signal is equal to the inverse Fourier transform of the product of the signal’s Fourier transform and its complex conjugate:  AC(x) = IFFT(FFT(x)FFT(x)*).  There are some subtleties there – you need to zero-pad the signals, otherwise you’ll compute the circular autocorrelation, which is far less useful.

Even if you get all this stuff right, monophonic pitch detectors using autocorrelation can be thrown off pretty easily, as real-world signals tend not to be as monophonic as we’d like.  Even with an instrument that physically only produces one note at a time (say a clarinet – ignoring advance playing with multiphonics), if you record it in a highly reverberant space, at any given time-slice the recorded signal will contain not just the current note, but also the echoes/reverberation of notes played slightly earlier.

In my more recent experiments with pitch detection, I generally use spectral approaches, as they can be applied to polyphonic pitch detection, and can be tweaked to deal with reverb in the monophonic case.

If this all seems like Greek – even without the mathematical notation, which tends to use Greek letters a lot! – well, that’s the nature of this domain… and that why it’s worth hiring experts

Incidentally, my AudioStretch app for iOS includes a spectrum analyzer graphically aligned with a keyboard display (which is playable). While it doesn’t do note-recognition per se, it shows you the spectrum of whatever notes are playing; by playing the keyboard you can audibly and graphically figure out which note(s) best line up with the spectrum. Eventually I’ll have the app automatically identify the notes.

For example, here’s AudioStretch displaying the spectrum for a major third played on a piano, specifically middle-C (C4) and the E just above it (E4).  The spectrum clearly shows the peaks at the C4 and E4, as well as at the harmonics of those notes.

Flash simulation of AudioStretch for iOS

At the moment, there’s no free version of AudioStretch for iOS.  I used to have a free download with a two-minute time-limit that could be removed via an in-app purchase, but I found that approach was problematic for a bunch of reasons I won’t get into here. I ended up switching to a regular paid app: you pay once to install.

Unfortunately this switch meant that few people could no longer try the app before buying.  In an attempt to partially address that, I’ve created a simulation of the iOS app entirely in Flash which runs in the browser of any reasonably fast PC or Mac. It looks like the real iOS version, and implements a lot of the essential features of the real thing.  It even has the LiveScrub™ waveform dragging feature.

It not as good as the real app of course. For one thing, in Flash the audio output has considerable latency, which means that there’s a fair bit of delay between actions (for example hitting the play button) and when sound actually starts going out the speakers. The latency is not too bad on Mac; on Windows it can be substantial.  On iOS, there’s basically no noticeable latency – you drag the waveform, and it plays perfectly in sync.  Another big difference is file formats – the Flash version only supports mp3 files (which must be on your local drive), whereas the iOS app can handle pretty much anything iTunes can – wav, aiff, mp3, m4a/aac.  And finally, there are loads of features in the iOS version that are not in the Flash version, notably the keyboard/spectrum display.

Despite these limitations, I hope you’ll find it pretty cool

AudioStretch Review on PlayRightAway

Just got a very positive review of my AudioStretch app for iOS on Gerry Malloy’s PlayRightAway channel on YouTube.

Malloy is a UK-based guitar tutor with 25 years of experience.  Many thanks for the review, Gerry!  Check out all the other excellent gear and software reviews on Gerry’s YouTube channel  and website.

 

AudioStretch in top 100 “What’s Hot” iPhone Music apps!

Daily downloads of AudioStretch were languishing for a couple of weeks, then suddenly on 31 Jan, they shot up by a factor of 16x in one day!  As far as I can tell, it’s because AudioStretch is now among the 100 “What’s Hot” app in the Music section of the App Store!

I don’t really know why it suddenly appeared there. Whatever the reason, I’m not complaining, and from what I’ve read, other app devs who get featured in “What’s Hot” have little idea how the got there.  There’s speculation that it’s some combination of getting good reviews, having a few updates, and Apple reviewers liking the app. But no one knows for sure. As far as I can tell, you don’t get into “What’s Hot” because you have a lot of downloads; it’s the other way around.

AudioStretch is still a free download, with $2.99 in-app purchase to play beyond 2 minutes. I’m anxiously awaiting figures for 1 Feb, and really curious whether this sudden rush of downloads results in a surge of in-app purchases!

 

Emailing Passwords

Many websites have some mechanism to change your password if you forget it. Typically you click a “Forgot your password?” link, then enter your email address. The site then sends you an email containing a time-limited link to a page where you can enter a new password.  Pretty standard stuff.

Why don’t they just email you your current password?  Because it’s considered really bad practice. Security-wise, it’s roughly equivalent to sending a postcard through regular physical mail – as it gets passed along, people can read the contents of it if they want to. You wouldn’t want someone to scribble your password on a postcard!  The other reason is that if they’re able to email your password in plain text, it means they’ve probably stored it in plain text (or at best stored it in encrypted form, but in using reversible encryption).

And yet…  even if you would never email passwords yourself, some sites will happily email you your current password in plain text form if you tell them you’ve forgotten it.  Typically this happens with smaller sites that apparently don’t know much about security.

But it also happens with some fairly well-known ones. Here’s one, from the Singapore branch of job site Monster.com. As a publicly-listed company (NYSE:MWW) with a market-cap of $600M, you’d think they’d know better.

So what to do?  Ideally, use a different password for every site. If that’s too much, at the very least, don’t reuse any important passwords, e.g. if you’re signing up for an account at some tiny online retailer, don’t use the same password that you use for your bank account.

Besides protecting yourself, dropping a note to offending websites might encourage them to improve their password security.  And finally, public naming and shaming might help, via Facebook, blog posts & comments, or submitting to sites like Plain Text Offenders (which I just found today).

 

Using Apple’s vDSP/Accelerate FFT

If you want to write code for signal processing on the Mac or iOS, you really should take advantage of Apple’s Accelerate framework. It provides an extensive library of highly optimized mathematical functions suitable for a wide range of signal processing applications.

While many of the functions are fairly straightforward to use, and well-documented in the vDSP Programming Guide, the FFT functions are a bit maddening, especially when doing FFTs of real signals, which is pretty much always the case when dealing with audio.

The following code example shows how to do a real-to-complex FFT of a real vector; convert from complex representation to magnitude and phase; convert it back to rectangular/complex representation; and do complex-to-real FFT, with the correct scaling to get back to the original signal.

//  main.cpp
//  AccelerateFFT
//
//	Demo of how to use Apple's blazingly fast but maddeningly confusing
//	real->complex and complex->real FFT functions.
//
//  Created by Gerry Beauregard (g.beauregard [at] ieee.org) on 2013-01-28.
//
//	Use this code however you like. No credit required, but if this code
//	saves you some hair-pulling, a hat-tip and kind comment is always
// 	appreciated  

#include <stdio.h>
#include <Accelerate/Accelerate.h>

const int LOG_N = 4; // Typically this would be at least 10 (i.e. 1024pt FFTs)
const int N = 1 << LOG_N;
const float PI = 4*atan(1);

int main(int argc, const char * argv[])
{
	// Set up a data structure with pre-calculated values for
	// doing a very fast FFT. The structure is opaque, but presumably
	// includes sin/cos twiddle factors, and a lookup table for converting
	// to/from bit-reversed ordering. Normally you'd create this once
	// in your application, then use it for many (hundreds! thousands!) of
	// forward and inverse FFTs.
	FFTSetup fftSetup = vDSP_create_fftsetup(LOG_N, kFFTRadix2);

	// -------------------------------
	// Set up a bunch of buffers

	// Buffers for real (time-domain) input and output signals.
	float *x = new float[N];
	float *y = new float[N];

	// Initialize the input buffer with a sinusoid
	int BIN = 3;
	for (int k = 0; k < N; k++)
		x[k] = cos(2*PI*BIN*k/N);

	// We need complex buffers in two different formats!
	DSPComplex *tempComplex = new DSPComplex[N/2];

	DSPSplitComplex tempSplitComplex;
	tempSplitComplex.realp = new float[N/2];
	tempSplitComplex.imagp = new float[N/2];

	// For polar coordinates
	float *mag = new float[N/2];
	float *phase = new float[N/2];

	// ----------------------------------------------------------------
	// Forward FFT

	// Scramble-pack the real data into complex buffer in just the way that's
	// required by the real-to-complex FFT function that follows.
	vDSP_ctoz((DSPComplex*)x, 2, &tempSplitComplex, 1, N/2);

	// Do real->complex forward FFT
	vDSP_fft_zrip(fftSetup, &tempSplitComplex, 1, LOG_N, kFFTDirection_Forward);

	// Print the complex spectrum. Note that since it's the FFT of a real signal,
	// the spectrum is conjugate symmetric, that is the negative frequency components
	// are complex conjugates of the positive frequencies. The real->complex FFT
	// therefore only gives us the positive half of the spectrum from bin 0 ("DC")
	// to bin N/2 (Nyquist frequency, i.e. half the sample rate). Typically with
	// audio code, you don't need to worry much about the DC and Nyquist values, as
	// they'll be very close to zero if you're doing everything else correctly.
	//
	// Bins 0 and N/2 both necessarily have zero phase, so in the packed format
	// only the real values are output, and these are stuffed into the real/imag components
	// of the first complex value (even though they are both in fact real values). Try
	// replacing BIN above with N/2 to see how sinusoid at Nyquist appears in the spectrum.
	printf("\nSpectrum:\n");
	for (int k = 0; k < N/2; k++)
	{
		printf("%3d\t%6.2f\t%6.2f\n", k, tempSplitComplex.realp[k], tempSplitComplex.imagp[k]);
	}

	// ----------------------------------------------------------------
	// Convert from complex/rectangular (real, imaginary) coordinates
	// to polar (magnitude and phase) coordinates.

	// Compute magnitude and phase. Can also be done using vDSP_polar.
	// Note that when printing out the values below, we ignore bin zero, as the
	// real/complex values for bin zero in tempSplitComplex actually both correspond
	// to real spectrum values for bins 0 (DC) and N/2 (Nyquist) respectively.
	vDSP_zvabs(&tempSplitComplex, 1, mag, 1, N/2);
	vDSP_zvphas(&tempSplitComplex, 1, phase, 1, N/2);

	printf("\nMag / Phase:\n");
	for (int k = 1; k < N/2; k++)
	{
		printf("%3d\t%6.2f\t%6.2f\n", k, mag[k], phase[k]);
	}

	// ----------------------------------------------------------------
	// Convert from polar coordinates back to rectangular coordinates.

	tempSplitComplex.realp = mag;
	tempSplitComplex.imagp = phase;

	vDSP_ztoc(&tempSplitComplex, 1, tempComplex, 2, N/2);
	vDSP_rect((float*)tempComplex, 2, (float*)tempComplex, 2, N/2);
	vDSP_ctoz(tempComplex, 2, &tempSplitComplex, 1, N/2);

	// ----------------------------------------------------------------
	// Do Inverse FFT

	// Do complex->real inverse FFT.
	vDSP_fft_zrip(fftSetup, &tempSplitComplex, 1, LOG_N, kFFTDirection_Inverse);

	// This leaves result in packed format. Here we unpack it into a real vector.
	vDSP_ztoc(&tempSplitComplex, 1, (DSPComplex*)y, 2, N/2);

	// Neither the forward nor inverse FFT does any scaling. Here we compensate for that.
	float scale = 0.5/N;
	vDSP_vsmul(y, 1, &scale, y, 1, N);

	// Assuming it's all correct, the input x and output y vectors will have identical values
	printf("\nInput & output:\n");
	for (int k = 0; k < N; k++)
	{
		printf("%3d\t%6.2f\t%6.2f\n", k, x[k], y[k]);
	}

    return 0;
}

Note that in order to do anything really interesting with audio on the Mac or iOS (for example the audio time-stretching in my AudioStretch app), there’s loads of other stuff you need to learn how to do: setting up a real-time audio input and output; how to grab and window (Hanning, Hamming, etc.) frames of audio data; how to manipulate signals in the frequency domain; synthesis using overlap-add methods, etc.  Time-permitting, I might cover some of that in future posts.