Whistle VAE-GMM

Beneath the waves,
the sea is never silent.

Whales and dolphins fill the deep with songs and whistles. Could a machine learn their shapes — well enough to whistle back?

Red fades first, then gold.
Sound never fades.

A few metres down, the sea drinks the colours one by one. Sound keeps going: four times faster than in air, and farther than any light can reach.

In the open blue,
a name is a whistle.

Some dolphins carry a whistle of their own and turn when they hear it. So we wondered: could a machine learn to hear these voices too?

01 · Listen

Slow a whistle down,
and it rises like a reef.

Spread a recording out over time and pitch, and every whistle rises out of the hiss of the sea like a ridge on the seafloor. Look down on it from above, and it becomes a single line: the whistle’s shape.

02 · Learn

Thousands of voices.
They swim together in schools.

The machine listens to about 63,000 whistles of false killer whales and gives each one a place in its memory. Whistles that sound alike drift together, the way fish gather into schools.

03 · Imagine

Swim between the schools,
and a new voice surfaces.

Pick a place among whistles it knows, and the machine turns it back into a shape, then into sound: a whistle no animal has made, in the voice of the ones that have.

Illustration · drawn by hand, not by the model

The sea hasn’t changed.
Now you hear it differently.

Your turn to listen. Bring a recording from the deep, or borrow one of ours.

04 · Try it

Bring a voice from the deep.
Hear what the machine makes of it.

Start with a recording made off Hawai‘i, or drop in your own. The machine finds every whistle in it, redraws it, and plays it back to you.

Checking the model…

Or bring your own recording

Imagine new voices

No recording needed: the machine dreams up brand-new whistles from everything it has heard. Choose how many, and if you like, ask for a certain kind.

05 · Encode

Write a message.
Hide it in a school of whistles.

Type a few words and a key. Each letter becomes bits, a checksum and spare whistles guard them, the key scrambles them, and every six bits become one whistle, sung among real whistles at the moments real animals called.

Checking the message server…

  1. Letters

    –

  2. Checksum

    –

  3. Key

    –

  4. Spare whistles

    –

  5. Whistles

    –

  6. Real rhythm

    –

06 · Decode

Bring a recording.
Only the key can find the message.

Drop in a recording and type the key. The decoder gets nothing else: no text, no timings, no record of what was sent. It listens for whistles, finds where the key says the message begins, reads each whistle’s shape and pitch, repairs what the sea broke, and checks the result.

Or listen with the microphone

Press Record, then play the message near this computer: from the Download WAV of Encode on another computer, a phone or a speaker. Press Stop when it has finished; the recording goes straight to the decoder.

No recording chosen yet.