A talking AY chip

1. What the chip gives you

The General Instrument AY-3-8910 and its relatives (the 8912, and Yamaha's YM2149) appear in the Vectrex, the ZX Spectrum 128, the Amstrad CPC, MSX computers, the Atari ST and many arcade boards. They all offer the same small set of parts:

• Three tone generators. Square waves with a 12-bit period: f = clock / (16 × TP).
• One noise generator, shared by all three channels, with a 5-bit period.
• A mixer that turns tone and/or noise on for each channel. When both are on, the chip ANDs them.
• A 4-bit volume per channel (logarithmic, about 3 dB a step), or the channel follows the envelope.
• One envelope generator: a 16-bit period and a handful of shapes (decay, attack, and the repeating sawtooth and triangle). There is only one, so every channel that uses it shares the same shape and timing.

There are fourteen registers in all. No filters, no waveforms other than square, no sample playback. The usual way to make speech on this chip is to give up and play samples by hammering a channel's volume register thousands of times a second. That works, but it costs a lot of CPU and a lot of memory (see below).

2. What a voice is made of

Voiced speech has two parts. A source: the vocal folds snap shut about 80–250 times a second, which gives a buzz rich in harmonics. A filter: the throat and mouth resonate at a few frequencies, the formants, and boost the harmonics near them. Moving the tongue and lips moves the formants, and the first two formants alone mostly decide which vowel you hear.

The AY has no filter, so it cannot shape a buzz into a vowel. What it can do is put tones at the formant frequencies. Three square waves at F1, F2 and F3 make a recognisable but organ-like chord with no voice in it. The missing part is the buzz.

3. Making the buzz

The envelope is meant to shape notes over tenths of a second. Its repeating sawtooth (shape 8, \\\\) can also run much faster: with a period EP it repeats at f0 = clock / (256 × EP), which is 110 Hz at EP 53 on a 1.5 MHz chip. Musicians on AY and YM machines use this as a bass "buzzer" sound.

Now put a tone channel on the envelope. The channel plays its square wave at the formant frequency, but its loudness is the sawtooth: loud at the start of every pitch period, then dying away. That is what a resonance does when the vocal folds strike it: a ring at the formant frequency, restarted f0 times a second. The tone gives the formant, the envelope gives the pitch, and together they make a voice.

Locking the formant to the pitch

A buzzing voice is periodic: all its energy sits at exact multiples of the pitch. A tone running freely at 730 Hz over a 110 Hz envelope is not a multiple, so the pattern drifts against the pulses every period and gives a rough shimmer instead of a buzz. The chip's tone and envelope counters run from the same clock, so a tone period of

TP = 16 × EP / n

is exactly the nth harmonic of the pitch, as long as n divides 16 × EP. With F1 and F2 locked this way, 99.7% of the sound's energy (60 Hz–5 kHz) lands on harmonics, against 73% for free-running tones. Turn the lock off in the widget above to see the spectrum smear. By ear the difference is smaller than the numbers suggest, but locking makes the voice clearer in a mix. The cost is that a formant can be up to half a harmonic away from its target.

The lock has to run on the target machine, because the pitch changes every frame. Division is slow on 8-bit CPUs, so it is computed with multiplies and a 64-entry table of reciprocals:

Fq = round(F / 16)                    formant in 16 Hz units: one byte
n  = (Fq × EP × K + 32768) >> 16      nearest harmonic, 1..63  (K = 179 at 1.5 MHz)
TP = (EP × R[n] + 512) >> 10          R[n] = round(16384 / n): 128 bytes of table

Compared with exact division, the result differs only on rounding ties (51 of 5,859 cases, never by more than about half a period).

What the trick costs

• One pitch. There is one envelope, so there is one voice, and no chords while speaking.
• No formant balance. A channel on the envelope loses its own volume, so F1 and F2 come out equally loud. In real vowels F2 is often 10–20 dB quieter. F3 is quiet anyway, so it stays on a steady low volume.
• Restarts. Writing the shape register restarts the envelope, so it is written only when voicing starts after silence or a voiceless consonant.

4. Consonants

Vowels carry the voice; consonants make it understandable. Each one is built from the same few parts:

Voiced sounds glide between targets over 2–4 frames rather than jumping, and diphthongs ("eye", "oh", "ay") move to a second target during their last two thirds. The shortest sound is one 20 ms frame, which is just enough for a plosive burst.

5. From text to registers

1. Words to phonemes. Each word is looked up in the CMU Pronouncing Dictionary (about 126,000 English words and 135,000 pronunciations in ARPAbet, with stress marks). Unknown words are tried as a known word plus an ending ("-s", "-ed", "-ing" and so on), then as two known words joined, and only then guessed from letter rules. Guesses are shown in the playground so you can fix them with /phonemes/. Numbers are read as words. Words with two pronunciations ("live", "read") can be clicked to switch.
2. Phonemes to frame states. Each phoneme becomes a few 20 ms frames: what is voiced, which formants, and which noise. This stage handles lengths (stressed vowels longer, unstressed shorter), glides and plosive bursts.
3. Pitch. The phrase falls about 20% from start to end. Stressed vowels rise, and a "?" rises from its last stressed vowel. ^150 sets the pitch in text, @140 on a phoneme overrides it, and each frame moves halfway to its target so the pitch glides.
4. State to registers. Every frame: the envelope period from the pitch, each formant locked to a harmonic, then the mixer, volumes and noise period. This is the only arithmetic the target machine has to do each frame.
5. The chip makes the sound for the next 20 ms by itself.

In phoneme mode you can write ARPAbet directly, with stress, length in frames and pitch: HH AH0 L OW1:15@140 | W ER1 L D ?, and ,:10 is a pause of exactly ten frames.

6. What the CPU has to do

Very little, and the amount is fixed per frame rather than per sample. Once per 50 Hz frame, the player:

1. counts down the current run of identical frames, and reads the next state when it ends (1–4 bytes);
2. looks up the envelope period for the current pitch;
3. locks each sounding formant: about four 8×8 multiplies and one table read each, for one to three formants;
4. writes the registers that changed, plus the shape register at a voice onset.

Measured on the VTX Vectrex cartridge player in the MAME emulator, playing every phrase of a game's speech three times (3,189 frames, 0 register mismatches against the reference model). That player drives F1 and F2 on channels A and B with noise on B, and leaves channel C to the game's sound effects. A Vectrex frame is 30,000 cycles. Physical hardware has not been measured.

The Vectrex cartridge stores speech as words: runs of identical states, each a header byte plus up to three operand bytes, with phrases made of word numbers. 27 words, with their phrases, alternatives and tables, came to about 1.1 KB. The player plus its 128-byte reciprocal table, as built in a test cartridge with that data, came to about 2 KB, and it uses 20 bytes of RAM.

7. On other AY machines

Nothing here is specific to the Vectrex. Any machine with an AY or YM chip and a 50 Hz interrupt could do the same. The data is the same everywhere; only the clock-dependent constants change (the lock's K, and the envelope period for each pitch). The playground's Machine setting plays your text at each chip's clock, and on the YM2149's 32-step envelope for the ST.

These are estimates, except where the Vectrex figure is marked as measured. The model is a fixed cost per frame, plus a cost per locked formant, plus a cost per register written. Its Vectrex coefficients are fitted to the measured player: on similar phrases it gives 578 / 1,080 cycles against the measured 641 / 1,103. The other machines use instruction timings for the same steps, with the fixed part scaled from that fit. None of them has been built or measured. The sample-playback column is a rough share of the CPU for 4-bit samples at 8 kHz, for comparison.

8. Keeping it small

Speech that is stored compactly and played with little work fits in a corner of a cartridge or a game frame. The chart compares ways of storing the phrase in the playground, and the list below covers both storage and CPU time.

How the state runs are stored

The cartridge format does not store registers at all. It stores runs: stretches of frames where nothing changes. That is the voicing, the noise, and F1 and F2 in 16 Hz units, one byte each. Each run is one header byte, followed only by the bytes that changed. The player rebuilds the registers from the run every frame. Below is the phrase from the playground. Click a run or a byte.

• Write only what changed. Keep a copy of the registers and write only the ones that differ. For the current phrase that is – writes per frame on average instead of 14. On the CPC, where every write goes through the PPI, this matters most.
• Store states, not registers. Runs of identical states (voicing, noise, F1 and F2 in one byte each) are far smaller than 14 bytes a frame. The registers are rebuilt from a state each frame, which is the lock arithmetic above.
• Fixed pitch, no lock. A robot monotone at one pitch can have its tone periods computed in advance: no multiplies at all, only table reads and writes. With the lock off, a formant's period no longer depends on the pitch and can sit in the phoneme table.
• Drop F3. F3 carries about a tenth of the energy. Without it, speech uses two channels and leaves C for music or effects. It saves one lock and two or three register writes per frame. The Layout setting in the playground does this.
• Phonemes on the target. Storing one byte per phoneme instead of per run is the smallest format (about 7 bytes per second of speech, less than the text itself). The catch is a synthesiser and a phoneme table on the target, estimated at 1–1.4 KB of code and data, and more work per frame. It pays off only for a lot of speech.
• Run from the frame interrupt. The update is short and regular, so it fits in the 50 Hz interrupt on every machine above. If a game's main loop runs late, count the missed frames and step the speech forward, so it keeps real time.

9. Further possible optimisations

Speculation. None of these has been built or measured. They are ideas for the next step, with rough costs.

The player never divides: the harmonic lock uses multiplies and a reciprocal table (section 3). What is left is about four or five multiplies for each sounding formant, every frame. On the Vectrex that is roughly 130 cycles a formant, about 40% of the player's time with two formants. A Z80 has no multiply instruction, so there it is the largest single cost, about 320 T-states a formant. These ideas remove some or all of those multiplies.

Fixed pitches: one table per pitch

If speech uses only a few pitches, for example eight voice settings rather than a free intonation curve, every answer can be stored: TP[pitch][n] = round(16 × EP / n) for n = 1–63. The formant is stored as its harmonic number at a base pitch, and a small map converts that number for each of the other pitches. Each formant then costs two table reads and no multiplies, and the lock stays exact. The cost is about 190 bytes of table per pitch (126 for the periods and 64 for the map), so eight pitches take about 1.5 KB. The pitch can then only step between the stored values, not glide.

Incremental update: no multiplies on most frames

Within a run of the same sound, the harmonic number n rarely changes while the pitch moves a step or two per frame. Keep the tone period and a remainder so that 16 × EP = TP × n + r, and adjust them as EP moves, as a line-drawing algorithm does:

EP up by one:    r += 16;  while r >= n: r -= n, TP += 1
EP down by one:  r -= 16;  while r < 0:  r += n, TP -= 1

For any n of 2 or more, that is a few adds and compares, and the result is the same as dividing (an offset of n/2 in r gives rounding instead of truncation). Only when n changes, usually where a new run starts, does the player fall back to the reciprocal multiply. Free intonation and the exact lock both stay. The cost is no extra ROM and about 3–4 bytes of RAM per formant (TP, n and r).

Log units: adds and table reads only

Store the pitch and the formants as steps of a sixteenth of a semitone. A ratio then becomes a difference: the nearest harmonic is NEAREST[logF − logf0], and the tone period is EXP[C − logf0 − LOG[n]]. The tables cover the formants' range of tone periods and would take about 2 KB. The weakness is that a table period can be one off from the exactly rounded value, which makes the sound slightly less periodic. That would need measuring before it could be trusted.

Lock once per run

Do the lock only when a new run of the same sound starts, and leave the tone periods alone while the pitch glides within it. The multiplies are spread over the frames of each run, and nothing is added to ROM or RAM. During pitch glides the formants drift off the harmonics.

No lock, or everything worked out in advance

With the lock off, a formant's period depends only on the sound, not the pitch, so the player just reads it: from a 510-byte table indexed by the formant byte, or stored directly in the speech data. The envelope still strikes the tones, so the voice still buzzes, but less cleanly in a mix. Going further, the tone periods can be worked out entirely in advance and stored in the speech data: no arithmetic at all, but the pitch can no longer change while playing, and every run grows.

Comparison

"Fixed ROM" is paid once per program, however much speech there is. "Speech data" is the change to the data for each run of identical frames; today a run is a header byte plus 0–3 operand bytes.

The incremental update looks the most promising for general speech: no extra ROM, the exact lock and free intonation. For a game that speaks at a few fixed pitches, the pitch table is simpler. On the Vectrex the saving would be modest (perhaps 200–250 cycles a frame); on the Z80 machines it could roughly halve the cost of speech.

10. What else it can do

11. Use it on a Vectrex

The player is one AS09 source file, vtx_speech.asm: 787 bytes of code and tables and 22 bytes of RAM, with no other files needed. Write your phrases below, export them as speech_data.asm, and include both in your cartridge. The guide shows the few calls it needs, and the example cartridge (built from example.asm) says four phrases on buttons 1–4.

The repository also has a local editor (python vectrex/editor.py phrases.txt): write phrases in English, phonemes or SpeakJet codes, hear each one exactly as the cartridge will say it with a spectrogram of it, set the voices' speed and pitch range with sliders, choose words' pronunciations, export, and build a test cartridge with the bundled AS09 assembler. python vectrex/export.py exports from the command line. Phrases can also be written in SpeakJet codes (\HE \EH \LE \OWWW, or 128–199), which become VTX Speech phonemes.

Checked: built with AS09 and run in MAME, the player's registers equal the reference player on every frame (3,733 frames, random choices included). It has not yet been heard on a real Vectrex.

Credits and licences

• Ayumi, the AY-3-8910/YM2149 emulator that plays everything here: Peter Sovietov; JavaScript port by Alexander Kovalenko. MIT licence, included unmodified: vendor/ayumi.js, licence. Source: github.com/alexanderk23/ayumi-js.
• The CMU Pronouncing Dictionary, Carnegie Mellon University. BSD-style licence, included unmodified (gzip-compressed): vendor/cmudict.dict.gz, licence. Source: github.com/cmusphinx/cmudict.
• Vowel formants: G. E. Peterson and H. L. Barney, "Control methods used in a study of the vowels", Journal of the Acoustical Society of America 24 (1952). Consonant values are hand-made starting points.
• Inspiration: SAM (Software Automatic Mouth, 1982) and the formant synthesisers before it. No code from them is used.
• "Shall we play a game?": the opening line's word lengths and pitches were measured from the film WarGames (1983), so each word sits on its own flat pitch as in the film. Only the timing and pitch numbers are used; no audio from the film is included.
• "Daisy Bell" (Harry Dacre, 1892) is in the public domain.
• The voice model was developed for VTX, a language and toolchain for the Vectrex, where it runs on a real cartridge player. This page is a JavaScript port that produces exactly the same registers as the reference model (a test checks 2,631 frames).

VTX Speech © 2026 Michael Simonds. MIT licence (LICENSE); the files in vendor/ keep their own licences.