English · 繁體中文

Melody and tone fit for Cantonese lyrics, measured rather than assumed.

Web app: https://cantojam.vercel.app — type lyrics, drag notes, hear it, find words that fit. No server, no upload, works offline.

Cantonese has six tones, and a melody that moves against them makes the words sound like different words. Lyricists have always known this. The rules of thumb (0243, 02493, 394052) encode it as a ladder of tone heights you match by ear.

cantojam checks that ladder against 37,877 sung syllables of real Cantopop and finds it is close, but wrong in a specific place. It then does two things no lookup tool does:

• check: you have a melody and lyrics. Does the melody fight the tones?
• contour: you have lyrics but no melody. The tones have already decided most of the shape. Here it is.

$ cantojam contour lyrics.txt --key "F major" --section verse --spread 2

今日天氣真係好
  G4 今 · 天 · 真 · ·
  F4 · · · · · · 好
  D4 · · · 氣 · · ·
  C4 · 日 · · · 係 ·
     今 日 天 氣 真 係 好
     1  6  1  3  1  6  2
  notes: G4 C4 G4 D4 G4 C4 F4

pip install -e .

No dependencies. Python 3.9 or newer.

Run cantojam model to print all of it. The headline numbers:

Tone-melody direction across 37,772 adjacent syllable pairs.

Direct contradictions are 2.8% of all pairs. 協音 is not a preference in commercial Cantopop, it is close to a hard constraint.

Where each tone actually sits, in semitones relative to a song's median pitch:

tone 1  +2.22   陰平
tone 2  +1.96   陰上
tone 3  -0.35   陰去
tone 5  -0.46   陽上     <- sits with tone 3, not tone 6
tone 6  -1.34   陽去
tone 4  -4.31   陽平     <- a cliff, not a rung

Two corrections to the four-level model fall out of this:

1. Tones 5 and 6 are not interchangeable. 0243 buckets them together. In the corpus, 6→5 rises 96.7% of the time and 5→6 falls 92.8% of the time. Treating them as the same height gets the direction wrong on nearly every instance. Tone 5 belongs next to tone 3.
2. Tone 4 is not one step below tone 6, it is three. Moving from tone 4 to tone 1 has a median interval of +8 semitones. Every other tone pair sits within a whole tone or two of level.

23 of the 36 tone pairs are lopsided enough (≥80% one direction) to be treated as hard rules. The rest are genuinely free, including every same-tone pair.

The two claims above are the ones that contradict the traditional ladder, so here they are with 95% Wilson intervals:

Sections have measurable pitch. Relative to the song median: verse -3, prechorus -1, chorus +1, bridge +2. Your verse-to-chorus lift is about four semitones.

Everything below also runs in the browser at cantojam.vercel.app, which is the easier way in:

• Type lyrics and watch the contour their tones demand appear as you go.
• Drag any note and the connector turns red the moment the melody starts fighting the words, with the corpus evidence on hover.
• Press play to hear it, or export MIDI into your DAW.
• Word finder: click a note, and it lists words whose tones that melody will actually accept, filtered by rhyme if you want. That intersection is the thing no other tool does.
• The melody already exists: paste a tune you wrote first, in numbered notation (1 2 3 5) or note names, or drop in a MIDI export.
• Fix the lyrics to the melody: every clash, with words that would sit on those exact notes instead. Click one and it goes into the lyrics, melody untouched.

It is a static page. The whole model is 109 KB of JSON and about 250 lines of JavaScript, so there is no backend, nothing is uploaded, and it works on a plane. web/cantojam.js is a hand port of the Python package, and tests/test_parity.py runs both over the same inputs and fails if they ever disagree.

$ cantojam check "係我" "G4 C4"
 X 我  ngo5     C4    -7

  X 係我 (tone 6->5) should rise; this melody falls.
    Corpus median +1 semitones over 1559 examples.

1 violation(s) in 2 syllables

Exits non-zero when there are violations, so it drops into a pre-commit hook or CI over a lyrics folder.

Two severities:

• violation (X): the corpus takes the opposite direction ≥80% of the time. A listener will hear the wrong word.
• unusual (?): no hard rule, but this move is rare. Rising off tone 1 onto another tone 1 happens 5.2% of the time, because nothing sits above tone 1. Singable, just uncommon.

The melody can come from numbered notation (簡譜) or a MIDI export instead of note names, which beats retyping a tune you already have:

cantojam check "今日天氣真係好" --degrees "1 2 3 5 6 5 3" --key "F major"
cantojam check "今日天氣真係好" --midi verse.mid

' lifts a degree an octave, . drops one, # and b bend a semitone, and 0 is a rest. A MIDI file is read with no dependencies, taking the track with the most notes and the top note of any chord.

check says which syllables fight the tune. fit says what to do about it.

$ cantojam fit "今日天氣真係好" --degrees "1 2 3 5 6 5 3"
   今  gam1     F4     .
 X 日  jat6     G4    +2
   天  tin1     A4    +2
 X 氣  hei3     C5    +3
   真  zan1     D5    +2
   係  hai6     C5    -2
 X 好  hou2     A4    -3

X 今日 (tone 1->6) should fall; this melody rises. Corpus median -3 semitones
  over 736 examples.
    phrases the corpus wrote as a unit:
    今日 -> 沒有    mut6 jau5    tones 65   -au    98x
        沒有天氣真係好
    今日 -> 爲何    wai4 ho4     tones 44   -o     33x
        爲何天氣真係好   sing 爲=wai4
    ...
    single characters that fit the notes, sense is yours to judge:
    今 -> 來       loi4         tones 4    -oi    237x
        來日天氣真係好

Every candidate has to satisfy the tone rules inside itself and at both seams with the syllables around it, so a fix at one clash cannot quietly break the next one. tests/test_parity.py checks that invariant directly: for every suggestion, substituting it and rechecking leaves that seam clean.

Phrases come first because the corpus wrote them as a unit. Single characters fit the notes but not necessarily the sense: the corpus's n-grams are raw sequences, so a real word like 舊日 and a boundary crossing like 日你 look alike by frequency and score the same PMI. There is no honest way to tell them apart from this data, so the two are labelled and separated rather than filtered.

A candidate is admitted on the reading the corpus sang it with, which is not always the default reading of those characters. 爲何 is offered as wai4 ho4 while 爲 alone reads wai6, and that one tone decides whether the line fits, so those cases print sing 爲=wai4 rather than being silently dropped.

Useful flags:

• --no-keep-rhyme lets a swap on the last syllable change the line's rime, which is protected by default
• --min-count 1 admits words the corpus used only once
• --allow-unusual keeps moves the corpus almost never makes
• --json for the whole report, if you are wiring this into something

cantojam contour lyrics.txt --key "F major" --center F4 --section chorus --spread 2

Each syllable is placed at the height its tone wants, then a beam search over scale-degree paths picks the line that best satisfies the hard tone rules while also resolving to the tonic or dominant, tracing an arc, taking the intervals the corpus actually takes, and echoing an earlier tone pair when it can. Add --json to pipe it into a DAW script.

The tone rules leave a lot of room: only 23 of 36 pairs fix a direction, so many legal melodies exist and the search chooses among them rather than taking the first that works. Measured over 300 corpus lines, the earlier greedy repair landed a phrase on the tonic or dominant 0% of the time; the search does it 55% of the time, with the same zero violations.

--spread widens the range without changing the shape. Tone height alone gives a compressed line, because tones only need to be distinguishable, not dramatic. Real melodies move further for musical reasons. Try 1.5 to 2.5.

This is a skeleton, not a melody. It fixes contour and leaves rhythm, phrasing, repetition, and every interesting decision to you. It is meant to unblock the blank page, not to fill it.

$ cantojam tones "今日天氣真係好"
  今  gam1     tone 1
  日  jat6     tone 6
  天  tin1     tone 1   polyphone: tin1/jik1
  氣  hei3     tone 3
  真  zan1     tone 1
  係  hai6     tone 6
  好  hou2     tone 2   polyphone: hou2/hou3

Polyphones default to the corpus's most frequent reading. Pin one per song with --override 話=waa2.

$ cantojam words --fits "今日天氣真係好:4" --rhymes 好 --limit 6
176 word(s) fit 今日天氣真係好 at position 4, where 真係 sits

  得到     dak1 dou2              tones 12    -ou    30x
  擁抱     jung2 pou5             tones 25    -ou    19x
  一早     jat1 zou2              tones 12    -ou    14x
  不到     bat1 dou3              tones 13    -ou    13x
  好好     hou2 hou2              tones 22    -ou    13x
  多好     do1 hou2               tones 12    -ou    12x

This is not a dictionary. It is a concordance of what the corpus's lyricists actually wrote: 12,782 words and phrases, each with its tone pattern, its rime, and how often it was used. For writing lyrics that is more useful than a dictionary, because every entry is already in register and already singable. It will not contain rare literary vocabulary, and it should not.

Without --fits it is a plain rhyme and tone search:

cantojam words --rhymes 好 --length 2         # rhymes with 好
cantojam words --tones "46" --min-count 5     # tone pattern 4 then 6
cantojam words --contains 心 --tones "1?"     # wildcards allowed

With --fits it intersects with the melody: a candidate must satisfy every tone rule inside itself and at both seams with the syllables on either side. The melody is fixed, so the tones it accepts are fixed, and only some words carry those tones.

from cantojam import build_contour, check, ToneModel

model = ToneModel()
model.required_direction("6", "5")     # 1, must rise
model.suggested_interval("4", "1")     # 8 semitones
model.section_offset("chorus")         # 1.0

check("係我", [67, 60])["violations"]
build_contour("我哋一齊去食飯", key="F major", spread=2)["notes"]

from cantojam import WordList
words = WordList()
words.search(rimes_with="好", length=2)
words.fitting(pitches, tones, start=4, length=2, rimes_with="好")

Be aware of these before trusting it:

• The corpus is 105 songs, 2000 to 2020. 林夕 and 黃偉文 wrote 60% of them. It models mainstream radio Cantopop and nothing after 2020. It does not model the contemporary indie register (Gareth.T, Kiri T, serrini).
• It is written in 書面語. Common 口語 characters barely appear, so data/colloquial.json supplements by hand. Coverage on colloquial lyrics is good but not complete. Unknown characters are reported, never guessed.
• Polyphones are resolved by frequency, which is right about 94% of the time by token. Use --override when it matters.
• Melody only. No harmony and no rhythm. Tone constrains contour, not everything.
• The word list is corpus vocabulary, not a dictionary. 12,782 entries drawn from 105 songs. Longer entries are raw n-grams, so a few cross word boundaries; the frequency counts let you judge. For a real Cantonese dictionary with 181k entries, use Canto-0243, which is excellent and which cantojam does not copy from.
• Contour output is a starting point. It resolves and arcs, but it has no sense of rhythm, phrasing, or what the song is about. A tone-correct line is not automatically a good line.

15.9% of syllables in the corpus are sung over more than one note, hiding about 6,500 notes that cantojam ignores by keeping each syllable on its first pitch. That was a convenience, so scripts/melisma.py tested whether it is safe.

The natural hypothesis is that a rising-contour tone (2, 5) should have a rising melisma and a falling one (4) a falling melisma. It is wrong:

The rising contours rise least (1.3%, n=1678), and the one tone that rises more than the rest is tone 4, the lowest and a falling contour (5.0%, n=501).

What does predict the movement is where the tone sits: r = −0.940 between a tone's measured height and the net movement of its melismas. High tones can only melisma downward, the lowest one can only go up. That is a ceiling and floor effect, not contour.

So keeping a melismatic syllable on its first note is correct, and the run itself is predictable from tone height rather than something to ignore.

python scripts/melisma.py Cantopop-corpus/Humdrum-files

The numbers above are descriptive: they describe the corpus the model was built from. The question that matters is whether the rules generalise, so scripts/evaluate.py runs 5-fold cross-validation split by song, builds a model from the training songs only, and scores it on the held-out ones.

Splitting by song rather than by syllable matters. Syllables inside one song are not independent, so a random split would put the same melody on both sides and inflate every score.

The task: given two adjacent syllables and their tones, predict whether the melody rises, holds, or falls.

The measured model beats the 0243 ladder by 10.7 points on songs it has never seen, and the gap is far larger than the spread across folds (sd ≈ 0.011).

The hard rules cantojam actually enforces, learned from training songs only, are broken by 4.22% of held-out pairs (fold range 3.51% to 4.99%). Compare that to the 3.91% in-sample figure from validate.py: the model loses only about a third of a point when moved onto unseen songs, which is the evidence that these rules are a real property of the genre rather than an artefact of 105 particular songs.

python scripts/evaluate.py Cantopop-corpus/Humdrum-files

The most useful thing you can do is add a song. Every number above comes from 105 transcriptions, so the model improves the moment the corpus does, with no code change at all. The gaps are specific and listed in corpus/README.md: post-2020 songs, colloquial lyrics, independent artists, lyricists other than the two who dominate the corpus.

cp corpus/TEMPLATE.krn corpus/X0001.krn   # edit it
python scripts/check_krn.py corpus/       # CI runs this on every PR

One song is a real contribution. Other open work, from wrong-result reports to rhyme extraction, is in CONTRIBUTING.md.

git clone https://github.com/jasonleeubc/Cantopop-corpus
python scripts/build_model.py Cantopop-corpus/Humdrum-files corpus/
python scripts/build_lexicon.py Cantopop-corpus/Humdrum-files corpus/
python scripts/sync_web_data.py          # refresh the browser bundle
python scripts/validate.py Cantopop-corpus/Humdrum-files corpus/

Pass any number of directories and they pool. Contributed songs in corpus/ fold in alongside the upstream corpus.

validate.py runs the checker back over the corpus it was built from, which only shows the model is self-consistent. For the harder question, use evaluate.py above. Real Cantopop should at least pass its own rules, and it does:

105 songs, 37772 adjacent syllable pairs
violations: 1477 (3.91%)
unusual:     287 (0.76%)
clean:     36008 (95.33%)

That 3.91% is the model's own error rate against professional lyricists. Drop in more songs, especially post-2020 and more colloquial ones, and it improves.

pip install -e ".[dev]"
pytest

Built on Jason Lee's Cantopop corpus (CC BY 4.0), and inspired by Canto-0243, which is the best offline 填詞 lookup workbench there is and the reason the melodic gap was obvious. No Canto-0243 code or data is used here. Full details in ATTRIBUTION.md.

MIT for the code. The two derived data files are CC BY 4.0, matching the corpus they come from. See DATA_LICENSE.md.