LilyNota

A symbolic-music generation model that writes scores in Lilylet — a compact, LilyPond-flavored text notation. Given a short metadata prompt (composer / genre / instrument / key / time signature …), the model autoregressively composes a multi-measure, multi-staff piece.

It powers the LilyScript Space, where generated Lilylet is rendered to a sheet-music score (Verovio) and played back as MIDI.

Note: the weights here are an early snapshot and will be refreshed as training continues.

Model

LilyNota — a hierarchical, two-level NotaGen-style decoder (Llama backbone):

• Patch-level decoder — 10 layers, hidden 1024, 16 heads. Consumes the score as a stream of fixed-size patches (16 tokens each) and produces a hidden state per patch.
• Token-level decoder — 4 layers. Expands each patch state into its concrete Lilylet tokens.

	value
params	~196 M
base type	llama (bf16 trained)
patch size	16 tokens
vocab	256
context	1024 patches

Files

model_*.chkpt                 torch training checkpoint (full precision)
.state.yaml                   training / architecture config
tokenizer.json                tokenizer
onnx/
  patch_kv_int8.onnx          patch decoder, int8, with KV-cache (incremental)
  token_kv_int8.onnx          token decoder, int8, with KV-cache (incremental)
  wte.npy                     token-embedding table [vocab, hidden]
  geometry.json               patch size, special ids, per-level KV geometry

The onnx/ bundle is torch-free: a generator needs only onnxruntime + numpy to run it (the embedding lookup and sampling live outside the graph). int8 dynamic quantization plus a two-level KV cache make it a fast CPU inference path.

Usage

The reference runtime is StreamingLilyletGenerator in the LilyScript Space (lilyscript/generator.py). Sketch:

from lilyscript.generator import StreamingLilyletGenerator

gen = StreamingLilyletGenerator(model_dir='onnx', asset_dir='onnx')
prompt = '[composer "Beethoven, Ludwig van"]\n[genre "Classical"]\n[instrument "Keyboard"]'
for raw, pretty, done in gen.generate_stream(prompt_text=prompt, measures=8, temperature=1.0, seed=42):
    pass        # `pretty` is measure-segmented Lilylet; streams one patch at a time
print(pretty)

Output is Lilylet text, e.g.:

[composer "Beethoven, Ludwig van"]
[genre "Classical"]
\key g \major \time 3/4 \clef "treble" \tempo 4=54 ^\markup "Andante con moto" r2. \\
...

License

MIT.