𰻞

辶穴月幺長言馬幺長刂心

biáng, as in biángbiáng noodles · 58 strokes · eleven parts, every one of them common

The topic

Chinese character classification

→

classifier
one output per character

→

集
U+96C6

Input: one glyph, already cut out of the page.
Output: which character it is, among thousands of classes.
The reading step of OCR, for scripts with no alphabet.

Why it is hard

Same class, different looks:

集

Different classes, near-identical looks:

未末 · 己已巳 · 呆杏

Andfartoomanyclasses!

The problem

A classifier cannot output a class it never saw

3,755

classes in the standard training set
(enough for daily reading)

97,000+

Han characters in Unicode
(old books use the long tail)

0 %

a softmax's accuracy
on everything else

Zero-shot: recognise a character from its description, as you would read a new word by spelling it.

The shared idea

A character is a tree of radicals

~500 radicals and 12 layout operators (⿰ left-right, ⿱ top-bottom…) build every character.
Unicode writes the tree as a string, the IDS. Every paper reads the same file.

The benchmark

Train on m handwritten classes (500 to 2,755). Test on 1,000 classes never seen. Fewer training classes = harder.

Three ways to use the tree

Predict it, encode it, or align with it

1

Predict

Read the image, write out its tree, look it up in a dictionary.

≈ image captioning

2

Encode

Turn each tree into a fixed vector; train the image network to hit it.

≈ attribute-based zero-shot

3

Align

Learn an encoder for the tree too, and match images to trees.

≈ CLIP

Family 1 · Predict

Write the tree, then look it up

森

→

CNN

→

attention decoder

→

⿱ 木 ⿰ 木木

→

dictionary lookup

→

森

RAN: CNN encoder, RNN decoder whose
attention moves over the character, one radical per step — Each radical the decoder writes comes with an attention map: where it looked. RAN, Zhang et al., ICME 2018

RAN / DenseRAN 2018 → RSST 2024 → CDC-RAN 2025

Interpretable: one radical, one region.
Errors pile up along the sequence; one wrong token, wrong character.

The decoder learns to write combinations it has seen. Faced with a new character, it writes a familiar one.

Family 2 · Encode

Compile the tree into a fixed vector

森

→

CNN

→

vector

↔

nearest class code

←

code, designed by hand from the tree

Building a code: HDE (2020), simplified

$\varphi(y)=\sum_{\text{radical } r} \alpha^{d_r}\, e_r \;+\; \lambda \sum_{\text{operator } s} \alpha^{d_s}\, e_s$

one dimension per radical / operator; $d$ = depth in the tree; $\alpha=\lambda=\tfrac12$

森 = ⿱ 木 ⿰ 木木	depth	adds
⿱	0	$\lambda\alpha^0 = 0.5$ to $e_{⿱}$
木 (top)	1	$\alpha^1 = 0.5$ to $e_{木}$
⿰	1	$\lambda\alpha^1 = 0.25$ to $e_{⿰}$
木, 木	2	$2\alpha^2 = 0.5$ to $e_{木}$

⇒ $\varphi(森)$: 木 = 1.0, ⿱ = 0.5, ⿰ = 0.25, every other dim 0.

HDE 2020 → CUE 2023 → HierCode 2025 → JRED 2025

$\hat y = \arg\max_{y\,\in\,\text{all classes}} \cos\big(f(x),\,\varphi(y)\big)$

train the image network $f$ on seen classes; unseen ones only need their $\varphi$.

The code is frozen before training. 呆 (口 over 木) and 杏 (木 over 口) get the same counts: only a tiny position term, in the 4th decimal, tells them apart.

Family 3 · Align

Learn both sides: CLIP, with the tree as caption

森

→

image encoder

→

similarity
contrastive loss

←

IDS encoder

←

⿱ 木 ⿰ 木木

at test time: nearest of
all candidate trees

What moved: the IDS encoder. Same images, same strings, same benchmark (m = 2,755)

Structure written nowhere: learned from token order.

FT-CLIP 2025 · 79.0 each node attends to its children, and knows where they sit.

GRSTR 2026 · 80.6 a GNN pushes layout down, and 1st / 2nd child differ: 呆 ≠ 杏.

Next step, outside the encoder: match image regions to each radical, not one vector to one vector. Hi-GITA 2025: 85.4.

And without the tree?

Match the image to a printed font

→

image encoder

→

similarity

←

same encoder

←

集

one font rendering
per candidate class

OpenCCD CVPR 2022 → PCSS 2024

Same CLIP idea, but the "caption" is a picture of the character in a font, not its tree.
Works where no tree exists: Japanese kana never seen in training, rare variants.

Needs a font that draws every class. Fewer than 5 fonts cover all 97k characters, and a modern font is not the shape of a 17th-century woodblock.

On the next chart it is drawn dashed: each unseen class comes with a printed glyph the tree methods never see, so it is not a like-for-like comparison.

Ten methods, one benchmark

Retrieval beats prediction, by a lot

HWDB → ICDAR 2013, Top-1. GRSTR, GL-HPN, Hi-GITA, OpenCCD tables. Dashed: given a printed glyph per unseen class.

+25

points. The first CLIP-style model beat six years of predictors.

5,400×

faster. Decoding: 1,666 ms per image. Retrieval: 0.31 ms.

10×

How finely you split into radicals: +27 pts. A better backbone: +2.7.

1:30. Click per family: predictors at the bottom, codes in the middle, alignment on top. Hollow dot = 500 training classes, the hard case; filled = 2,755. Click: +25 points between best predictor (RSST 47.4) and first aligner (CCR-CLIP 73.0), a paradigm gap, not tuning. Click: and it's 5,400x faster (DenseRAN 1,666 ms vs FT-CLIP 0.31 ms). Click: the tree's granularity mattered ten times more than the backbone (GRSTR on AHCDB: coarse→fine +27.4, ResNet-c→b +2.7). Click 4 (before the punchlines): font matching, OpenCCD 90.9 at m=500, but it is handed a printed image of every unseen class, which is extra information; MegaHan's common protocol (next) puts it at 76, below CLIP. If asked: CoLa 2025 learns its own components and reaches 68.6 at m=500; dual-head methods trade closed-set accuracy.

Where my work fits

A learned embedding, used to fix a reader's mistakes

I train a contrastive glyph encoder, family 3's recipe on the image side.
Each glyph's nearest neighbours vote on its label: a graph over the whole book.

What the graph is for

Detect errors: a glyph whose reading disagrees with its look-alikes.
Catch hallucinations: the reader writes a plausible character that is not on the page.
Expose model conventions: one reader writes the Taiwan form, another the Hong Kong form, of the same glyph.

Run on a real Qing-dynasty volume: 770 pages, no ground truth.

24 glyphs corrected on
volume 611: original reading, correction, and a second reader's verdict — Corrections kept on volume 611. Blue: a second, independent reader agrees.

謝謝

Thank you for your attention

Questions?