Methodology · models, dataset, toolkit

From the conceptual conditions to the apparatus

Chapter 1 of the thesis derives the conceptual conditions under which legal meaning can be observed at scale: meaning as use, language as space, geometry as legal instrument (§1.2–§1.4). Chapter 2 turns those conditions into an apparatus. This page summarises the apparatus — the language models, the lexicon, and the statistical tools — that the two experiments in §3.1 and §3.2 rely on.

Ten language models

Ten language models span three families and two language traditions. The selection is deliberately heterogeneous: it includes a legal fine-tune (FreeLaw-EN) to test whether legal specialisation helps; a small multilingual model (Qwen3-0.6B) to test the limits of generalisation; and a single bilingual model (BGE-M3) deployed twice — once on the English side, once on the Chinese — to provide a same-model bilingual control used in §3.1.3 and §3.2.4.

The Western-trained vs Chinese-trained contrast is the language-tradition contrast that organises the empirical chapters. The thesis is not about that contrast per se: it is methodological. The choice of Hong Kong ordinances co-drafted in English and Chinese (§2.2) makes the contrast naturally available, and the design uses it as the test for whether embedded geometry is sensitive to tradition.

Dataset · 364 + 9 045 + 100 terms

The dataset has three tiers. Each term is a parallel English/Chinese pair drawn from the Hong Kong DOJ bilingual legal glossary, and each has been re-attested against the post-1989 ordinances co-drafted under the Bilingual Laws Project. The filter retains a term only if it appears at least four times in real ordinance contexts; the threshold is justified empirically in §3 of the thesis. The complete 364-term lexicon and the 100 control words are browsable verbatim under Inside the inputs, each term expandable to two real Hong Kong ordinance passages.

Core · 364 terms

The curated, vetted core of the legal lexicon. Distributed across 7 domains (administrative, civil, constitutional, criminal, international, labor & social, procedure) with band-balanced support (41 – 60 attestations per domain). Each term is encoded twice by each model: bare (the lemma in isolation) and attested (the mean embedding of its real ordinance contexts).

Background · 9 045 terms

The remainder of the bilingual glossary that satisfies the four-context threshold but was not hand-vetted. Used in the robustness analyses of §3 to test the principal results under pool perturbation, and as the input for a k-nearest-neighbour assignment that labels each background term with the domain voted by its seven nearest core neighbours.

Control · 100 terms

Everyday-language vocabulary with no legal content: pronouns, deixis, common nouns (I, you, he, this, here, water, day, year, etc.). Encoded bare only — controls have no Hong Kong ordinance attestation by design. The control pool grounds the §3.1.1 legal-vs-control test and underpins the control-pool subtraction in §3.1.3: the bare Δρ on the control pool is statistically indistinguishable from the bare Δρ on the core, isolating the contribution that attestation in legal context adds.

Statistical toolkit

Six tools, each with a specific role and a specific limit. They are introduced here in compact form; their full application — including the inferential discipline that separates measure, interpretation and limit — is the subject of §2.4 of the thesis.

Representational Similarity Analysis (RSA)

For each language model, build a Representational Dissimilarity Matrix (RDM): a 364 × 364 symmetric matrix where cell (i, j) is the cosine distance between term i and term j as encoded by that model. Comparing two language models then means computing the Spearman ρ between the upper triangles of their two RDMs. ρ = 1 if they rank pairs identically; ρ = 0 if they are uncorrelated. Reference: Kriegeskorte, Mur & Bandettini (2008).

Mantel test (B = 10 000)

The null hypothesis for an RSA ρ is that the two RDMs are unrelated. The Mantel test draws a permutation distribution by shuffling one RDM's rows (and matching columns) and recomputing ρ. In §3.1.3 the test applied to all 17 pre-registered model pairs returns p ≤ 1 × 10⁻⁴ on every pair.

Holm correction (K = 17)

The 17 pre-registered model pairs in §3.1.3 mean 17 simultaneous tests. The Holm–Bonferroni correction guards against multiple-testing inflation; the corrected p_max across all 17 pairs is ≤ 1.7 × 10⁻³.

Block bootstrap on terms (B = 10 000)

The 66 066 upper-triangle entries of an RDM are not independent: each term contributes to 363 of them. The block bootstrap resamples terms (not pairs) and re-subsets the RDM, then recomputes ρ — yielding term-level confidence intervals that honour the dependence structure. Reference: Nili et al. (2014).

Kozlowski axis construction

For each value axis (e.g. individual ↔ collective) curate up to twenty antonymic seed pairs and compute the centroid difference vector: positive minus negative, averaged over pairs and normalised to unit length. A term's score on the axis is its cosine with the axis vector. The six axes of §3.2 are built this way. Reference: Kozlowski, Taddy & Evans (2019).

Mann-Whitney U with rank-biserial r

For two distance distributions (e.g. legal-legal vs legal-control) compute the non-parametric Mann-Whitney U. The associated effect size is the rank-biserial r = 1 − 2U / (n_x n_y); r > 0 means the first distribution sits below the second. Used in §3.1.1.