lang-sme

Gold-set hygiene: two more case-context rows - espánjagiella expected capitalized Espánnjagiella, contradicting its own duplicate row (which expects espánnjagiella) — language-name compounds are common nouns. The country rows (espánja/Espánnja) stay: the country is a proper noun. - sátnejoðiheaddje expected Sátnejođiheaddji for a lowercase common noun (chairperson). Both surfaced by the rank-2-5 decomposition as structurally impossible for the speller (a lowercase input's correction is offered lowercase); same class as the seven rows fixed in 2ed5bccc32.

Fix the word-final rule block: it compiled to the empty relation The epsilon-left-side abbreviation-period rule (0 -> ".") shared a parallel-replace block with the ordinary word-final rules; an epsilon-LHS rule in such a block empties the entire block, so every rule in it — eamos<->mus and the word-final s<->š added since — has been dead since 16807fb996 (2026-04-01, 940 commits). Proven by compiling the block with and without the line. The period rule now sits in its own block (the pattern the -iid rule already uses), and the word-final o->u/e->i stay at their vowel-matrix price of 8 rather than reviving at 5: measured equal gain, a third of the churn. Real rebuild, both archives: +5 top-1 / +6 top-5 on the gold set (9161 / 10377 = 85.81% / 97.21%), +6 top-1 on the independent hand-tagged eval (63.98%), zero false accepts/negatives. Also restores abbreviation-period suggestions, which were impossible while dead.

Sweep the remaining tag penalties, errorth weight, and beam Same train/held-out discipline as the +Prop sweep; all confirmed by real builds and positive on both corpus halves plus the independent hand-tagged eval. - tags.reweight: every whole-class penalty was overpriced, as with +Prop: +Der/Dimin 20->5, +Coll 15->4, bare hyphen 40->5, +ABBR/+ACR/+Arab 20->5; +Foc/* is the exception and rises 13->26. Combined +30 top-1, effects measured individually additive. - ERRORTH_WEIGHT 40->32 (regenerated regex identical to errorth-regen --weight 32): the reachability collapse sits below 30, not below 40 as the old comment claimed; 32 over 30 on held-out parity and 24 MB smaller. +3/+6/+8. - config.json beam 28->80: top-1 is beam-invariant; 80 recovers 97% of the 264-word offered-at-all gap (+135 top-5) at p95 254 ms, within the parallel-execution latency budget. Also audited, no change: final_strings.default.txt is load-bearing (+25 top-1) but repricing fails held-out; words.default.txt all-help; STRING_REGEX_EDIT_DISTANCE=2 is zero-gain at +17% size. Production totals: top-1 9156 (85.77%), top-5 10371, offered 10544.

Production config: search-budget 1000000 Caps the suggestion search at 1M node pops per search, returning the best suggestions found so far (divvunspell 93bacd7; older releases ignore the unknown key harmlessly). On the current model the budget never fires across the entire gold set — zero words differ, lists and weights — so this is pure insurance against pathological inputs, and it is the measured knee for the prospective edit-distance-3 model (99.8% of accuracy kept, worst case 14.5s -> 3.5s).

Chunk-mining round 2: 21 rules + the missing ŧ-correction entries Second mining pass over the rotated failure residue (134 never-offered + 204 ranked 6+), same collateral discipline as round 1: every left side checked against the 9,076 attested forms, rewrites of correct forms checked against the acceptor. All 21 rules measure zero true collateral. Families: distorted Norwegian -sjon loans (átio/áksio/áves), loan haplology (dabbo/hansá/anial/perme), the -vdn- cluster written short (uvna/ádni), truncated place-name generics word-finally (jár->jávrri, joht->johka), dropped Slavic -sk in exonyms, and word-final abbreviation repairs (fn->lf, Ra->R). Separately, an editdist asymmetry the mining surfaced: t->ŧ and u->ŧ existed at 30, but no entry corrected a TYPED ŧ back, so a one-letter ŧ-for-t slip fell to the generic substitution at 59 (rank 8 for roavvarávddiŧ). Added ŧ->t and ŧ->u at 30. A/B on the gold set: +13 top-1 / +25 top-5 / +12 offered (9129 / 10362 / 10553; top-5 97.07%); the independent hand-tagged eval improves as well (+10 top-1 / +6 top-5). Zero false accepts. The mining also concluded: no chunk families with 2+ support remain in the deep tail — the residue is now one-off proper nouns (44), distance-4+ one-offs (68), ED3-reachable cases (51), and plain ranking losses (122). The chunk vein is mined out.

Chunk-mining round 2: 21 rules + the missing ŧ-correction entries Second mining pass over the rotated failure residue (134 never-offered + 204 ranked 6+), same collateral discipline as round 1: every left side checked against the 9,076 attested forms, rewrites of correct forms checked against the acceptor. All 21 rules measure zero true collateral. Families: distorted Norwegian -sjon loans (átio/áksio/áves), loan haplology (dabbo/hansá/anial/perme), the -vdn- cluster written short (uvna/ádni), truncated place-name generics word-finally (jár->jávrri, joht->johka), dropped Slavic -sk in exonyms, and word-final abbreviation repairs (fn->lf, Ra->R). Separately, an editdist asymmetry the mining surfaced: t->ŧ and u->ŧ existed at 30, but no entry corrected a TYPED ŧ back, so a one-letter ŧ-for-t slip fell to the generic substitution at 59 (rank 8 for roavvarávddiŧ). Added ŧ->t and ŧ->u at 30. A/B on the gold set: +13 top-1 / +25 top-5 / +12 offered (9129 / 10362 / 10553; top-5 97.07%); the independent hand-tagged eval improves as well (+10 top-1 / +6 top-5). Zero false accepts. The mining also concluded: no chunk families with 2+ support remain in the deep tail — the residue is now one-off proper nouns (44), distance-4+ one-offs (68), ED3-reachable cases (51), and plain ranking losses (122). The chunk vein is mined out.

Typosliste generert frå lexc-koden - 167717 testpar!

Add opt-in compact speller target: make se.bhfst Packs the acceptor and the UN-determinised error-model union into divvunspell's native BHFST box format. The xre expansion step exists only so NFA-walking runtimes can search quickly; divvunspell with on-the-fly mutator determinisation searches the plain union at the same speed, so the expansion (and its hundreds of MB) can be skipped. Measured against the regular se.zhfst on the full typos corpus at the production config: every accuracy figure identical (top1 8881, top5 10115, 0 without suggestions), speed at parity, artifact 66 MB vs 131 MB. Not part of the default build and not a replacement: C++ hfst-ospell consumers and divvunspell releases without subset search need the regular se.zhfst.

Mark ollusat as substandard; add ollosat The grammar checker's own gold data already treats ollusat as an error: five test files mark {ollusat}${olusat} with a correction (real-bidjui-FAIL.yaml:71, real-DerNomActSgGen-PrfPrc-FAIL.yaml:358, real-girjji-FAIL.yaml:34, msyn-congruence_subj-verb-FAIL.yaml:234, msyn-Sg3-FAIL.yaml:47), while ollosat appears as ordinary unmarked text (msyn-congruence_subj-verb-FAIL.yaml:190, real-PlNomPxSg2-PlNom-FAIL.yaml:189). The lexicon applies the same treatment to the sibling spelling variant a few lines away (ollugat+...+Err/Orth:ollog). The speller was the last component still accepting the form, which also made it uncorrectable. ollosat itself was missing entirely and is added as a normative entry. Revert this commit alone to restore the previous behaviour.

Refresh speller test report with the fixed divvunspell lookup divvunspell 176c691 + 040e58e corrected unknown/identity symbol handling in the suggestion search: out-of-alphabet characters can now take both identity and substitution paths, and unknown outputs enumerate real lexicon symbols instead of echoing the input. Summary metrics are unchanged (83.19% first position, 94.75% top five); the per-word deep ranks now reflect the corrected search.

Regenerate errorth rules and test reports on the merged tree make errorth-regen after the merge picks up the new error markups and drops rules for the removed kultuvra- long-form compounds. Full make check green: 12/12, suggestion quality 83.19% first position / 94.75% top five / 0 without suggestions / 0 false accepts at the current n-best 100 test configuration.

error markups

error markups

gierrit

skávuhit avádatcaggan oktoáhčči

[Template merge] Add support for generating typos data from Err/ entries in lexc files

tog bort areálaplána finns i incoming/prinsihppajearaldagat normeremii-giellagaldu-2026-05

Nyaste spellertestrapport, etter oppdateringar i leksikon og typoslista

Leaváš PLACE-DIM-org

Oppdaterte stavekontrolltestresultat etter justeringa av kultur-