OverviewPipelines
language-englishlanguage-englishlanguage-norwegian-bokmallanguage-norwegian-nynorsk
Login
Divvun BuildkiteOverviewPipelines
language-englishlanguage-englishlanguage-norwegian-bokmallanguage-norwegian-nynorskLogin
Pipelinesdivvun-runtimeBuild #360

build-number

FAILED
duration-label: 13m 46s
started-label-colon: time-days-ago
Single-flight the asset caches The caches were lookup-then-intern: a miss loaded and the intern kept whichever copy won. Correct at rest, but a host growing a worker pool builds its handles concurrently, and every one of them missed before any had interned - measured on an sme bundle at eight workers, that is 24 CG-3 grammar loads for 3 grammars and 7 tokenizer core loads for 1 core, a transient resident peak of ~2.1 GiB for a process that settles at a third of that, and every one of those loads is ~0.5 s of CPU thrown away. The convergence also hid itself: the losers adopt the winner and drop their copies, so the steady state looks right while the peak is what OOMs a container. cache_get_or_load now owns the whole miss path: each key holds its own slot lock, the first caller loads under it, racers wait for the result, and a failed load releases the key so the next caller retries. Waiting blocks the thread, so the hfst tokenizer and CG-3 grammar sites run the lookup inside the spawn_blocking task they already load on; the lookup transducer and speller sites already loaded synchronously in their constructors and now wait there instead. cache_peek keeps the cheap fast path for the two sites that mmap before loading. Loads for different keys still run in parallel - the lock is per key, not per cache. 99 lib tests pass in debug and release; the new racing_callers test pins the property itself (8 racers, one load).
main
49cc109d

jobs-heading

divvun-actions cidivvun-actions ci
ran-in▶
build-aarch64-pc-windows-msvcuv venv --python 3.12; uv pip install pyyaml setuptools; $en...
ran-in▶
lib-build-x86_64-unknown-linux-gnu./x build-lib --target x86_64-unknown-linux-gnu mkdir -p lib...
ran-in▶
aarch64-apple-darwin - Build./x build --target aarch64-apple-darwin mv target/aarch64-ap...
ran-in▶
lib-build-aarch64-apple-darwin./x build-lib --target aarch64-apple-darwin mkdir -p libdivv...
ran-in▶
build-x86_64-unknown-linux-gnu./x build --target x86_64-unknown-linux-gnu mv target/x86_64...
ran-in▶
build-x86_64-pc-windows-msvcuv venv --python 3.12; uv pip install pyyaml setuptools; $en...
ran-in▶
lib-build-x86_64-pc-windows-msvcuv venv --python 3.12; uv pip install pyyaml setuptools; $en...
ran-in▶
lib-build-aarch64-pc-windows-msvcuv venv --python 3.12; uv pip install pyyaml setuptools; $en...
ran-in▶
aarch64-apple-darwin - Signecho '--- Downloading unsigned binary' buildkite-agent artif...
not-started▶
Publish (Dev)divvun-actions run divvun-runtime-publish
not-started▶
Divvun Buildkite Overview

Resources

Buildkite DashboardGitHub - divvunGitHub - giellalt