Abstract
The fleet's cycle now closes without a person:
- a search's result is adopted with its held-out reads;
- a candidate that passes both, with families that hold and current defaults, promotes itself;
- a promotion whose tests and families pass is pushed to main;
- every machine's build is then checked for the promoted values.
Candidates and other news are progress, not flags. The first full-size spsa (r0014) and tune (r0015) runs cleared no bar on their own, and both are adopted for held-out reads. The first auto promotion, r0019 (grove/w0034), is running.
I. Introduction
A. Intent
Make the queue a cycle that runs itself on the fleet's CPUs and GPUs, in auto mode by default, with proof that a promotion's values reach every machine.
B. The prompts
why do I need to review the canidate flags. I'm going to always accept them. Just make them visible, so I can see progress.
read the results of r0014 and r0015. tell me about them.
queue the held-out reads for both. can we have this system auto incorporte the reults. accept them. incorporate them, sync them. do all the steps that we seem to overseen. We want this a a cyclical process where a fleet of CPUs and GPUs can destribute the workload. I'm actaully a little questioning, what happends when I shut down Orrary. the system seem to continue to compute. I'm ok with this, I'm just looking into what happends when we resync. we collect the results. we report back. we update the values on all other fleets wit the new values, The Orrery mentions these. also we get Flags. What are they about. And Cadidates. Should self promote, unless we are not in AUTO mode. We should be in auto mode; since the queue keeps queueued.
i want to see some sort of confirmation that seach confirmation candidate shows that the values propigated to all computers in lab/infleet.
II. Decision Log
| # | Cause | Decision | Not taken, and why |
|---|---|---|---|
| D1 | "Just make them visible" | Candidates, runs finished, promotions: progress records, not flags | Acknowledging each by hand |
| D2 | "queue the held-out reads for both" | A run's result is adopted as a trial, its census and both held-out reads queued together | Held-out reads without a census (evidence needs the census too) |
| D3 | A candidate measured on old defaults is not a measurement of the new | Only current candidates (every census's COMMIT on today's defaults) promote themselves | Promoting on top of changed defaults |
| D4 | "Should self promote ... We should be in auto mode" | Auto on by default; the strongest current candidate whose families hold; one at a time, each once | Promoting several at once |
| D5 | A test every candidate trips (as r0009's did) would waste the Mac hours a candidate | Two failures in a row pause auto mode with a flag | Trying every candidate |
| D6 | "Prepare, you approve" is superseded by auto | A prepared promotion is pushed by the planner; off, Approve as before | Keeping Approve in auto mode |
| D7 | "confirmation ... propagated to all computers" | Every machine's build brings back its defaults; each promotion is checked per machine against its values | Trusting that a pushed commit is built |
| D8 | "what happends when I shut down Orrary" | Documented: the keeper is its own process; the window reads and sends | — |
III. Methods
- orrery
tools/plans/treadcore.py:- new:
auto,new_run,adopt,adopt_runs,adopt_jobs,
census_commit,auto_promote,propagation;_approve, the quiet half ofapprove, also used by the planner;- in
evidence:commitsandcurrent; prepare_jobsfetches every machine'sdefaults.toml;rsavewrites progress for news;planorders the pass: raise, adopt, promote, runs (approve),
builds, propagation, adopted jobs, families, search;
adoptis a command.
- new:
- orrery
tools/fleet-queue.py:auto on|off;statuscarriesauto. - orrery
src/training.rs:RunRowcarries the candidate, who asked, and each machine's state.Queue.auto. The Candidates page: the mode and its switch, promotions with their machines, and candidates only. - The checks:
treadcore.py Q evidencelists each candidate as current or older
defaults;
fleet-queue.py Q runsshows r0019 by auto;- the build peaked at 577 MB under
mem-guard.sh.
IV. Results
| r0014 (spsa, dodge) | dodge_clear 2.5 → 1.25 (its bound), dodge_horizon 2.0 → 1.65; largest step −3.84 ± 2.51 (1.5 errors) |
| r0015 (tune, 15 weights) | best +2.52 ± 2.47 (1.0 error); w_near up, w_in_range up, w_focus down |
| Candidates | 19 of grove's current, families hold; the converge run's on older defaults |
| First auto promotion | r0019, grove/w0034: held out +2.67 ± 0.27, +3.17 ± 0.20; seven defaults; tests and families running |
| Builds of 9c9abc1 | queued on all six machines, each to bring back its defaults |
Errors:
- r0014 and r0015 are single runs with errors as large as their effects.
- r0019's outcome, and the first propagation check, are not yet known.
V. Discussion
The fleet already searched, held out and read families. What it lacked was the last two steps and the proof between them. A promotion is now measured against today's defaults, made by the fleet, and confirmed on each machine by the values that machine compiled, not by the commit it was told to build.
Auto mode trusts the bar that was already Paul's: two held-out reads past two errors, every family over its floor, the brain's tests. Paul can turn it off with one button.