The idea
A normal parameter tuner changes how a function behaves. It cannot change which function runs. Paul Richeson's design adds that second axis, and takes its model from an analogue FM synthesizer.
A synthesizer has two kinds of control. A dial sweeps smoothly through a value. A rotary switch jumps between distinct circuits, such as the positions 0 to 7 that select a waveform. The sound is the product of the whole patch. Nearly any patch makes a sound, some are dull, and a few are greater than the sum of their parts.
Algorhythm treats a program the same way:
The important property is that the choice stays a whole number. Each switch is read through one rounded, clamped integer and dispatched by a single match, so a tank never runs a blend of two functions and never spends time on a version it did not choose.
Is there a name for this?
I found no single established name for the exact combination. The closest name for the core idea is Programming by Optimization. Here we call the method patch programming and the system Algorhythm.
Both names are working titles. A web search found no established use of "patch programming" as the name of a method, though "patch" usually means a software fix, so the term needs explaining when first used. "Algorhythm" is already taken in several places: Algorhythm Holdings is a Nasdaq-listed AI company, and algorhythms.ai is a generative music tool. It works as a project name here, but it would need a trademark check before any commercial use, which we have not done. The name fits the method because Yamaha's DX7 calls each of its 32 operator arrangements an algorithm.
| Existing field | What it shares | How this differs |
|---|---|---|
| Programming by Optimization [1] | The closest match. Treat a program as a large space of programs, avoid committing early to design choices, and let an optimizer decide. | Applied here to adversarial game agents, with a learned stage on the end. |
| Algorithm configuration, SMAC [3] | Tunes many categorical and numeric parameters together: switches and dials. | Usually tunes one algorithm over problem instances. It is a search method, and we use simpler reads. |
| Algorithm selection [2] | Chooses among whole algorithms. | We choose among parts, one decision at a time, and the parts combine. |
| Hyper-heuristics [4] | Selects among low-level heuristics, possibly at each decision point. | We fix a choice for a brain and test it by whole games. |
| Neural architecture search, DARTS [5] | Chooses among candidate operations. | DARTS relaxes the choice into a blend of operations during search. Ours never blends. |
| Residual learning [6], [7] | Learns a correction on top of a hand-written controller. | Our transform adds a confidence gate and several steps. |
| FM parameter search [10] | Genetic search of a synthesizer's patch space, a direct precedent for the analogy. | It matches a target sound. We have no target, only who wins. |
What is novel
Choosing among implementations by optimization is not new by itself: Programming by Optimization argued for it in 2012. A stronger claim is available, and it is the one this work can make. We found no system that combines all of the following:
- many hand-written, interchangeable implementations of each decision;
- whole-number switches and continuous dials, optimized together;
- selection by paired Monte Carlo games, with sequential tests and held-out reads;
- a learned last stage that reads the rules' own answer and overrides it only when confident.
The search behind this was brief. A proper literature review should come before this claim goes into a real publication.
A/B/C/D/E: one switch, worked through
Bots' pings used to mean one thing: come here. They now carry a kind (help, caution, info, warning or defend), and ping_style selects one of five behaviours:
| Position | What it calls | How it responds | When it might win | |
|---|---|---|---|---|
| A | as_built | Every call means help, if the pings switch is on | Converges | The baseline |
| B | help | Only when outnumbered | Comes within ping_help_reach | Calls are rare enough to mean something |
| C | warn | Warnings and cautions | Moves away from danger; drops a nearby rally | A spread-out team beats a massed one |
| D | defend | Only a base under attack | Comes and holds longer | Bases decide the game |
| E | all | Whatever it saw | Each kind in its own way | Shared information outweighs the distraction |
Each position may lose, and that is an acceptable result. The fleet has already shown that a team with no pings beats one where every ping is a summons. Position B tests whether communication is worthless or whether that one form of it was bad. Dials then refine the winner: if B wins at a reach of 60 tiles, the fit can test 40.
Some combinations will read badly. warn with a long flee distance may scatter a team, and all with three guards may leave nobody to answer. They still run, they read badly, and they do not get promoted.
How the variables compound
Choices multiply. Six switches with 8, 4, 5, 5, 8 and 4 positions make 25,600 combinations. With 18 on/off capabilities, that becomes about 109.8 programs before a dial is turned. This cannot be searched exhaustively.
Dials depend on routing. ping_warn_flee does nothing unless ping_style carries warnings, in the same way that a filter knob is silent if no signal passes through it.
Patches can exceed their parts, and this is the method's largest blind spot. Each version is read alone against today's defaults, so a pair that only helps together stays invisible unless one of them is promoted by itself. Testing switch pairs is the obvious next step.
The last stage: a multi-step transform
x = senses (143) ; waves (3) ; the rules' answer (3) y[0] = 0 y[k+1] = y[k] + tanh( A[k] [x ; y[k]] + b[k] ) if confidence >= patch_confidence: move the rules' point
Each step reads the steps before it, so a later step can correct an earlier one. At all zeros, nothing changes, so training can start exactly where the rules are. A learned model that replaces the rules loses by about 20 points a seat today. A transform keeps the rules' strength as its starting point, and we expect that to matter, though it has not yet been shown.
Intuition
The bio switch selects a wave shape (off, sine, triangle, square or saw). Three waves run in the ratio 23:28:33, each tank has its own phase, and they swing how bold it is. Biorhythm theory has been reviewed and found unsupported as a claim about people [9]. Nothing here depends on it: for a tank, this is a slow oscillator from the synthesizer's own kit, and it may help by stopping a whole team from making the same choice at once. The census may find that it does nothing.
The inspector
Results
The current brains are intact. With the new layers at their defaults, all nine scenario families read exactly as before: 319/400, 400/400, 326/400, 400/400, 998/1000, 993/1000, 1000/1000, 997/1000 and 812/1000. All tests pass.
The new switches change play. This is a smoke test on one Mac: 9 seeds, 900-second games, three maps.
| Trial, against the defaults | Every map (points a seat) | Naval only (3 seeds) |
|---|---|---|
ping_style = all | +2.16 ± 2.29 | +9.43 ± 3.19 |
bio = sine | −0.69 ± 1.02 | −3.03 ± 2.34 |
bio = sine, bio_amount = 0.8 | +0.24 ± 0.65 | +0.79 ± 1.81 |
These figures show only that games change. The fleet's own first read is 55 seeds and its bar is two held-out reads of 400. The naval figure is worth a proper test and is not yet a finding.
| Part | State |
|---|---|
| Typed pings, five styles, seven dials; people always answered | built, tested |
| Biorhythm switch and waves | built, tested |
| Multi-step transform, confidence gate, slots | built, tested |
| Inspector and version tracking | built |
| Training a transform | built, running; no result yet |
| Promoting a transform | not built: the game has no file for a slot |
| Hybrid crucible | built, tested, on three GPU islands |
| Fleet reads of the new choices | running; too few reads to report |
| Separate paths for each tank within a team | not built: choices are per team |
Rewritten functions, such as target_pick | designed only |
Does a confident override help? A first test
crucible is a separate GPU arena with simple tanks, far faster than treadcore. We gave it the same structure: a scripted tank decides, and a net reads the senses and that answer, then gives a correction and a confidence. Below zero confidence, the gate stays shut and the tank plays the rules. Each population was trained against varied opponents, and its best net was then read against the fixed scripted tank.
| Generation | Hybrid: win / draw / loss | Plain, from scratch | Hybrid overrides |
|---|---|---|---|
| 24 | 5 / 89 / 5% | 5 / 6 / 89% | 0.2% |
| 99 | 22 / 70 / 8% | 5 / 4 / 91% | 9.3% |
| 299 | 37 / 53 / 10% | 4 / 7 / 88% | 4.6% |
| 1,000–2,000 mean | 58 / 34 / 8% | not run | about 4% |
| 2,000–3,000 mean | 61 / 32 / 7% | not run | 4.4% |
The first row is the important check: with its gates almost always shut, the hybrid ties the rules, as designed. It then gains while overriding on only a few percent of steps. The rows up to 299 come from one 300-generation run of each kind, and the later rows from a separate 3,000-generation hybrid run with another seed.
Limits. This is one simple arena and one fixed opponent, which is also the very rule the hybrid builds on, so some of the gain may be exploiting that rule. Wins level off near 60% from about generation 1,000. Plain islands on the fleet did eventually reach about 95% against this opponent after over 100,000 generations, so the hybrid's advantage here is mostly how fast it starts, and we have not shown where it ends. crucible's senses and arena differ from treadcore's, so its nets cannot be moved across. Its transform is a single step, unlike treadcore's. This supports the design, and it does not show that a transform will help in treadcore.
How the project got here
A fixed hand-written Commander came first. Then about 120 weights tuned by paired censuses, followed by on/off capabilities. Whole-number choices arrived next (ladder, roles, model), along with a league of learned nets. A check on 30 October found that no switch could ever have been promoted, because of two bugs. Once they were fixed, pings off became the first switch promoted, at +1.99 a seat. Algorhythm is the next step: functions treated as a patch, with the learned stage placed after the rules instead of replacing them.
Acknowledgment
The design, the synthesizer analogy and the requirements are Paul Richeson's. Implementation and drafting were done by Claude.
References
- H. H. Hoos, "Programming by optimization," Commun. ACM, vol. 55, no. 2, pp. 70–80, 2012. Citation
- J. R. Rice, "The algorithm selection problem," Advances in Computers, vol. 15, 1976. Purdue report
- F. Hutter, H. H. Hoos and K. Leyton-Brown, "Sequential model-based optimization for general algorithm configuration," Proc. LION, 2011, pp. 507–523. Springer
- E. K. Burke et al., "Hyper-heuristics: a survey of the state of the art," 2013. Nottingham
- H. Liu, K. Simonyan and Y. Yang, "DARTS: Differentiable architecture search," ICLR, 2019. arXiv
- T. Silver et al., "Residual policy learning," 2018. arXiv
- T. Johannink et al., "Residual reinforcement learning for robot control," 2018. arXiv
- J. Chowning, "The synthesis of complex audio spectra by means of frequency modulation," J. Audio Eng. Soc., vol. 21, no. 7, pp. 526–534, 1973. CCRMA
- "Biorhythm (pseudoscience)," Wikipedia, a secondary source; the original studies were not read for this report. Wikipedia
- A. Horner, J. Beauchamp and L. Haken, "Machine tongues XVI: Genetic algorithms and their application to FM matching synthesis," Computer Music J., vol. 17, no. 4, 1993. Listing