The ASIC testudo is fundamentally an infrastructure play, made possible through down-clocking.
Using software to optimize ASIC performance can appear a daunting quest. Surely the undertaking is reserved for the hero-nerds of old; those who dared to brave blazing heat, smoking power supplies and an untimely end of warranty. Mining alchemists, who sought in software a digital Philosopher's Stone that could manifest precious digital gold. Their methods may have succeeded when GPUs began mining Bitcoin in 2010, but how could they possibly be implemented now, in an industrial datacenter with thousands of ASICs? Bitcoin mining today is a business of tremendous scale, not a test bench. Yet, we once again find ourselves drawn to the old ways.
At time of writing, the 2024 Bitcoin Halving looms near. The block subsidy will be slashed in half and miners will see a substantial decrease in mining revenue. As margins shrink, miners will feel pressure to reassess their operations. Creative methods will be sought to optimize infrastructure and maintain margins. For some, there will be opportunities to expand in the fallout. The question is, how to accomplish these objectives in the face of constrained resources? The digital Philosopher’s Stone has once again found its allure.
A general leads an army through standardized commands and equipment, but adapting to the enemy and terrain secures victory. Likewise, a profitable miner ought to make use of standardization while also engaging in opportunistic tactics. The trick to surviving and even flourishing after the Halving is to blend the strength of standardization with the flexibility of a programmer in the lab. To better understand this, let us turn to military history for inspiration. In doing so, the reader will also find a framework for understanding the mining strategies and tactics that follow.
Prior to Roman domination, the phalanx was arguably the most powerful military formation in existence. It was the backbone of the Macedonian army, which conquered much of the known world when led by Alexander the Great. The Romans themselves used a phalanx formation in their early conquests before inventing its successor, the manipular legion.
A phalanx consisted of hoplite soldiers equipped with long spears, typically 13 to 23 feet long, and round shields. They were assembled into tight rows that created a wall of shields and spears toward the enemy. This formation enabled multiple rows of men to participate in the front-line conflict. Hoplites in the back could thrust their spears toward the enemy while the front row held the line, shields at the ready. This formation was extremely powerful when opponents were forced to engage head-on. Phalanx contemporary and Greek historian Polybius wrote of them, “...when the phalanx has its characteristic virtue and strength nothing can sustain its frontal attack or withstand the charge…”
However, the phalanx was not without weakness. The sides and rear of the formation were vulnerable since the formation could not adapt quickly. This meant that faster moving light infantry or cavalry could disrupt the enemy phalanx formation, giving their own phalanx the upper hand in their frontal assault. Battles of this time were usually decided by flanking maneuvers. The phalanx was only strong if its formation was maintained.
Then came the Roman manipular legion. It was a hybrid formation that blended the strengths of a phalanx with the versatility and tactics of fast-moving troops. It consisted of groupings of men called “maniples.” Though called the “triple line” formation, there were four lines of men (compared to the phalanx single line) and there were spaces between each maniple. This formation allowed pressed Roman warriors at the front to retreat for support. The retreat would also entice hoplites at the front to give chase, disrupting the phalanx formation.
The first line was composed of velites (“fast men”). They wore light armor and were equipped with javelins that could be used to antagonize the enemy from a distance. If the velites became pressed, they could quickly retreat into spaces between the second line, the hastati (“spearmen”). If the enemy withdrew to reform, the velites and even hastati (who also had javelins) would rush forward in pursuit. Lendon writes of them, “The velites seem to have fought individually as an irregular swarm, without officers or standards of their own…” (Soldiers & Ghosts p. 179).
If the hastati became pressed, they could similarly fall back between the principes (“first men”). If the enemy hoplites at the front chased after the hastati, they would find themselves separated from their phalanx formation and surrounded on three sides. If this was not enough for the Romans to win the day, the velites, hestati and principes could fall on the triarii (“third men”).
The triarii were the legion’s most experienced soldiers. They were equipped similarly to the hastati and principes, except that their spears were longer, similar to those of the phalanx. If the enemy did manage to press through the hastati and principes (all the while being antagonized by the velites) then they would suddenly face Rome’s best troops, well rested and equipped to hold a phalanx formation. Lendon writes, “When the triarii finally came into action the Roman legion could aptly be described as a phalanx.” (Soldiers & Ghosts p. 181)
The manipular legion allowed Rome to rise from a fledgling city-state to become the dominant world power of its day. This trajectory was made clear at the battle of Pydna, where Romans engaged the Macedonian phalanx head-on in the flat lands. Lendon writes,
“Despite Roman bravery, ground and equipment favored the Macedonians, and they steadily pushed the Romans back. Finally, the Macedonian’s very success was their undoing. Their advance met more or less resistance at different points, and so some parts of the phalanx pressed forward and some held back. Their progress also took them onto higher and less regular ground which interrupted the exact dressing of their ranks. … As the phalanx came apart the Roman advantage turned into a slaughter. More than twenty thousand Macedonians were slain and only a handful of Romans.” (Soldiers & Ghosts p. 208 & 209)
On flat ground and in true form the phalanx could push back the Roman legions for a time. Once faced with uncertain terrain and disrupted form, the phalanx fell. Let the reader lift this lesson from the pages of history: do not optimize for ideal conditions, learn to thrive in chaos. History has shown there is strength in highly ordered systems, but dependence on order alone during changing circumstances creates great weakness. Just as the Macedonians pushed forward onto uneven ground at the battle of Pydna, some miners are running stock firmware into the Halving.
In times of plenty, an ASIC army is raised by purchasing, building, and deploying infrastructure. Bear markets are a war of attrition. Hash rate clashes against hash rate, until the “enemy” margins are vanquished and they are forced to capitulate.
Many ASIC armies today operate like the phalanx did. Their “commanders” consider each machine to have one power draw and hash rate. They imagine revenue to be a linear function, changed only by the number of machines online and the tides of hash price. They suppose the “battle” can only be won by fighting for high uptime and low power costs. Of course all farms can benefit from high uptime and low expenses. However, there exist tactical advantages in adopting a manipular strategy. Let’s take a closer look.
As Cathedra has explored the art of the possible, it was discovered that an air-cooled S19J Pro can reach into the 21 J/T (joules per terahash) range without any hardware modifications. Since peak values in the lab rarely manifest in practice, we will leave a buffer and assume an efficiency of 23 J/T under ideal conditions (54ºF or less). Knowing this creates an array of viable configurations for each miner. This flexibility allows us to operate our data centers as a manipular legion.
We first define four “lines” of efficiency that act in a similar way to the four lines of the manipular legion. To the velites, hestati, principes and triarii we assign the efficiency values of 34, 30, 26 and 23 J/T, respectively. As hash price declines, miners can “retreat” to a more profitable efficiency, thereby improving fleet-wide profitability. This flexibility also offers an advantage when hash price is increasing. As the Roman velites could antagonize a retreating phalanx, so too can we chase higher hash prices by up-clocking. Historically, network difficulty has trailed large run-ups in Bitcoin price. The manipular ASIC legion can quickly capitalize on this imbalance.
As with any strategy, there are secondary effects and tradeoffs to be considered. At industrial scale, Bitcoin miners need to be aware of demand charges. Many utilities issue demand charges based on the 15-minute interval of greatest power use during a billing period. If power consumption varies from this peak, the difference becomes the basis for a demand charge in addition to charges for generation and delivery. Since down-clocking ASICs causes them to use less power, an ASIC operator who frequently changes ASIC settings during a billing period may be surprised to find that their effective cost per kilowatt-hour is higher than expected.
Thus, down-clocking miners connected to the grid can create a counter force in the retreat to better efficiency. Operators in this context should implement down-clocks at the end of a billing period to avoid peak demand charges. Conversely, up-clocks should be enacted at the beginning of a billing period. Otherwise, the preceding month will effectively be billed at a higher rate.
In other scenarios, an operator may have inflexible power purchase agreements or may have leased infrastructure to generate their own power. In this context, simply down-clocking will not be beneficial. Costs will remain unchanged while revenue takes a hit. This scenario calls for a different strategy.
Roman warriors were quite familiar with being pressed in battle, but some situations called for an extreme defensive posture. To assail highly fortified positions under an onslaught of arrows, javelins and rocks, or to simply retreat in desperate situations, maniples needed to minimize losses. For this purpose, the Romans developed the “testudo” formation, which in Latin means “tortoise.” It consisted of legionaries forming close ranks with their fellow soldiers and interlocking their rectangular shields (each called a “scutum”) at regular intervals overhead and on the sides as needed.
When done properly, the formation resembled its namesake and was quite effective at shielding a maniple from projectiles. Whether in attack or retreat, the formation allowed a Roman legion to develop into an effective fighting position. However, the tight spacing required to make the formation a success impeded mobility.
For a Bitcoin miner, heavy resistance on the “battlefield” means high electricity prices and low hash price. In some situations, as margins are squeezed, down-clocking alone may not be sufficient to remain profitable. In other situations, down-clocking may not even be possible. It is here that we may draw inspiration from the testudo.
Consider the following scenario. Suppose a mining company is evaluating two of its data centers, each with 1,000 S19J Pro miners. The first site is subject to a take-or-pay power purchase agreement (PPA) with a cost of $50/MWh (megawatt hour). The second site is a hosting facility and charges $75/MWh. Both sites are running stock firmware, where each ASIC uses 3,000 watts and produces 100 TH/s. With today’s hash price around $60/PH (petahash), these sites are running at a profit.
Speculating a bit, suppose several months from now the Halving has occurred and hash price plummets to $40/PH. The losses incurred at the hosting facility are now overwhelming profits from the other site by three to one. Naturally, this scenario would inspire one to unceremoniously pull the plug on both hosting agreement and miners in kind. What has not yet been accounted for is the hosting deposit of, let’s say, $330,000. Terminating the hosting contract early would forgo the deposit. Since this represents a greater loss, machines will be left to bleed cash until the hosting contract expires or renewal is met with better conditions.
Back in the present, what can be done to mitigate losses should this bleak scenario come to bear? If these sites were each being operated as an ASIC legion, the triarii would be called upon. Each machine would be down-clocked from 3,000 watts to 1,500 and from 100 TH/s to 65. While this would not be enough to spare the hosting facility from producing a loss at $40 hash price, profits from ASICs running under the PPA are now more than enough for the company to remain cash-flow positive.
In discussing this approach with the hosting facility, suppose the request to alter machine clocks is rejected. The host is concerned with network security and is not incentivized to see reduced revenue due to selling less power. Besides, they have minimum offtake requirements. Looking back, recall that the other site also has a take-or-pay agreement in place. Down-clocking alone will not achieve the intended effect, since costs will not be reduced. The ASIC legion appears to have fallen, forced to operate as a phalanx on uncertain ground. In spite of this bleak outlook, there is a strong path forward.
The ASIC testudo is fundamentally an infrastructure play, made possible through down-clocking. It can be defined as increasing the number of operating ASICs per megawatt for a given machine type. Just as the Roman legions could succeed in retreat by pressing tightly together and forming the testudo, ASICs can take a similar form.
Recall that when down-clocking to 23 J/T, power use dropped by half. Now suppose the hosting agreement is up for renewal prior to the Halving. The agreement could be allowed to expire and machines consolidated under the $50/MWh PPA. Since the total power of 2,000 ASICs running at 1,500 watts is the same as 1,000 running at 3,000 watts, the change will not incur any losses from the take-or-pay nature of the PPA. This allows ASICs from both sites to operate with greater efficiency and consolidates them on a lower average power cost. Even more stunning, this new configuration can produce positive cash flow at $30 hash price!
Not so fast. How exactly are all those machines simply going to find available rack space and cables to plug in? Some sites may have physical gaps between miners that can be filled and splitter cables can be used to provide power. If this is not viable to accommodate all the incoming machines, then additional infrastructure must be supplied. Fortunately, collapsing hash price is the perfect time to purchase fire-sale infrastructure. In addition, there will be ready capital to act on this, since the $330,000 hosting deposit will become available upon expiration of the hosting agreement.
The testudo, while effective in retreat, is intended to develop into a position of strength. The Romans typically used the testudo formation to move troops and siege equipment like the battle-ram closer to a fortified enemy before attacking. Similarly, as mining conditions improve, the testudo could be an excellent strategy for tactical ASIC pre-orders. Machines could arrive before a new site is energized, placed in mobile containers (possibly Rovers), down-clocked, and used to increase profitability on existing sites rather than waiting around in storage. Furthermore, if the new site was designed in anticipation of this strategy, the containers could be transferred to the new site, up-clocked and begin mining quickly; without the delay of unboxing and racking the ASICs. Containerized solutions unlock the full potential of the digital Philosopher's Stone.
There are many strategies and tactics that can be employed with CathedraOS. A complete list of them would be outside the scope of this writing. We look forward to inventing new ones with you and would certainly entertain technical follow-up pieces if the reader has found value in these pages. Happy mining, friends!
To download Cathedra Bitcoin’s after-market firmware, CathedraOS, please visit our website at: cathedra.com/firmware
Isaac Fithian is Cathedra Bitcoin’s Chief Field Operations and Manufacturing Officer.
This post was originally published on Cathedra's Blog
