ROBB: Recurrent PPO for Blockchain Block Formation

Balancing waiting time against block utilization

Bitcoin's fixed 1 MB block size is a blunt instrument: it keeps waiting time bounded but leaves block space unused whenever the mempool doesn't fill it exactly. ROBB reframes block formation as a sequential decision problem. At each timestep, an agent chooses to hold (wait for more transactions) or create a block sized dynamically to whatever is pending, trained to trade off the two costs directly instead of fixing one and accepting whatever the other turns out to be.

State, action, reward

• Observation: previous block size, average waiting time of the last block, pending-transaction count in the mempool, and mempool growth rate, enough for the agent to judge both current backlog and its trend.
• Action: a binary choice between hold and create; a created block is sized to the mempool at that instant rather than capped at a fixed size.
• Reward: a weighted combination of pending-transaction utilization and waiting-time cost for holding, vs. block-size utilization and waiting-time cost for creating. The weights (p_r=0.9, w_h=0.15, b_r=0.7, w_c=0.2) were tuned by grid search to minimize waiting time across the sweep in the paper's ablation.

Why recurrent PPO

Standard PPO trained on single-timestep observations converged to creating blocks too often. With no memory of recent transaction history, it couldn't learn to hold productively. Switching to a recurrent policy and value network (RPPO) let the agent condition on a trajectory of past mempool state rather than one snapshot, which is what made patient, well-utilized block formation learnable at all.

Results

Simulated against a custom OpenAI Gym blockchain environment, RPPO was compared to vanilla PPO, DQN, and A2C on the same reward, and to fixed- and random-block-size baselines modeled on real Bitcoin and Ethereum block sizes:

*Non-recurrent baselines hit 100% utilization by creating a block almost every step (thousands of tiny blocks) instead of accumulating transactions. RPPO reached the same utilization with two orders of magnitude fewer blocks, which is the actual point of holding.

Read alone, lowest-waiting-time-wins would favor the random baseline; read alongside utilization, it's the worst strategy in the comparison. RPPO is the only strategy that is simultaneously competitive on waiting time and maximal on utilization. That's the tradeoff the reward function was built to force.

Grew out of an MSc thesis on recurrent PPO for blockchain block formation, published in IEEE Access, Vol. 11 (2023).