jakiro
Source: Bitcoin Magazine; Compilation: Wuzhu, Bitcoin Chain Vision
Everyone who has used Bitcoin has used a mempool.So, what is a memory pool?
From a technical point of view, there is no so-called “unique” memory pool.Each Bitcoin full node operates its own memory pool, which is a cache of valid Bitcoin transactions that have been broadcast to the network but have not been confirmed in the block.Nodes exchange messages with each other to view transactions that they own or do not own and exchange transactions that they do not have.
Each memory pool is essentially a separate “island”, with its own set of unconfirmed transactions and sometimes even its own configuration variables and settings.The size of the memory pool can be configured, and the default is 300 MB.In addition, there is a minimum rate that is dynamically adjusted and can be configured as a value.This is used to decide which transactions will be kicked out of the memory pool when the memory pool is full and more transactions are constantly pouring in.There are also some other configurable options, such as the datacarrier and datacarriersize options, which affect transactions containing the output of OP_RETURN.
Different nodes have different reasons for running memory pools, so the requirements are different, but ultimately, the satisfaction of these requirements is achieved by synchronously running their respective memory pools and interacting with each other.
Think of each memory pool as a real pool, which is connected to each other through the underlying channel.The larger the memory pool, the deeper the underlying pool.Miners, exchanges, block browsers, these will all become the deepest pool.They all have their own motivations to understand every unconfirmed transaction waiting to be packaged into blocks.Miners, to make sure they have the most profitable deals in the next block.Exchanges, to make sure they understand all pending transactions.Block Browser, because their entire service is displaying datasets about blockchain and memory pools as completely as possible.Your normal nodes really only need to be deep enough to include the highest rate part of the “memory pool”.
Now think of each transaction as a drop of liquid, the higher the fee rate, the greater the density of the liquid.These liquids flow in channels between mining pools, and upon reaching each mining pool, a drop of liquid received is copied and sent through the channel to any other mining pool that has not yet received the liquid.As the pool fills up, when the liquid overflows, the lower density liquid (lower rates) will first overflow the edge of the pool.
Eventually, some lucky miners will scoop a certain amount of liquid from the bottom of the pool and pour it into the latest glass trough, forming a long winding glass trough filled with liquid and stay there forever (blockchain).It’s just a way to think about the system intuitively and cover most of its dynamics.
This arrangement of mining pool interconnections provides different uses for different users.
Trader
Memory pools have two uses when users conduct transactions.First, and most importantly, is sending their transactions to miners.If the transaction does not enter the miner’s memory pool, it is impossible to be packaged into blocks.Memory pools are linked and shared with each other, ensuring that once the transaction is placed in a memory pool, it will eventually enter the memory pool of all miners.It is invaluable to have a strong and decentralized network that ensures that transactions can eventually be sent from users to all miners without being affected by changes in network connections and fragmentation.
The second purpose is fee estimation, which is especially important for Layer 2 users, as they need to ensure that response transactions to invalid states can be confirmed in a timely manner.The fees can be estimated to a certain extent by just looking at the rates of transactions in these blocks, but this does not provide any information about the state of the memory pool after the latest block.It cannot explain the sudden spike, the influx of opportunists into the memory pool, or the next wave of unfinished trading surges.Because the memory pool cannot be viewed, the fee estimate cannot ensure that it takes into account the current status of the pending transaction.
Receiver
When you receive Bitcoin, your node verifies the transaction and the entire block containing the transaction.The transaction paid to you is broadcasted and eventually enters the miner’s memory pool, and the miner finds a block that is broadcast to the network, and your node will download and verify the block.
But this is not the way it actually works (unless you disable the memory pool of the node and run in block-only mode).Your node is verified when the memory pool receives each transaction for the first time and caches it as a valid Bitcoin transaction.When miners find a block, they actually forward only the block header and a small piece of compressed information (due to the lack of a better simple explanation), which can be used to determine which transactions are included in the block.Your node will then crawl pre-verified transactions, verify the block headers, and if all pass, forward the “compact blocks”.
This optimization is actually why miners no longer rely on centralized and licensed relay networks, such as the FIBRE previously maintained by Matt Corrallo and the briefly existing Falcon Network.Because the relay speed of point-to-point networks is slow, miners used to have to connect to Falcon Network to ensure low block relay latency with other miners.
miner
The miners obviously want to see everything.They are profit-driven entities that want to be able to filter out transactions containing the highest paid fees from the largest set of pending transactions possible.This is how they maximize profits and earn revenue to continue to expand their business and stay competitive.
They are actually getting funds from the memory pool.Their motivation for obtaining any effective paid transaction is so strong that they have built numerous systems historically, now and almost certainly future, even informal arrangements available in society, designed to allow users to submit transactions directly to miners rather than through an open peer-to-peer network.
Block browser, on-chain analysis tools, etc.
Like miners, they want to see every pending transaction that has been created and broadcast to the world.The main difference between the two is that miners directly charge fees for these transactions, while blockchain browsers and analytics companies indirectly profit these transactions by presenting, analyzing and providing information analysis in profitable products.
I can’t list any specific examples involving cached memory pool data, but it is well known that on-chain analytics companies regularly purchase on-chain transaction activity metadata that is acquired privately.They also operate witch bitcoin nodes that will peer to peer with nodes throughout the network as widely as possible in order to narrow the range of nodes that initially broadcast transactions.
Block browsers also profit from visual presentation of blockchain and memory pool data, and their entire business model revolves around this.If this information can be presented in a practical or novel way or information obtained from it, accessing more data and presenting it to users means more potential profit information can be obtained.
Information desires to flow
All these different types of users benefit from “a” public memory pool for a simple reason: information flows freely between them.As long as there is enough fee to pass the minimum relay filter, it conforms to consensus and does not pose a legitimate denial of service or resource exhaustion risk to a single node, it can provide value for all kinds of users to spread information in each individual memory pool in the network.
Without a fully functional public memory pool, the only viable alternative for all these individual users of different purposes is a centralized solution, or a difficult-to-manage chaos, a hasty and messy attempt to build fragmented public memory pools, each user needs to track them individually.
This may not only bring concerns about manipulating rate data, cheating users, and retrieving value by private relay transactions.Without a healthy and open public memory pool, these are problems that Bitcoin must face.
In a subsequent article, I will look at these issues, as well as the different types of memory pool filters and their reasons for their existence.
