Please login or register.
This location

Monero scaleability ?

dnaleor posted this 8 years ago

dnaleor | 8 years ago

I'm far from the best technical guy around here, but recently I was thinking about scaling of Monero. Better think about it early than ignoring the problem (assuming there is one).

It's seems the anonymity of XMR has a trade-off: full pruning isn't possible. Bitcoin nodes can have full pruned nodes with a node only keeping the last x blocks. XMR doesn't have" UTXO's", only "TXO's", therefore images keys are used to check for double spends. When sending a transaction, you need also a chunk of TXO's to be able to construct your ring signature. Using the same chunk for a long period of time reduces the anonymity.

Therefore, I think we have these options for different types of nodes:

Watch-Only node: A Watch-Only node could be used to monitor your incoming transactions for your cold storage wallet. To be able to verify incoming transactions, you need to scan the blockchain once from the point where you received your first transaction. After that you only need to check every new block to see if there is an incoming transaction. Key images aren't necessary because invalid transactions (double spend) won't be included in a block. You don't need to store the blockchain, although it's advisable that your node keeps your TXO's (or blocks) associated with your transactions so you can easily start sending transactions if necessary

Send-Only node: Asuming a wallet knows his balance (and thus the TXO's he owns) either by scanning the blockchain once or because the watch-only node stored the specific data needed to calculate the balance, a "Send-Only" node needs a subset of TCO's to be able to build a ring signature. The node can either download some TXO's "on the fly" from the network (but isn't this dangerous for privacy? not sure) or periodically download a (small) subset of TXO's and "refresh" this subset for example on a weekly basis (the smaller the timeframe, the better the anonymity, I assume). Also, one need to be able to check if the TXO's he downloaded form the network are valid. If someone somehow tricked you in downloading a fake subset of transactions, and you try to broadcast a ring signature with those fake TXO's, you will expose your real TXO's! So one need to double (or triple, or quadruple?) check the network. Maybe the checkpointing can be a part of this. if the checkpoints are relatively close to each other (for example one checkpoint every week) one can just download some subsets of TXO's from a few weeks and build his ring sig with those subsets. Don't use only one subset, because it will be obvious that all TXO's in the ring sig you use are from the same week, and only the one's you own are from another period. The TXO subsets should ideally be unrelated on the dates referring to the TXO's you own. Use case for this strange type of node: faucets, lotteries, hot wallets for exchanges

Light Weight Node: Basically the watch-only node and send-only node in one node. When you create your account, the node need to start checking all new blocks from that point in time. When there're no incoming transactions in a certain block, this block can be deleted. When a block contains an incoming transaction, the TXO (or the full block) is stored. The node also stored some blocks unrelated with the TXO's you own to be able to construct ring sigs. The node need to periodically refresh the anonymity set (subset of TXO's)

Mining node: A miner needs the last x blocks of the blockchain and all image keys to be able to verify blocks and build on the most recent version of the blockchain. Maybe our checkpointing system could determine from where a miner needs to keep it's blockchain? This is a "full miner" that constructs his own blocks, not a miner that gets it's work from a mining pool. (so in essence, this means that solo miners and pool operators need to have a "mining node")

Blockchain sharing node This node doesn't store the full blockchain, but only parts of it. The operator can choose the % of the blcockhain he wants to store. This type of node will often be combined with the mining node, but this isn't necessary. Why should a node only store a part of the blockchain? Imagine a future where the blockchain is several hundreds of TB large. If there are scaling problems for the storage and/or bandwidth (not sure this will happen, but one never knows) then it will be costly to store a full blockchain. We maybe won't see free blockhain explorers (or at least, the explorers would have a lot of advertisements, delays etc). If one wants to create a ring signature and needs a certain part of the blockchain that is related to his TXO's, this node (the "send-only node" or the component of it in the "light weight node") pings the network and downloads the blocks/TXO's he need. For this service, the node pays a small fee (in XMR) based on a "payment channel" technology (check [http://www.joystream.co]). Both seeder and downloader can set the rates they want to pay in their client. Refreshing your anonymity set a lot is costly, but better for anonymity. tripplechecking if the data your downloaded if valid is more costly than only doublechecking, but again, better for your anonymity. It's a trade-off.

Full Archival Node: Basically how the nodes work now: all blocks, all images keys, all TXO's are downloaded, stored and shared (at the moment still for free). This node contains the mining node and the network node with setting for storing the blockchain at 100%.

That's about it

And please, correct me if I'm wrong :) I didn't really dive deep in the code. So there are mistakes in this post, I'm sure :)

Replies: 2
Gingeropolous posted 8 years ago Weight: -220 | Link [ - ]

I think one of the problems with breaking the nodes like this is that it allows the network to become fragile. Imagine the case where, for whatever reason, full archival nodes just drop off and the above peer divisions have been implemented. Unless you, yourself, was running one of these variants, would your outputs still be accessible?

Ultimately, these subdivisions further allow the network architecture to depend on the altruism / good will of the participants, which is a faulty dependency as we're witnessing in bitcoin.

The only variant from the above that would secure the ledger in the absence of full archival node is a strong subset of blockchain sharing nodes. In general, I think any effort to decrease the # of full copies of the ledger is dangerous, slippery slope territory.

I need to dive into @smooth's pruning methods. But whom I kidding. I'll barely understand it.

But ultimately, perhaps a good compromise solution would be to have varying levels of compression instead of outright data abandonment. So, you run a full archival node, nothings compressed. Full 500 gig blockchain, ready to go. You're the kind of peer that will be priority #1 when a new peer comes online and needs blocks.

All other variants then have varying levels of compression. Blockchain sharing node, 10% of the blockchain is uncompressed, the rest is tucked away in some CPU intensive compression algorithm. I know indexing of compressed files is possible - DNA sequencing software can retrieve data from compressed files without uncompressing them. This type of node is priority #2 for other peers.

Mining node - last n blocks are uncompressed.

etc., etc.

smooth edited 8 years ago Weight: -382 | Link [ - ]

We don't need to worry too much about this for quite some time. We can't do exactly the same kind of pruning as bitcoin because there is no TXO/UTXO distinction, but we can disable storing older transaction signatures on disk once they are verified, unless you want to be an archive node to serve blocks (in which case you can't prune bitcoin either). At this point you can still be a full node, verify transactions, mine, etc. The only thing you can't do as a non-archive node is serve old blocks to new users.

Signatures represent most of the size of transactions, and the remaining portion of transactions is encoded more compactly than Bitcoin, so this has largely the same practical effect as Bitcoin's pruning (disk storage requirements reduced drastically). In fact most of the transaction itself could also be dropped (other than the output keys and spent key images) but that difference is small enough not to worry about at first.

The various compromise modes you suggest are likely reasonable at some point in the future (such as the partial storage in particular), but the simplest distinction that is very useful and not particularly hard to implement is just archive nodes vs non-archive nodes, with the latter not needing to store signatures.