Recent blocks
The Parallax client stores recent blocks in a LevelDB database. This is a persistent key-value store that can be queried very quickly. The LevelDB database is supposed to be run on top of a fast SSD hard disk so that the disk IO is not bottlenecked by the underlying hardware. In addition to basic storage, the LevelDB database supports batch writes and iterations over the keyspace in binary-alphabetical order. The database is periodically compacted to reduce the operational cost of accessing individual items. This is achieved by flattening the underlying data store for a given range of keys. Any deleted or overwritten items in that key range are removed and the surviving data is reorganised for efficiency. The Parallax client also tracks several performance metrics for the LevelDB database that can be monitored via the metrics subsystem. These are:| METER | FUNCTION |
|---|---|
compTimeMeter | Meter for measuring the total time spent in database compaction |
compReadMeter | Meter for measuring the data read during compaction |
compWriteMeter | Meter for measuring the data written during compaction |
writeDelayNMeter | Meter for measuring the write delay number due to database compaction |
writeDelayMeter | Meter for measuring the write delay duration due to database compaction |
diskSizeGauge | Gauge for tracking the size of all the levels in the database |
diskReadMeter | Meter for measuring the effective amount of data read |
diskWriteMeter | Meter for measuring the effective amount of data written |
memCompGauge | Gauge for tracking the number of memory compaction |
level0CompGauge | Gauge for tracking the number of table compaction in level0 |
nonlevel0CompGauge | Gauge for tracking the number of table compaction in non0 level |
seekCompGauge | Gauge for tracking the number of table compaction caused by read opt |
Freezer/ancients
Older segments of the chain are moved out of the LevelDB database and into a freezer database. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification. The Parallax client also tracks some basic metrics relating to the ancients database that can be monitored:| METRIC | FUNCTION |
|---|---|
readMeter | Meter for measuring the effective amount of data read |
writeMeter | Meter for measuring the effective amount of data written |
sizeGauge | Gauge for tracking the combined size of all freezer tables |
chaindata directory, which is inside datadir, but it can be defined by passing --datadir.ancient <path> to the Parallax client on startup. The freezer is designed to have a read operation complexity of O(1), involving only a read for index items (6 bytes) and a read for the data. This design makes the freezer performant enough to run on a slow HDD disk, permitting people to run Parallax nodes without requiring a huge SSD. The ancient data can also be moved later by manually copying the directory to a new location and then starting the Parallax client passing the new path to --datadir.ancient.
Using the freezer
If the Parallax client cannot find the freezer, either because the database was deleted or because the Parallax client received an incorrect path, then the node becomes unstable. It is explicitly forbidden to start the Parallax client with an invalid path to the freezer. However, if the state database goes missing the Parallax client can rebuild all its indices based upon data from the freezer and then do a snap-sync on top of it to fill in the missing state data. This can be used to deliberately clean up a node. Passing--datadir --removedb starts the process. The Parallax client will ask whether it should delete the state database, the ancient database and the LES database. Choosing to delete the state database only leaves the block bodies, receipts, headers etc intact in the freezer, meaning rebuilding the state will not include re-downloading data from the network. The Parallax client will then rebuild the state from the freezer reusing that existing block and receipt data. In doing so, unused data and accumulated junk data will be pruned from the state database. This process can take an hour or more.
Unclean shutdowns
If the Parallax client stops unexpectedly the database can be corrupted. This is known as an “unclean shutdown” and it can lead to a variety of problems for the node when it is restarted. It is always best to shut down the Parallax client gracefully, i.e. using a shutdown command such asctrl-c, docker stop -t 300 <container ID> or systemctl stop (although please note that systemctl stop has a default timeout of 90s - if the Parallax client takes longer than this to gracefully shut down it will quit forcefully. Update the TimeoutSecs variable in systemd.service to override this value to something larger, at least 300s). This way, the Parallax client knows to write all relevant information into the database to allow the node to restart properly later. This can involve >1GB of information being written to the LevelDB database which can take several minutes.
If an unexpected shutdown does occur, the removedb subcommand can be used to delete the state database and resync it from the ancient database. This should get the database back up and running.