Data Availability
“Not all data should be broadcasted to the blockchain, but we need to ensure that all the data are decentralised and can be verified.” This is the core principle of OpenSocial’s Data Availability solution, which is a hybrid server of Ethereum JS and Orbit DB.
It seems to make sense to record posts, likes, comments on-chain since they are all social actions of a user. However, in reality, the vast amount of social action and engagement may cause a lot of trouble to the blockchain and user experience. For instance, there may be fifty thousand likes and seven thousand comments or replies of comments just within one post. If all the social actions have to be broadcasted into the blockchain, it will cause three major problems:
Huge latency as the chain cannot process such large amount of transactions
Bad user experience as the user will need to sign every social action as a transaction to be broadcasted onto the blockchain
High cost as gas fee is required for every transaction on the chain
Therefore, OpenSocial’s Data Availability solution is aimed to solve the problems above without undermining the authenticity of the social data. Depending on the properties of the social data, some of them will be broadcasted to the blockchain, while some will be stored in a decentralised server (i.e. Orbit DB).
Please be assured that whether the data is stored on-chain or in a decentralised server, all the data in OpenSocial’s Data Availability is transparent, public and owned by the OpenSocial user. For data stored in Orbit DB, there will be an Explorer page for verification and searching. It is an essential solution for scaling and to accommodate millions of users.
Protocol layer
DA stands for Data Availability, which means where the data is stored and how it can be reached under different situations. Ideally, it should be accessed safely, easily and cost effectively with no latency. Therefore, our DA is a hybrid of on-chain & off-chain data: EthereumJS and OrbitDB.
Data that goes on-chain will be stored in EthereumJS. It is nearly perfect as it is immutable (decentralised servers) and can be easily accessed. However, it may incur high costs and latency when the amount of data exceeds its limit.
Hence, data that does not go on-chain will be stored in OrbitDB. It is a decentralised server so the data is safe without any latency, meaning it can be queried in real time in the Explorer page.
Through this process, we aim to build a standard DA that can unify on-chain and off-chain data, so that not only on-chain data can be verified, but also data stored in OrbitDB is valid and can be verified.
DA Mechanism
On-chain (EthereumJS)
Sync transactions under OpenSocial Protocol to OpenSocial DA
Off-chain (OrbitDB)
The mechanism is basically a simulation of an on-chain transaction and stores the data in OrbitDB.
Ensuring identical result by simulating the same transaction in the same block
Ensuring identical result by different nodes in OrbitDB
Off-chain content ID
uint256 = Random first 128 decimal + content ID 128 decimal = Off-chain ID
LevelDB query
Search based on Off-chain TxID (Entry: offchainId)
Search based on user (Entry: address + time)
Off-chain slot to store value (ensuring off-chain Tx works)
Signature as boolean, avoiding duplicate transactions (same mechanism as EVM)
Interoperability between on-chain & off-chain
Posting a post (off-chain)
Off-chain engagement on an On-chain post
Off-chain engagement on an Off-chain post
All social action stated above will be recorded as an unique transaction, which will be recorded in IPFS and create a query.
chainProofs contains the value + sig in typeData, representing the input of the contract.
Logs represent the output of the contract.
blockNumber represents the block simulated at that time stamp. The corresponding key & sig is equivalent to the public key and signature in OrbitDB nodes, which can be verified by other nodes for the data authenticity.
OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and Libp2p Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses Merkle-CRDTs for conflict-free database writes and merges making OrbitDB an excellent choice for p2p and decentralised apps, blockchain applications and local-first web applications.
EthereumJS is the JavaScript / TypeScript project within the Ethereum Foundation. Work is mainly done within the following GitHub organisation: https://github.com/ethereumjs
Last updated
