Blog
Education

Understanding Data Availability on Arweave

By
Bithiah Koshy
November 6, 2024
5 min read
Share this post

Data availability (DA) refers to the assurance that all data referenced in a block is actually available to network participants. A block is a fundamental data structure that contains several key pieces of information such as transaction data, state, etc.  Without full data availability, nodes can't fully validate the state of the network, which can lead to security issues and potential attacks.

Data Availability Committees

Data Availability Committees (DACs) focus on data management and availability, particularly for scaling solutions. These do not apply to Arweave and AO but L2s instead, where scalability issues do exist. 

For instance, optimistic chains use DACs for managing off-chain data storage. In terms of AO, Arweave is for both storage and data availability. It is important to note that Arweave as DA is not a validium, as a validium is a scaling solution that enforces integrity of transactions using validity proofs like ZK-rollups (for Ethereum).

How Arweave approaches DA 

Wildfire

Wildfire is Arweave's implementation of their Adaptive Interacting Incentive Agents (AIIA) game that optimizes data availability through a peer ranking system. 

Nodes rank peers based on two key factors: 

  1. Generosity in sharing new transactions/blocks
  2. Responsiveness to information requests

How does this maintain high data availability? Nodes keep a list of peers and score them based on their data-sharing speed. High-scoring peers receive messages first (in parallel), while low-scoring peers receive them second (one after another). The list gets cleaned up regularly by removing poor performers, but new peers get a grace period to prove themselves.

Succinct Proofs of Random Access

Succinct Proofs of Random Access (SPoRA), is a mechanism within Arweave to make sure data gets stored and retrieved efficiently on the network.Instead of just focusing on computing power, SPoRA rewards miners who can access and retrieve data the fastest. The faster and more efficiently miners can store and fetch data, the more they earn. 

This approach uses less energy than their previous system and encourages miners to actually maintain good data storage systems rather than just pooling their computing resources together.

Blockshadows

Blockshadows contain transaction IDs instead of full transactions, allowing nodes to reconstruct blocks if they already have the transactions in their mempool. 

When a block is formally proposed, parts of its transaction data are already distributed across the network. This results in transactions to be included in blocks almost as quickly as they can be spread throughout the network. 

Here we effectively split data availability into two phases: 

  1. Pre-block distribution of transactions 
  2. Block proposal using references (IDs) to already-distributed data

AO and Arweave

Messages and process logs in AO are stored on Arweave, ensuring permanent data availability. Scheduler Units (SUs) are required to ensure that message data is uploaded to Arweave and made permanently available. The protocol's holographic state mechanism relies on logs of interactions being recorded and accessible on Arweave, allowing state to be derived from these logs rather than requiring consensus on computation results.

Unlike Layer 2 solutions that rely on DACs, AO leverages Arweave's established infrastructure for both storage and data availability, creating a robust foundation for its decentralized computation platform.

Share this post

A monthly exploration of the challenges and opportunities of a decentralized future.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Related blogs

Education
October 7, 2024

Private Compute and Why It Matters

AO enables private compute by leveraging FHE and ZKPs to perform secure, verifiable computations on encrypted data, ensuring privacy for applications like healthcare, AI, and blockchain.
Education
September 5, 2024

Quick Guide to ArFleet: The Decentralized Storage Layer Built on Top of Arweave and AO

ArFleet is a decentralized temporary storage system on AO and Arweave, handling data encryption, bundling, provider selection, and verification.
Education
September 3, 2024

Quick Guide to Permanent Storage on Arweave

Arweave is a decentralized storage protocol for permanent data preservation, aiming to reshape the internet by ensuring digital content cannot be altered or deleted.