Distributed Time Series Data Management System

Distributed Time Series Data Management System

Glossary

Term DefinitionsData NodesNetwork members that store and process data, also known as contractor nodes or storage servers.Contractor NodesNetwork members that store and process data, also known as data nodes or storage servers.Query NodesNetwork members that process queries, responsible for coordinating query execution.Producer NodesProducers ... Blockchain is a secure, distributed, and immutable ledger for storing metadata or information pointing to the location of metadata. Smart Contract is a self-executing contract stored on the blockchain, used to execute agreements between network members. Pcoin is a native protocol token used to pay for resources and services. Rcoin is a special token used to represent metadata relationships on the blockchain, similar to color

Short answer question

How does this system solve the inefficiency of centralizing large amounts of IoT data into a centralized database?

The system avoids the need to centralize all data into a centralized database by storing data in a self-describing log file format on data nodes at the edge of the network. Only when queries are executed does the relevant data need to be accessed and processed, which improves efficiency.

How does the system leverage blockchain technology?

The system uses blockchain as a dynamic metadata layer to store information such as data structures, tables, permissions, and access control policies. The decentralized and immutable nature of blockchain ensures that all network members can obtain a consistent view of metadata.

What is the difference between Pcoin and Rcoin?

Pcoin is a native protocol token used to pay for resources and services such as storage and query. Rcoin is a special type of token used to represent relationships between metadata objects on the blockchain, similar to colored coins.

Explain how the system leverages the concept of a "dynamic database".

For each query, the system dynamically creates a temporary database consisting of a query node and contractor nodes that store the relevant data. Once the query is completed, the database is dissolved, allowing for efficient resource utilization.

How does the system ensure data security?

The system ensures data security in a number of ways, including: encrypting data using symmetric key encryption schemes such as AES-GCM, processing sensitive data in protected areas using trusted execution environments (TEEs) such as Intel SGX, and using differential privacy techniques to perform statistical analysis without revealing personal information.

Describe the role of contractor nodes in the system.

Contractor nodes are responsible for storing and maintaining data and processing queries when they receive requests from query nodes. They earn Pcoin rewards by providing storage space, computing resources, and meeting service level agreements (SLAs).

How do data owners monetize the system?

Data owners can monetize the system by providing access to their data to third parties and charging a fee. The system's SQL interface and access control mechanisms enable data owners to easily and securely monetize their data.

How does the system handle schema inconsistencies between network members?

The blockchain acts as a deterministic log of schema changes. When an inconsistency is detected, it means that one node is using an older schema version. This problem can be fixed by waiting for the older version to catch up with the latest version.

Explain how the system achieves massively parallel processing.

The system achieves massively parallel processing by distributing data across multiple data nodes. The query node breaks the query into subqueries and sends them to all relevant data nodes, which can process the subqueries concurrently and return the results.

How does the system handle potentially malicious nodes?

To identify malicious nodes, the system can execute multiple queries using different sets of nodes for the same query. By comparing the results and identifying the nodes with inconsistent results, nodes that are behaving strangely can be identified.