Indexer Nodes

Indexers respond to user requests which are specified with a schema to create, update, and manage datasets through a NoSQL protocol. Requests to write to a dataset are processed in the Database Engine, where a Vector Index is constructed. These data are then permanently stored in the decentralized storage system via the Glacier DA Layer.

Architecture

The Glacier Database Engine efficiently stores and manages large volumes of unstructured vector data in a decentralized environment. It harnesses the power of decentralized agents (dAgents), large language models (LLMs) like OpenAI's ChatGPT-4, and blockchain technology to provide a secure and scalable data storage solution.

dApp/dAgent Integration

At the start of Glacier VectorDB lies the dApp/dAgent, a decentralized application that interacts seamlessly with the system. The dAgent utilizes the LLM's embedding capabilities to convert unstructured data into vectors, essentially turning complex information into a more manageable numerical representation. These vectors are then stored in Glacier VectorDB using the Glacier SDK, a dedicated software development kit.

Ensuring Data Integrity

The Glacier Database Engine incorporates several safeguards to maintain data integrity. Each transaction's signature is meticulously verified before it is applied to the internal table model, ensuring the authenticity and immutability of the data.

Data Permanent Available

Periodically, the Vector Engine automatically generates Data Blocks, which aggregate Tx vector data from the previous interval. These Data Blocks are then committed to the Glacier DA Layer, a programmable, modular, and scalable data availability layer that integrates with decentralized storage to support AI applications requiring heavy and reliable data workloads.