Indexer Nodes
Last updated
Last updated
Indexers respond to user requests which are specified with a schema to create, update, and manage datasets through a NoSQL protocol. Requests to write to a dataset are processed in the Database Engine, where a Vector Index is constructed. These data are then permanently stored in the decentralized storage system via the Glacier DA Layer.
The Glacier Database Engine efficiently stores and manages large volumes of unstructured vector data in a decentralized environment. It harnesses the power of decentralized agents (dAgents), large language models (LLMs) like OpenAI's ChatGPT-4, and blockchain technology to provide a secure and scalable data storage solution.
At the start of Glacier VectorDB lies the dApp/dAgent, a decentralized application that interacts seamlessly with the system. The dAgent utilizes the LLM's embedding capabilities to convert unstructured data into vectors, essentially turning complex information into a more manageable numerical representation. These vectors are then stored in Glacier VectorDB using the Glacier SDK, a dedicated software development kit.
The Glacier Database Engine incorporates several safeguards to maintain data integrity. Each transaction's signature is meticulously verified before it is applied to the internal table model, ensuring the authenticity and immutability of the data.
Periodically, the Vector Engine automatically generates Data Blocks, which aggregate Tx vector data from the previous interval. These Data Blocks are then committed to the Glacier DA Layer, a programmable, modular, and scalable data availability layer that integrates with decentralized storage to support AI applications requiring heavy and reliable data workloads.
