Welcome to Antimatter Python library documentation!#

About#

The antimatter library offers a comprehensive solution for efficient data processing, leveraging intelligent algorithms to automatically infer data types and formats. Designed to encapsulate data into capsules, load capsules, and handle a variety of administrative API tasks related to the Antimatter APIs, it simplifies the interaction with data by abstracting the complexity of underlying data operations. Supporting a wide range of data types including dictionaries, lists of dictionaries, scalars, PyTorch tensors, and LangChain objects, the library is versatile and powerful for a multitude of data processing needs.

Features#

  • Intelligent Data Processing: Automatically infers the datatype and format of the input data, ensuring optimal processing strategies are employed.

  • Wide Data Type Support: Handles dicts, list of dicts, scalars, PyTorch, LangChain, and more, providing flexibility across different data handling scenarios.

  • Encapsulation and Capsule Loading: Offers functionality to encapsulate data into a processable format and efficiently loads these capsules for further operations.

  • Administrative API Support: Comprehensive coverage of administrative API functionalities, allowing for seamless management of data processing tasks.

  • Performance Optimized: Engineered for efficiency, the library minimizes processing time and resources, catering to high-demand environments.

Getting Started#

Installation#

pip install antimatter

Quick Example#

Creating an Antimatter Capsule with data:

sess = new_domain("email_address")
sess.encapsulate(data=data, write_context="mycontext", path="s3://mybucket/capsulefile")

The type and format of the data will be automatically inferred and encapsulated into a capsule. To load the data:

cap = sess.load_capsule("s3://mybucket/capsulefile", read_context="readcontext")

# Do something with the capsule

The capsule allows easy conversion between data types, or even changing the default type of the data when it’s loaded out of the capsule:

# given Capsule cap, get the data as a pandas DataFrame:
data = cap.data_as(dt=Datatype.PandasDataframe)

Data Types Support#

The library supports a variety of data types, each handled uniquely to optimize processing:

  • Dicts and List of Dicts: Ideal for structured data resembling JSON.

  • Scalars: Basic data types (e.g., integers, floats) are processed directly.

  • PyTorch Tensors: Integration with PyTorch for ML model data handling.

  • LangChain vector stores: Integration with FAISS for efficient vector storage and retrieval of embeddings.

Getting started#

First step is to create an account and get a domain ID and an API key. You can do this by visiting the API console here: https://app.antimatter.io Once done, follow the instructions in the installation section below to install the python client library and follow the examples to get started.