QuokkaContext.from_arrow

Create a DataStream for a pyarrow Table. The DataFrame will be materialized. If you don't know what this means, don't worry about it.

Parameters:

Name	Type	Description	Default
df	PyArrow Table	The polars DataFrame to create the DataStream from.	required

Returns:

Name	Type	Description
DataStream		The DataStream created from the polars DataFrame.

Examples:

>>> import polars as pl
>>> from pyquokka.df import QuokkaContext
>>> qc = QuokkaContext()
>>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}).to_arrow()
>>> stream = qc.from_arrow(df)
>>> stream.count()

Source code in pyquokka/df.py

def from_arrow(self, df):

    """
    Create a DataStream for a pyarrow Table. The DataFrame will be materialized. If you don't know what this means, don't worry about it.

    Args:
        df (PyArrow Table): The polars DataFrame to create the DataStream from.

    Returns:
        DataStream: The DataStream created from the polars DataFrame.

    Examples:

        >>> import polars as pl
        >>> from pyquokka.df import QuokkaContext
        >>> qc = QuokkaContext()
        >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}).to_arrow()
        >>> stream = qc.from_arrow(df)
        >>> stream.count()

    """

    self.nodes[self.latest_node_id] = InputPolarsNode(polars.from_arrow(df))
    self.latest_node_id += 1
    return DataStream(self, df.columns, self.latest_node_id - 1, materialized=True)