Skip to content

DataSet.to_arrow_refs

This will return a list of Ray ObjectRefs to Arrow Tables. This is a blocking call. It will NOT move data to your local machine.

Return

List of Ray ObjectRefs to Arrow Tables

Source code in pyquokka/quokka_dataset.py 
40
41
42
43
44
45
46
47
48
def to_arrow_refs(self):

    """
    This will return a list of Ray ObjectRefs to Arrow Tables. This is a blocking call. It will NOT move data to your local machine.

    Return:
        List of Ray ObjectRefs to Arrow Tables
    """
    return ray.get(self.wrapped_dataset.to_arrow_refs.remote(self.dataset_id))