Stored value

A value that is stored on disk and loaded lazily.

This is useful for transmitting large objects between processes.

StoredValue

from amltk.store import StoredValue
import pandas as pd
from pathlib import Path

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
path = Path("df.csv")
df.to_csv(path)

stored_value = StoredValue(path, read=pd.read_csv)

# Somewhere in a processes
df = stored_value.value()
print(df)

path.unlink()

   Unnamed: 0  a  b
0           0  1  4
1           1  2  5
2           2  3  6

You can quickly obtain these from buckets if you require

StoredValue from bucket

from amltk import PathBucket
import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
bucket = PathBucket("bucket_path")
bucket.update({"df.csv": df})

stored_value = bucket["df.csv"].as_stored_value()

# Somewhere in a processes
df = stored_value.value()
print(df)

bucket.rmdir()

       a  b
index      
0      1  4
1      2  5
2      3  6

class StoredValue
dataclass

Bases: Generic[K, V]

A value that is stored on disk and can be loaded when needed.

def value()

Get the value.

Source code in src/amltk/store/stored_value.py

def value(self) -> V:
    """Get the value."""
    if self._value is None:
        self._value = self.read(self.key)

    return self._value