Stored value

amltk.store.stored_value

A value that is stored on disk and loaded lazily.

This is useful for transmitting large objects between processes.

StoredValue

from amltk.store import StoredValue
import pandas as pd
from pathlib import Path

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
path = Path("df.csv")
df.to_csv(path)

stored_value = StoredValue(path, read=pd.read_csv)

# Somewhere in a processes
df = stored_value.value()
print(df)

path.unlink()

   Unnamed: 0  a  b
0           0  1  4
1           1  2  5
2           2  3  6

You can quickly obtain these from buckets if you require

StoredValue from bucket

from amltk import PathBucket
import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
bucket = PathBucket("bucket_path")
bucket.update({"df.csv": df})

stored_value = bucket["df.csv"].as_stored_value()

# Somewhere in a processes
df = stored_value.value()
print(df)

bucket.rmdir()

       a  b
index      
0      1  4
1      2  5
2      3  6

StoredValue dataclass

Bases: Generic[K, V]

A value that is stored on disk and can be loaded when needed.

value

value() -> V

Get the value.

Source code in src/amltk/store/stored_value.py

def value(self) -> V:
    """Get the value."""
    if self._value is None:
        self._value = self.read(self.key)

    return self._value