Skip to content

Prefetch Loader

Thread-based batch prefetcher for DataLoader(num_workers=0).

PrefetchLoader 

PrefetchLoader(dl: DataLoader, prefetch: int = 2)

Proxy around DataLoader that prefetches batches in a background thread.

Parameters:

Name Type Description Default
dl DataLoader

a DataLoader (typically with num_workers=0)

 required
prefetch int

number of batches to buffer ahead

 2
Source code in tsfast/tsdata/prefetch.py 
89
90
91
def __init__(self, dl: DataLoader, prefetch: int = 2):
    object.__setattr__(self, "_dl", dl)
    object.__setattr__(self, "_prefetch", prefetch)