Overview ¶

Package modelcache provides an LRU model file cache for pre-caching GGUF models on Kubernetes nodes via a DaemonSet.

Experimental — this package is not yet wired into the main framework.

Package modelcache provides an LRU model file cache for pre-caching GGUF models on Kubernetes nodes via a DaemonSet.

Index ¶

type Cache
- func NewCache(dir string, maxSize int64) *Cache

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Cache ¶

type Cache struct {
	// contains filtered or unexported fields
}

Cache manages local model files with LRU eviction.

func NewCache ¶

func NewCache(dir string, maxSize int64) *Cache

NewCache creates a cache that stores files under dir with a maximum aggregate size of maxSize bytes.

func (*Cache) Evict ¶

func (c *Cache) Evict()

Evict removes least-recently-used entries until the total cache size is at or below maxSize.

func (*Cache) Get ¶

func (c *Cache) Get(ref string) (string, bool)

Get returns the local file path for the given model ref if it is cached.

func (*Cache) Prefetch ¶

func (c *Cache) Prefetch(refs []string, pullFn func(ref string, dest string) error) error

Prefetch downloads models for the given refs using pullFn, which should fetch the model identified by ref and write it to dest. Already-cached refs are skipped.

func (*Cache) Put ¶

func (c *Cache) Put(ref string, srcPath string) error

Put copies the file at srcPath into the cache under the given ref.

cache package - github.com/zerfoo/zerfoo/model/cache - Go Packages

cache

Details

Repository

Links

Documentation ¶

Overview ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

Types ¶

type Cache ¶

func NewCache ¶

func (*Cache) Evict ¶

func (*Cache) Get ¶

func (*Cache) Prefetch ¶

func (*Cache) Put ¶

Source Files ¶