Feed Apps Remote Create

Appgithub-com-wici-ai-inferssd

Download Pinokio

Sign in Create accountJoin

Sign in Create accountJoin

inferssd

github.com/wici-ai/inferssdUpdated 1d ago

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

Loading community details…

Loading project owner…

Loading launchers…

Loading check-ins…

Community

Sort byCreate post

Ask or share about inferssd…

Loading...

Loading references…

Loading project activity…

Pinokio

Feed Apps Create Remote

0

Projects using this repository

No references listed yet.

Check-ins (0)

Recent commits

cuda: admit every device allocation against free memory (#243)

Jiayang Xuyesterdaye91c7d7

cuda: drop the VRAM tier's prefill scratch retry (#230)

Jiayang Xuyesterday995a620

engine, ggml: grow a Vulkan KV cache in place with sparse buffers (#225)

Jiayang Xuyesterday9a04614

engine: limit a KV cache allocated at once to 32768 cells (#219)

Jiayang Xuyesterdaybf7e15d

engine, metal: let the graph lane read RAM cache slots in place (#218)

Jiayang Xuyesterdayf54c188

engine, cuda, framework: allocate the KV cache as it is used (#215)

Jiayang Xu2 days agoff1e9d4

engine: grow a room-sized RAM cache into room that frees during decode (#214)

Jiayang Xu2 days agoa408a4f

cuda: reclaim expert cache when cuBLAS handle allocation fails (#212)

Chen Zerui2 days ago83369da

engine, ggml, framework: leave a discrete GPU its commit on Windows (#211)

Jiayang Xu2 days agof79ec88

engine, metal: price a unified-memory GPU's buffers as host memory (#209)

Jiayang Xu2 days agobd2cd9f