Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
nathanasmith
on Sept 12, 2024
|
parent
|
context
|
favorite
| on:
Serving AI from the Basement – 192GB of VRAM Setup
The system has 512 GB of RAM so while it'll be slower at inference, he really has about 704 GB at his disposal to run the model assuming he distributes the weights across the VRAM and system RAM.
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: