it makes you wonder if it may be more efficient to spend all the weights on one ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

DoctorOetker 29 days ago | parent | context | favorite | on: A global workspace in language models

it makes you wonder if it may be more efficient to spend all the weights on one layer, and have a repeating stack of the same layer, one would presume this axis has already been explored with metaparameter sweeps?

smallerize 28 days ago [–]

That's called a recurrent neural network (RNN).

DoctorOetker 28 days ago | [–]

distinction is that GPT allows a lot of parallel computation compared to RNN, but I see how your remark indicates a convergence towards RNNs indeed.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact