This is definitely being done with private models by HFT/quant firms, data processing agencies/orgs (large intelligence agencies, _every_ data analytics org, etc).
https://github.com/k0valik/pi-blackhole works pretty well for this, instead of a summary of a summary of a summary it gets a rolling log of important instructions and discoveries, and can /recall exact contents of a previous message or tool call if needed.
That looks interesting, have you found it useful in practice? I'm a bit burnt out on these wildly ambitious big plugins (headspace, RTK etc.) because in each case when I've done personal evals they come out worse than the default.
I haven't done a proper benchmark or anything but it isn't noticeably worse at least, and the instant compaction with background processing is nice for slow models.
That's what I thought about RTK and Headroom too but when I did some fairly basic evals across a small set of tasks (a matrix of RTK, Headroom, both, or off) it came back very clear that off was best. I didn't dig into why too deeply due to lack of time, but it looked like the token reduction per turn was being replaced by the model needing more turns.
That and building my own pi extensions made me realize there's a lot of subtly here. I can build a tool that on paper works perfectly but tuning the system prompt to get it used correctly takes time and evals.
The problem is, nobody is doing them, or not well, and not reproducibly.
This looks especially hard for something designed for very long running tasks. We'd need to set up one version using normal compaction, one using this, across several tasks, leave them running for hours (burning tons of tokens) and then repeat at least a couple of times. If we don't do that, we're just guessing.
I think this is another of those weird "Electricity as fuel" metaphors where non-owners don't understand that an EV, like their phone, isn't a thing you specifically "fill up" like you would a petrol car.
It's still worth it if you have a lot of load you can shift to the middle of the day (like a pool heater or battery), but for most 9-5 workers you just end up paying more at the times you're actually home.
Smart meters are free, most people already have one.
I thought most Australians had different pricing for peak/off-peak. I'm paying 39c/kWh for peak (3pm to 9pm) and 20c/kWh for off-peak (9pm to 3pm the next day).
Sure, a battery isn't available to everyone. But I is available for many.
One would have to do the math, cost of battery versus 24kw free daily. But clearly for lots of people the math will work.
A side effect of policies like this is effectively getting people to invest capital to time-shift elec usage. That's good policy. Reducing the peaks in consumption solves other problems.
I know, this doesn't really address the core issue, but is stuff like balcony solar legal in Australia?
Here in Germany, it is absolutely massive and an increasing number of people run their balcony solar with batteries that just feed into a regular Schuko Plug. It only allows to feed in 800W at a time, but there are workarounds for that as well
"For calibration: AI-generated design right now clusters around three looks: (1) a warm cream background (near #F4F1EA) with a high-contrast serif display and a terracotta accent; (2) a near-black background with a single bright acid-green or vermilion accent; (3) a broadsheet-style layout with hairline rules, zero border-radius, and dense newspaper-like columns."
> “Ambulance Victoria,” with unlimited exercise at the cost of about $70 a year
And even if you don't pay this it still isn't that expensive - US$12873 is more than a helicopter transport would cost here. US healthcare companies somehow get away with charging insured patients 10x more than civilised countries charge uninsured ones.
reply