It's a mix of extreme sparsity but with the routed expert doing a non trivial amount of work (and it is q8), and projections and routing not being quantized as well. Also the fact it's a QAT model must have a role I guess, and I quantized routed experts out layers with Q2 instead of IQ2_XXS to retain quality.
Not trying to give anyone homework thinking out loud :
One thing I would love to see is if this dogfoods itself
Like would dsv4 with q2 be able to do this task itself on this hardware ?
Sidenote: I wish I had a M4-m3 … thinking about getting a ASUS ROG Flow Z13 Gaming Laptop (Model GZ302EA-XS99) uses pcie 4.0 so disk might be a little slower, but I want to see how this does on like Vulcan :)
Why does a CoC have anything do do with a maintainer stepping down due to a technical disagreement?
And the way you word it makes it seem like you think CoCs are bad or “forced”; I don’t particularly want to engage there, but I’d encourage you to reflect on why you think that.
As someone that has lived in several countries, and currently in Canada, I will respectfully have to disagree both about your opinion of the CBC and beliefs about government intervention. Some parts of Canadian local/provincial/federal government seem deeply dysfunctional, but some are extremely helpful and better than I’ve seen elsewhere.
You're seeing what remains of a strong and healthy period of Canadian history. But almost every facet is currently under threat and significant stress.
You can't fund a "news" organization with public money and expect them to be an independent agency that holds the government's feet to the fire. They don't do much more than add a thin veneer of objectivity and independent analysis for any significant government initiative.
The article posted above is just a small piece of evidence for the broken nature of the media. There is a torrent more spewing out daily.
Counterpoint: You can't fund a news organization with private money and expect them to hold private organizations' feet to the fire. Any time they are critical of a major business, they lose advertisers.
Personally, I have a problem with the viewpoint that democratic governments are some sort of adversary when in actuality, private businesses are the entities that have no accountability and we have zero direct control over. I agree there should be independent criticism of government, but IMO state media from a democratic government is naturally going to be better overall than only having private media.
Without government picking winners and loser (through protective legislation and regulation) the free market means that honest competition keeps corporations in check. There is an adversarial element inherent in the system. The reason we have so many entrenched monopolies is because they have captured the government.. a government that has no adversary and isn't even properly monitored by the corrupt and complicit media.
Everyone has their own interests in mind and so everyone has to be balanced against each other. Come on people we figured this out hundreds of years ago.
Tell that to companies doing extremely large-scale machine learning. Or any cloud infrastructure provider. Or CDNs. Or literally any video production company that owns a render farm. Or any company doing large-scale media transcoding/streaming.
Maybe "don't have thousands of servers" is just a bad take :)
