indeed, I'm switching from claude to codex

I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.

even a 3090 will give you the VRAM headroom. i run Q8 on an 3090/A6500 combo. well, Q8 of 3.6-27B. I'm building the Q8 GGUF for 3.8 now, assuming mine will finish before someone else's.

Others in this thread said it runs on RTX 4090.

i am switching to codex, opus 5 failed me

great for mobile apps, fine for desktops, basically unusable for browsers unless you do wasm, if web can be revamped/improved it can be the best option for cross platform GUI

You are in luck then, use jaspr.

I love this framework! It offers all the benefits of Dart and the Flutter widget paradigm on the web, while still providing access to the DOM. I'm using Jaspr for all of my marketing sites. It's beautiful, and I can ship in minutes.

> basically unusable for browsers unless you do wasm

Relying on WASM means that nobody on Apple platforms who has Lockdown Mode enabled for better security will be able to use it.

I see that as a win

I’ve used flutter for a couple of toy web apps. One is just a personal tool , another is a small party game.

Now I might vibe code such things using React, but 5 , 6 years ago Flutter got me what I wanted fast.

Firebase integration is absurdly easy. You can make a crud app in about 30 minutes

Yes but its best work with SPA or WASM type of app like games for example

It's probably not the best for anything in particular.

It's the fastest way to build something if you're not a big html person.

I've never particularly been good at hmtl tbh. The older I get, the more I'm concerned with the ease of tools/frameworks vs what's the most optimal.

its good at mobile, developer experience simply just better compared to java,kotlin xml UI declaration back then

Censorship being explicitly a function of government, at least in legal terms, seems pertinent here.

Which law are you thinking might make censorship a crime?

"this is not a criminal activity where I am"

true, switched from ollama to llama.cpp these days and it's good. wonder if this is also the best option for edge ai deployment(currently use it on desktop)

all my LLM coding is in go these days

that undercuts their core business, so it will be a defensive play at most to fend off mac and amd's local inference offerings

I don't know how people can say this with a straight face. Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced, specifically for edge robotics, computer vision and ML.

The absolute fastest desktop Mac GPUs cannot beat an Nvidia laptop GPU in prefill or inference speeds. Apple Silicon is a non-entity for professional datacenter deployment and arguably unusable for frontier models at agentic context sizes. AMD is Nvidia's primary worry, and they're not doing much better in terms of GPGPU SOC compute.

>Nvidia was selling desktop-grade ARM SOCs before Apple Silicon was ever announced

You can believe all you want that the dinky little jetson boards were desktop grade when historically the ARM SoC portion of a jetson board couldn't even keep up with broadcom/rockchip SoCs. It's taken until recently for the actual arm compute portion of Nvidia SoC's to be worth a damn at all, and they still fall far behind Apple let alone the rest of the pack like Qualcomm/Samsung.

I don't have to believe. I've run KDE and GNOME on the Tegra boards, you get full-fat CUDA support without sacrificing Vulkan drivers. It's incredible.

You can believe all you want that good single-core performance will corner the edge compute market. It hasn't, Graviton has more buy-in than any Apple Silicon chip ever got.

Every time nvidia takes its fab time and uses it to build anything other than datacenter chips it is losing money due to the massive markups the datacenter products have. Expanding their consumer offering means the datacenter backlog is going down which is very bad for their margins. Consumers will never pay 10-100x what it costs to fab something like datacenter users will.

In many cases it isn't Nvidia paying for the fab time. For instance, the Nintendo Switch 2 is basically a pure-play design and support product for Nvidia, while Nintendo negotiates with Samsung for the actual SOC prices. Their IP philosophy is closer to AMD's than Apple's, Nvidia has long roped in 3rd party manufacturers to mark up, integrate and sell their hardware.

anecdotal: golang, with c or simplified modern c++

what are those cheap fine-tuning services these days?

Tinker is pretty cheap. Prime Intellect if you want more flexibility.

unsloth for small use cases, tinker for enterprise level use cases?

I haven’t used their platform so can’t speak to this, but super brilliant team and big fans of their work overall!

I would not say cheap per say, but in the context of "where is the business success going to be" that's one of the reasons why Mistral focuses on providing tuned model on premises to their customers.