My system is quite similar to your, my GPU is a 6950 XT and CPU a Ryzen 5 2600x, same amount of RAM, and I feel your pain. It sounds very similar to my experience from a few months ago. When it comes to tool calling, there are multiple possible issues; some models have borked templates bundled with the model file, some models are not trained on tool calling, some agent harnesses doesn't support the tool call output from the model very well, some quantizations ruin the models' abilities to call tools.

My suggestions if you want to further experiment with local models are to use llama.cpp instead of ollama [1], learn a little about the parameters that tune how much VRAM is used [2], look online for jinja template fixes for the model you're testing [3], and choose a model that was designed to do the task you want to achieve, with as high quantization as you can fit. The maximum model size you can run is VRAM + RAM, although you want as little of the model to be in system RAM as possible.

I'm running North Mini Code IQ3_XXS with some tuned parameters to fit my current tasks, and while it is not perfect for everything, it has not failed any tool calls I've asked it to make, or that it figured it should make on its own.

[1]: https://sleepingrobots.com/dreams/stop-using-ollama/

[2]: https://github.com/ggml-org/llama.cpp/blob/master/tools/serv...

[3]: https://gist.github.com/jscott3201/e4b155885cc68c038d6ac8909...

Interesting. Making low latency correct tool calls correctly is pretty important in voice AI cascading models(STT LLM TTS). Realtime Models are still 2x the cost and there are only 2 providers openai and google that are in the race. For cost control it has to be cascading models

For llms Sadly the only model right now that fits the bill for LLM is GPT 4.1 and it’s standard in my stack because thinking models have unacceptable latency(>=1 sec) even though they are good at tool calling. The main issue with 4.1 is that it can make still mistakes and prompt prose has to be tuned quite a bit.

I wonder if any local models can be tuned to match the response time and tool calling while supporting many languages.

"My suggestions if you want to further experiment with local models are to use llama.cpp instead of ollama"

Or at least LM Studio if you want to play around with a lot of different models. Im currently using it with my 7800xt and Vulcan as i found it left my OS more stable ROCm does. I had a few system crashes with ROCm and running out of VRAM for the OS.

I'm in the same boat as you, and every time I touch a Windows PC I am again and again surprised at all the junk they put in there by default. Not only Microsoft, but also the OEMs. So much junkware, so many accounts they want you to create, and all without any benefit to the user.

The written price is the written price, and that's final. No tips.

Thanks for the link and tl;dr, even though it's a quite short video (3:16). I found your explanation very interesting because I have intuitively felt like this is accurate, but never knew what the underlying process is. I have been following this for years already, first absorbing information about anything where I need to make a decision, and then just leave it to stew in the back of my mind. And after a while the answer just appears in my mind, without me really understanding where it comes from.

I’ve heard this being discussed as focused and diffuse modes of thinking - https://fs.blog/focused-diffuse-thinking/ - although not in relation to verbal and non-verbal thinking.

There are probably several libraries that offer this functionality, the question is just what your other requirements are. It is also built into most modern web browsers, just check out the MDN for Intl.DateTimeFormat with the dayPeriod option [1].

[1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

This page has horrible scrolling. I really don't understand why anybody creates this kind of scroll. Are they not using what they create?

I don't think they create it, they just use some template that comes with it.

And then they don't ever open the page, right?

You are right, these quirks are not something you struggle with very often. The only one that has been troublesome at some point during my now 8 years as a professional, mainly Javascript with Vue, web developer is the automatic semicolon insertion example.

The simplest fixes for it is to just insert semicolons yourself, always use const, and not start any lines with [ or (.

I recently switched jobs, and as I've become more comfortable in my new job, I realized that my old job completely sucked the creativity and fun out of programming for me. I did not have any energy for any projects in my spare time while working in my previous job, so the 1,5 months I've been focusing on taking small ideas and running with them.

This weekend I got an idea to create a 2D-pattern from a 1D initial pattern. I put it on https://ige.land/, and I find it very fun and satisfying to play around with. It's not so elegant and pretty, but it's fun for me, and that's all I was looking for! Click any cell to change that color in the pattern. Right click or long press to set the length of the 1D pattern. It's also fun to first create a pattern and then resize the browser window to see how it looks like.

So, what I'm working on is not really an interesting project or product. I'm working on my creativity, playing with code, and in general having more fun with programming. Making it magical for myself again.

I'm not sure if it's 100% in line with the spirit of the thread, but it's what I'm most excited to be working on! :-)

Your site resets the connection.

I clicked around a little and I think that the correct page for me is this one, perhaps it is the correct one for you too? The script doesn't work, but it looks like it could work with some tweaking. https://www.facebook.com/adpreferences/ad_settings/?section=...

I love userscripts, and I'm so happy to see and hear that other are using them and developing them. Well done, Will! I hope you keep making userscripts!

Thank you and I really appreciate userscripts too. While I'm taking a break from userscripts to make my website on Val Town, I only handled the main problems of my two primary forums. I'm sure I'm going to at least make versions of what I have for my other forums and new userscripts to improve their forum searches.