I've been working on a project myself over the last few weeks where the documentation is quite minimal. To no surprise the LLMs fell flat at being able to generate any sort of meaningful code. However, I realized that if I focused first on building out documentation and coding tools (linters, parsers, formatters, etc...), LLMs can do a decent job at solving fundamental problems.
I'm curious what percentage of PRs are just the AI blindly writing code and submitting a PR without testing, and which have at least been locally tested to some degree. Any OS maintainers have any insights on this?
Thats the thing, what if the codebases had CLAUDE.md / AGENTS.md files, which clearly dictated that
A) tests need to pass
B) anything you write needs tests
C) the code quality must adhere to these standards
etc.etc.... Helping the LLMs that people Vibe code with, produce better quality results.
By not having these in place, it means people who want to help out, cant. because htey dont understand whats going on.
adding stuff to these files, woudl allow developers to give guidelines / guardrails for developement using these agents.
Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software.
> Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software.
The motivation to help the OSS project should also come with the obligation to learn how the software operates, at least on a conceptual level. The desire to help does not grant people the pass to sledgehammer their way into adding in a feature.
> If someone can’t write their own Agent.md for a project how are they going to validate the auto-completed text?
> This strikes me as the ideal LLM first contribution/PR, a file explaining the projects standards and testing and structure.
Why should the project maintainers write such a file, when the info already exists within the README? It is duplicated work at best, and a definitive sign of the incapabilities of the agent to properly parse the project's contribution guidelines.
What motivation? Is it motivation to start Claude Code and let it run when you have no idea what’s going on? Is motivation the same as token spend?
Yes, the barrier should definitely be someone who knows how to code when submitting, well, code.
And since the training data seems to be very lacking, no amount of markdown would fix that.
> Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software.
Probably yes? QED submitting slop PRs is not helping. If "helping" is sticking it through an LLM, the developers can do that themselves with better insight and guidance? If you must help via an LLM, donate cash for tokens.
If you can't code, and cant donate cash/machine time, help by confirming issue reproductions, design, wikis, documentation, whatever.
> someone who is motivated to help with open-source software.
I don’t mean to pile on, but like… are you actually helping if you don’t understand the code you’re fixing, don’t understand the problem you’re addressing, and don’t understand the potential solution you’re submitting for that unknown problem? Or are you just making a lot of distracting noise so you can pat yourself on the back?
I think people need to be a bit more self-critical about what they’re actually up to, and who is actually benefiting from it. Generally, from comments like yours, the answers seem to be “self-aggrandizement” and “no one”, but people really don’t want to think they might be the bad guys.
I agree, and yet I think even with a well engineering agent harness, there are a lot of unknown unknowns out there.
I imagine the problem will persist if users continue to submit PRs that pass the harness without being able to validate for themselves that it actually works.
I've used Claude Max awhile now, and I usually only get to around 50% usage in a 4/5hr block (using medium effort). Yesterday, I switched from high -> medium effort using the /model command, but afterwards it still felt like I was burning through tokens at the high effort rate.
Haha, I think the experience is a bit different at the higher levels (between ranks 50-1000), but overall people are quite a bit nicer than those playing League of Legends or Dota.
Does anyone have any recommendations on best practice security methods? As others have said, it sounds like there may be an order of magnitude more vulnerabilities found / exploited, and I'm wondering if security such as 2FA and Password Managers will be enough? Should people be getting on board with other protections such as security keys?
I think the key part here is the bootstrapping phase. You may not use a specific English word every week, but maybe you use it every 2-3 months. SRS is great for getting information to these different thresholds!
As someone who has used spaced repetition extensively I will just provide a few insights that might be helpful:
1. Decide on what's important. Just because you learn something doesn't mean that it should be logged to the system. I used to log a lot of minor details (like niche method signatures or command flags to the system). If you make cards for every detail like this then you will be trapped reviewing 100s of cards daily that you likely never use.
2. For the cards you deem are important, make sure you understand the concept. This often means making 2-5 cards for the concept that test your understanding from different angles (definition, pros, cons, how would I explain this to someone else, etc...). This helps to cement the concept at a foundational level.
3. Try to move from the existing flashcards to 2nd order flashcards or pure application after the first couple reviews. So your foundational cards are now set to review in 6 months or 1 year. At this timescale if you prioritized what was important and made sure that you understood the foundational concepts, then usually simply doing things related to the concepts will be the reviews (and sorry to say but if in 1 year you get a card related to what you are doing, but never used, chances are it probably wasn't that important). In addition to doing, you can also create 2nd order flashcards (which might compare 2 concepts). These types of cards test the foundational knowledge indirectly, and are helpful for higher order thinking.
In conclusion, I think spaced repetition is a very effective tool for efficient learning (especially in the first 60 days or so after learning something). I think the major pitfall is not prioritizing what cards get made and being stuck in review hell.
I think a lot of it is: feels good to think that you're smarter than most than to acknowledge the reality that you're probably not.
My intuition is that this type of thinking is becoming more and more entrenched in American culture. I think by the time the culture as a whole wakes up, things will be significantly worse and many of the people who tried to prevent it will have been pushed out or willfully left.
Exactly. I realized alcohol was something that caused me to act in ways that got in the way of my life goals (nothing crazy, just things like I sleep worse after alcohol, I feel worse after alcohol, and I forwent activities I wanted to do).
After that realization I stopped drinking and have never felt like it again.
What has been most reliable for me has been instead of going from bad habit A to good habit Z, I just replace A with the easiest alternative that is less bad than A.
Obviously, this means going from A to Z can take years instead of weeks. Though, from my own personal experience and from what I see of others, trying to go too quickly from A to Z just results in whiplash and irractic behavior--where I have seen it work is when there is an existential crisis demanding that the behavior change.
