My amazon orgs leadership has been obsessed with spec driven development while individual engineers tell me the only use they have is to placate leadership. I'm tired
TDD is done in a tight loop (minutes) while coding. For every little micro-feature/fix, you write a test for the new behavior you want, implement the minimal ugly fix to get the test to pass, then rely on the tests so you don't regress as you clean up.
LLMs struggle with TDD. They want to generate a bunch of code and tests in large passes. You can instruct them to do red/green TDD, but the results aren't great.
SDD starts before implementation, and formalizes intent and high-level design. LLMs eat it up. The humans can easily reinvent the worst parts of waterfall if they're not careful.
I'm an engineer at Amazon - we use Kiro (our own harness) with Opus 4.6 underneath.
Most of my gripes are with the harness, CC is way better.
In terms of productivity I'm def 2-4X more productive at work, >10x more productive on my side business. I used to work overtime to deliver my features. Now I work 9-5 and am job hunting on the side while delivering relatively more features.
I think a lot of people are missing that AI is not just good for writing code. It's good for data analysis and all sorts of other tasks like debugging and deploying. I regularly use it to manage deployment loops (ex. make a code change and then deploy the changes to gamma and verify they work by making a sample request and verifying output from cloudwatch logs etc). I have built features in 2 weeks that would take me a month just because I'd have to learn some nitty technical details that I'd never use again in my life.
For data analysis I have an internal glue catalog, I can just tell it to query data and write a script that analyzes X for me.
AI and agents particularly have been a huge boon for me. I'm really scared about automation but also it doesn't make sense to me that SWE would be automated first before other careers since SWE itself is necessary to automate others. I think there are some fundamental limitations on LLMs (without understanding the details too much), but whatever level of intelligence we've currently unlocked is fundamentally going to change the world and is already changing how SWE looks.
I saw somewhere that you guys had All Hands where juniors were prohibited from pushing AI-assisted code due to some reliability thing going on? Was that just a hoax?
> Much of the coverage of the service incidents has focused on a weekly Amazon Stores operations meeting and a planned discussion of recent outages. Reviewing operational incidents is a routine part of these meetings, during which teams discuss root causes with the goal of continuing to improve reliability for customers.
This is something that's a part of every FAANG afaik. I know for a fact that there's no prohibition on pushing AI-assisted code. How would that even technically work? It'd basically mean banning Kiro/CC from the company.
> Only one of the incidents involved AI-assisted tooling, which related to an engineer following inaccurate advice that an AI tool inferred from an outdated internal wiki, and none involved AI-written code.
and this doesn't seem as "AI caused outage" as it was portrayed.
Not a hoax, saw it in the news. I'm not at Amazon but can confirm massive productivity gains. The issue is reviewing code. With output similar to a firehose of PR's we need to be more careful and mindful with PR's. Don't vibe code a massive PR and slap it on your coworkers and expect a review. The same PR etiquette exist today as it did years ago.
> I have built features in 2 weeks that would take me a month just because I'd have to learn some nitty technical details that I'd never use again in my life.
In the bucket of "really great things I love about AI", that would definitely be at the top. So often in my software engineering career I'd have to spend tons of time learning and understanding some new technology, some new language, some esoteric library, some cobbled-together build harness, etc., and I always found it pretty discouraging when I knew that I'd never have reason to use that tech outside the particular codebase I was working on at that time. And far from being rare, I found that working in a fairly large company that that was a pretty frequent occurrence. E.g. I'd look at a design doc or feature request and think to myself "oh, that's pretty easy and straightforward", only to go into the codebase and see the original developer/team decided on some extremely niche transaction handling library or whatever (or worse, homegrown with no tests...), and trying to figure out that esoteric tech turned into 85% of the actual work. AI doesn't reduce that to 0, but I've found it has been a huge boon to understanding new tech and especially for getting my dev environment and build set up well, much faster than I could do manually.
Of course, AI makes it a lot easier to generate exponentially more poorly architected slop, so not sure if in a year or two from now I'll just be ever more dependent on AI explaining to me the mountains of AI slop created in the first place.
It’s too bad, really. While it’s easy to get discouraged about such things, over the course of my career all that learning of “pointless” tech has made me a much better programmer, designer, architect, and troubleshooter. The only way you build intuition about systems is learning them deeply.
> make a code change and then deploy the changes to gamma and verify they work by making a sample request and verifying output from cloudwatch logs etc
This has been a godsend over the past week while deploying a couple services. One is a bridge between Linear and our Coder.com installation so folks can assign the work to an agent. Claude Code can do most of the work while I sleep since it has access to kubectl, Linear MCP, and Coder MCP. I no longer have to manually build, deploy, test, repeat. It just does it all for me!
How do you deal with a risk of LLM generating malicious code and then running it? I suspect it's a bit more difficult to set it up tailorer to your needs in a big corp.
Quite. On the face of it: possible career faux pas.
I own (with two other folk) my own little company and hire other people. I actively encourage my troops to have a bash but I suspect that a firm like AMZ would have differing views about what used to be called moonlighting. Mind you we only turnover a bit over £1M and that is loose change down the back of a sofa for AMZ ...
We built a messaging platform for exactly this use case and instruct claws to check in with each other or share context with each other at regular intervals.
Delivering a software solution is often tantamount to reading multiple codebases, getting some alignment with a team on a proposal and then writing the code and deploying. Most of which can be done by calling APIs (apart from aligning a team of humans)
No, that sounds like what consultants would do on an integration project, but not at all how software development works. Sure, you call APIs - after you write them. Which comes after you define what connections are needed between various layers of the system, which odds are you are also writing.
Saying that most software development is done by calling APIs is like saying that most driving of cars is done by moving your hands to the left and the right. Sure, that is an important piece of what is going on, but it really misses the bigger picture.
One can imagine a manager given an outcome to achieve or a director and then a team of agents carrying out the task. Perhaps the agents are adversarial to some extent so they get reasonable pushback on their decisions (ex one agent always sides on taking a long term approach, another agent wants to be scrappy, another agent is on a PIP and approves everything etc).
I've worked jobs where I'm paid 60-80k and where I'm paid 100k plus.
I can tell you for sure I cared much less about the job where I wasn't getting paid as much. I would never answer an after work call or go out of my way to follow up organizational or systemic issues. For me working overtime is part of the package when I get paid loads more.
> For me working overtime is part of the package when I get paid loads more.
I've tended to think this way, but it's all relative. Plenty of people who've never had a $50k job will treat the $100k job as the baseline/minimum, and wouldn't ever think of going the extra mile like you do. For you (and me?) - yeah, you're paying "a lot", I'll make sure I do a bit extra/more when needed. But if you've never had less, that $100k is your bottom, and you might only consider doing 'extra' for a $200k job.
Looking at how Nvidia share prices developed in the last 12 months should be reason alone to enter the AI accelerator business. And OpenAI does have software experts and a major component of Nvidia's success in AI is that they have a major advantage in software compared to their competitors (the hardware advantage isn't that large). Furthermore, there is Google's "We have no moat, and neither does OpenAI" memo: OpenAI definitely needs to strengthen its moat.
But of course, doing research with pytorch is not the same as developing driver code for some hardware bus or scheduling algorithms.
They're apparently not interested in building the chips themselves per the article, but rather 'getting' TSMC or another manufacturer to build them with the funds.
Which raises more questions than it answers. Why does OpenAI even need to be involved -- is this a common situation?
I would think the manufacturers would anticipate the huge impending demand for chips and raise the necessary investments to expand themselves.
Without any special knowledge I would assume that their motivation is two-fold. First remove the middleman that is potentially adding a huge margin on top of the silicon, second remove the extra fluff on the silicon that is not necessary for their specific use case, making the cost smaller.
This is really interesting to see these two posts. I can now imagine where AI tools actually inhibit innovation in many domains simply because they’re optimized for things that are already entrenched and new entrants won’t be in the training data. Further inhibiting adoption compared to existing things and thus further inhibiting enough growth to make it into model updates.
It is a healthy mindset to see this phenomenon as "interesting". I can get there when I dial up my mindfulness, but my default mode here is rather judgy; as in "please ppl! pick the better tool as evaluated over a 4+ hour timeframe (after you've got some muscle memory for the API) instead of a 15 minute evaluation".
Forgive me for ranting here, but have people forgotten how to bootstrap their own knowledge about a new library? Taking notes isn't hard. Making a personal cheat-sheet isn't hard. I say all this AND I use LLMs very frequently to help with technical work. But I'm mindful about the tradeoffs. I will not let the tool steer me down a path that isn't suitable.
I'm actually hopeful: there is an unexpected competitive advantage to people who are willing to embrace a little discomfort and take advantage of one's neuroplasticity.
> I can now imagine where AI tools actually inhibit innovation [...] new entrants won’t be in the training data
I still imagine the opposite impact... Welcome to no-moats-lang.io! So, you've created yet another new programming language over the holidays? You have a sandbox and LSP server up, and are wondering what to do next? Our open-source LLMs are easily tuned for your wonderful language! They will help you rapidly create excellent documentation, translators from related popular languages, do bulk translation of "batteries" so your soon-to-be-hordes of users can be quickly productive, and create both server and on-prem ChatOverflowPilotBots! Instant support for new language versions, and automatic code update! "LLM's are dynamite for barriers to entry!" - Some LLM Somewhere Probably.
Once upon a time, a tar file with a compiler was MVP for a language. But with little hope of broad adoption. And year by year, user minimum expectations have grown dauntingly - towards extensive infrastructure, docs, code, community. Now even FAMG struggle to support "Help me do common-thing in current-version?". Looking ahead, not only do LLMs seemingly help drop the cost of those current expectations to something a tiny team might manage, but also help drop some of the historical barriers to rapid broad adoption - "Waiting for the datascience and webdev books? ... Week after next."
We might finally be escaping decades of language evolution ecosystem dysfunction... just as programming might be moving on from them? :/
Because it’s like willfully choosing the more painful and difficult tool that occasionally stabs you in the hand, because you’re now used to being stabbed in the hand.
Continuing to choose it in the face of - in their own words - a better option, is a bit mind-boggling to me.
