Search for a command to run...
Will Lord Windy discover he cannot pay attention?
No comments yet. Be the first to comment.
I am just checking to see if this works for my sub domain!
Lord Windy's Journey
2 posts
To learn what Spec Driven Development actually means, and whether I can do it at all.
No really. I've heard SDD spoken about for ages. At the Brisbane AWS Community Day (where I came first in the Game Day challenge, btw ;)) the whole event was about Kiro and Bedrock. Kiro was the first time I heard about this new way of using LLMs, but they announced it far too early, because by the time I would have been able to use it I had forgotten about it entirely. Still, the promise that by writing a really good product requirements document I will get a faultless program is something to strive for.
My Gameday Champion award and prize.
I want a dashboard for my personal projects. I do not want to have to think too hard about this dashboard, and I want to develop it with this new thing I've heard called Spec Driven Development. Traditionally, when I make something, it will be a CLI tool. That works really well for some projects, and AIs are able to interface with them easily when I provide JSON input and output. A humble human like me finds it hard to use sometimes, and worse, CLI tools can be out of sight and out of mind, and I will completely forget about them when I move on to the next new thing. I cannot tell you how many times I've made three-quarters of something but, because it isn't in my peripheral vision, I forget it. Sometimes I think I am a human T. rex: unless it is moving, I forget.
I have some tools currently called Samtasker and Queensland. Samtasker is my Jira equivalent, while Queensland is my orchestration engine. I'll explain them properly in a future post, but essentially I will create Samtasker tickets, give them a dependency chain, and then Queensland will implement each one and merge it in. It has frankly been transformational for me at work. Now I want to build a dashboard to view the tickets and set Queensland off to complete work for me. Observability in general is not amazing, requiring that I go into worktrees to view what is happening.
I built Samtasker and Queensland via small sets of TRDs (technical requirements documents) over time, and they required quite a bit of human review. My goal is to try and learn how to do L4/L5 Agentic Coding — the kind where there is no human review of code. I want to evaluate whether this is a way forward for me personally to develop harder and faster. I've come to peace with the fact that I personally struggle to complete personal projects when no one is paying me for it. Prior to this year I don't think I could say that I ever made anything complete, but since about October of 2025 I have slowly been becoming very productive. First in writing out function stubs and getting AI to complete them, now to nearly handing over complete control to the AI.
I don't want to experiment with this at work. Work itself would be extremely supportive of me trying, to be honest — after a major success rebuilding a pipeline in record time, I have a lot of leeway. That project was just me and two others. All three of us were given unlimited credits in AI to get the job done, and I believe we were only a single week late in completing it. It was transformative work, but there is no deadline forcing the pace at the moment, so I would be too concerned about quality and bloat to try this there. I wouldn't be the first to say that AI bloats code and often accidentally adds in bugs and landmines as a matter of course.
So instead I will experiment outside of work with subscriptions to MiniMax and GLM. That comes with upsides and downsides. The upside is that I get to do whatever I want with only my input being important; the downside is that I get to do whatever I want with only my input being important. I love being told that I am wrong — so hopefully you guys tell me in the comments as the project goes forward what I could be doing better!
The promise of L4/L5 Agentic coding is the ability to create a project from a spec sheet and have the dashboard work out of the box.
So my criteria for this project are:
A dashboard is created from a spec and will be created in one 'run' of the Queensland orchestration engine. It must let me:
View tickets
Change a ticket's status
Filter tickets by project
Update the contents of a ticket
Iteration on the spec once finalised is limited to three changes to the spec. This will be a measure of how AI and I work together to create a complete spec. If we are finding missing sections as we go, then it is an indicator of either my lack of attention or how hard it is to actually make a foolproof spec.
If the spec worked once to generate a complete product, it will work a second time and pass all the same tests. There should be no luck involved when it comes to Spec Driven Development.
Can be completed in one combined five-hour block run-through of my MiniMax and GLM Max subscriptions. As a sub-criterion, the ticket-building aspect is able to accurately determine the 'power' of the LLM required for the ticket, so as not to let me run out of tokens on the stronger GLM plan.
AI generated section. The wording of criteria 5 and 6 and the appendix below were written by Claude. I've kept them because I couldn't have picked the tooling myself without a few hours of reading, but the numbers are mine to defend.
Bloat. Four budgets, all declared in the spec before the first run and all checked by the harness rather than by me reading the diff:
No function above a cyclomatic complexity of 15.
Under 4,000 lines of non-test, non-generated source.
No more than 20 direct dependencies, and zero unused ones.
At least 85% line coverage from the verification suite alone. Code the hidden tests never execute is code I did not ask for, and that is the definition of bloat I actually care about.
I have no idea whether these numbers are right. I am going to run the tools over Samtasker first and see what my own hand-written code scores, because "not worse than what I write by hand" is a bar I can defend and a number off the internet is not. If the first run comes in at triple the ceiling, I will report that rather than quietly raising the bar — being wrong about what a spec costs is a more interesting result than passing a budget I invented after the fact.
The bloat numbers hold across runs. Two runs from the frozen spec must land within 25% of each other on all four measures above. This is criterion 3 with teeth: if run one is 3,000 lines and run two is 9,000, both passing the suite, then "there is no luck involved" is dead regardless of the pass/fail. Passing twice is not the same as being reproducible, and without this I have no way to say two runs produced "the same thing" other than squinting at them.
Frankly, Queensland is not ready for my overarching goal of the dashboard, and it will not be part of the spec at all. Queensland will need to become a server, output plans in a manner that can be rendered, be remotely callable, and frankly have a lot better observability. If it were not for the fact that the dashboard gives me a much nicer view of Samtasker and would be extremely useful in itself, this would be a spec of Queensland instead.
We will not attempt to code anything more than is necessary for my experiment. I am going to hide the database, Samtasker and the dashboard behind a VPC, and not attempt to overcomplicate the problem with auth, users and monetisation.
I actually don't really know, and I don't think anyone has a good answer. I was introduced to it by the daily.dev agentic AI hub a few days ago. I had heard of Spec Driven Development but didn't pay much attention to it, because it was largely what I was doing. My small branching out shows that the idea could have come from The Five Levels of AI Coding, but I am not sure.
My definition is as such: L4/L5 agentic development limits the human review to tests and specification. If you are looking at the generated code, you are not doing it right. You basically place yourself into the role of a product manager. You define the stories, the success criteria and what should be tested — along with a special hidden test suite that the AI is not able to view. That test suite is your verification and your proof that your spec was completed successfully and that the AI was not just telling you it got it right.
I am stealing the gist of this from Addy Osmani's How to write a good spec for AI agents and from Justin McCarthy's StrongDM Software Factory.
My understanding is that a spec should be a tight, focused explanation of what you are trying to build. It is also a living document and should be updated on any change you complete. You want to avoid letting AIs make decisions wherever and whenever possible. This is mostly how I make things anyway; what does interest me is the change to Test-Driven Development from the start. AI LOVES getting things right and will do anything it can to make tests say it passes. I've noticed in the past that tests I would consider 'sacrosanct' and 'correct' were being updated to get around pesky test failures.
I'm writing this after having just done the first pass of my dashboard spec. I am treating this as much as an experiment as anything. But what everyone has agreed on so far, from what I can see, is the need to create success criteria AND a testing layer that the AI cannot access. The reason for the testing layer is to specify what the spec should do and not allow the AI to game it. You will know your spec was correct IF it successfully completes the test suite. I am not sure whether failing the verification test suite automatically means a restart of the process after fixing the spec, or whether the implementation is allowed to be given feedback and fix it. If it is the former, it would count against the three changes in my success criteria; if it is the latter, I will need to adjust the success criteria first. That is an open question on my end that I hope to answer in the next blog post. I do not want to paint myself into a corner until I've seen what happens.
What it should include / what I am saying it needs to include:
About section
References/Glossary/Assumptions
Objectives
Features
Architecture & Dataflow (+ recommended tech stack)
Constraints
Step-by-step plan
Success Criteria + Testing Layer
And ultimately you should be able to take the spec and your tests and recreate the project near perfectly every time. In fact, I think it is expected that a major spec rewrite would mean a total project rewrite. That is very interesting IMO, because I know my company spends $$$k+ on tokens currently, and that is just for pair programming or feature development. If we rebuilt from a spec on major change, I could only see that dollar value exploding each month. A good experiment for the future would be to see how modularising the specs would go, so the blast radius of a change is small enough.
This is an active area of research. Or as close to research as a bunch of nerds trying to go fast can be. With AI going as fast as it is, I think everyone is just flying by the seat of their pants. Advice is pretty general in nature, and I cannot see any concrete examples of this working.
I am finding it hard to explain my approach at the moment, because I too am doing this. How I previously worked would be to create a 'TRD', as linked in the Samtasker v1 TRD. I specify what needs to be completed and how I want it to work, and hope for the best. If you were to compare the v1 to what exists, even before I added Postgres integration, you would see that it had shifted. That would mean I couldn't use the spec to drive a rewrite of Samtasker. This comes from the human review element: once the initial tickets were made, I personally did updates to the code to match my vision.
I think the big thing I have taken away from this is that testing must come first. If there can be no human review of the code, then our tests constitute our eagle-eye input. Given that this is a combination of website and server, I will create two things:
A contract-based integration test suite. Some kind of OpenAPI/Swagger type setup that will allow me to test each endpoint against test data that gets set up.
Some kind of website test suite. I will use Selenium or Playwright to do all the things I expect to be able to do with the dashboard.
I will achieve this using Bash or Python inside a Docker environment. This environment will be the complete setup for Samtasker, Queensland and the dashboard, amongst other things, and will be a spec within itself, I believe. So I am looking forward to that — I have always wanted this kind of testing environment, and it will be awful to fashion one.
The test suite must be hidden from the AI for two reasons. First, the suite is a verification, not a development tool. The spec should be good enough that it passes the verification without seeing it. The verification tests are a sanity test. Second, if the AI can see it, it can change it. My experience with LLMs in the past is that they will silently modify a test they are failing, because they care more about green ticks than about understanding what the test is. I've defensively created outside integration tests that catch Opus every time it tries to rewrite test data that we legally need to be able to reproduce.
Obviously this is a massive amount of work on top of the spec. I will achieve it using my old method of writing TRDs, similar to the ones I have linked in the past. But this environment will be added to in the future — likely to become a modular system that other programs within my ecosystem (Samtasker, Queensland, new items) will build upon.
Given that this is my first attempt, I will fail horribly initially. I will need at least three attempts and will spend most of my time fixing the test environment until it plays nicely with the spec.
I will work mostly on the testing environment and not the spec.
I will end up getting a testing environment even if the dashboard fails.
Even if I get the spec working once, I will not be able to get it going a second time from the same spec.
My next post will deal with the spec I make itself — regardless of the success or not of the dashboard. I have a first pass, so I will continue working on it and, in conjunction, a spec for a separate testing environment for Samtasker and Samdashboard — some kind of Docker thing that sits above it all and makes the specs testable.
Henry is my constant companion in everything I do.
Obviously I am using AI in my code, and you can assume anything within a code block is AI assisted at best and generated at worst.
For writing, I will use AI to assist me in proof-reading and I will allow it to tell me what I am missing, but for the most part I want to write in my own voice. I don't know about you, but AI sounds samey, and I enjoy reading and hearing people's own content. So if AI writes something, I will clearly label it and probably put it in a code block.
For referencing something, I will be using AI to insert the links in the right places. I hate referencing, but obviously it is important to do.
If I use images I will label where I found them or how they were made. Frankly, AI images are terrible and personally I find them an indicator of terrible content ahead.
AI generated section, as above.
clippy.toml in the project rootcognitive-complexity-threshold = 15
too-many-lines-threshold = 100
too-many-arguments-threshold = 5
type-complexity-threshold = 200
cognitive_complexity is a nursery/restriction lint, so it is off by default and has to be switched on. In Cargo.toml:
[lints.clippy]
cognitive_complexity = "warn"
too_many_lines = "warn"
too_many_arguments = "warn"
type_complexity = "warn"
Then in CI, cargo clippy --all-targets -- -D warnings turns those warnings into a failed build. Note that Clippy's own maintainers demoted this lint years ago on the grounds that it isn't accurate enough to be a measurement — it is fine as a tripwire, but the numbers below are the ones worth publishing.
Mozilla's rust-code-analysis gives per-function McCabe, cognitive, Halstead and SLOC/PLOC/LLOC as JSON:
rust-code-analysis-cli -m -p ./src -O json -o ./metrics
jq '[.. | .cyclomatic? | numbers] | max' ./metrics/*.json
cccc is the alternative if you want an exit code instead of a report, and it covers Rust and TypeScript in a single pass — useful when the project is a server plus a frontend.
rustup component add llvm-tools-preview
cargo llvm-cov nextest --all-features \
--fail-under-lines 85 \
--ignore-filename-regex '(tests?/|_generated)'
Run this with the verification suite only. Including the implementation's own tests measures the wrong thing entirely. (cargo-llvm-cov, nextest)
cargo machete # exit 1 if any dependency is unused
cargo tree --depth 1 | wc -l # direct dependency count
cargo deny check bans # duplicate versions of the same crate
cargo bloat --release --crates # binary size by crate
tokei src/ --exclude '*_generated*'
(cargo-machete, cargo-deny, cargo-bloat, tokei)
Rust inflates cyclomatic complexity structurally — every match arm and every branching Result is a path, so idiomatic Rust scores worse than the equivalent Python. Calibrate the threshold against existing hand-written code rather than against published industry figures.
Exclude generated code aggressively. sqlx macros, serde derives and any OpenAPI codegen will dominate a line count and tell you nothing about whether the agent bloated anything.