Who guides the work

FindingCustomer-facing features were 19% of 2,386 app and infra change sets; the weekly share peaked at 32% in the week of 27th of July and ran 10%–18% in September, and the fall from 22% in July to 15% survives re-weighting to July's repo mix (9%) and workflow mix (7%).

QuestionHow did the weekly mix of customer-facing features and structural work change by repository and model, and does the September decline in feature share remain after accounting for repository and workflow mix?

Sourcedetectors.cs_sets; detectors.cs_units; detectors.cs_worktype; git.commits (trailer_model); pr_commits.pr_commits (subjects); plans.plan_items, plans.goals (workflow).

FindingOf 1,162 change sets merged since 25th of August, 82% trace to a decision of mine, but only 40% had a work item and 30% one written before the work began.

QuestionWithin one common observation window, what share of shipped change sets had an identifiable owner decision, what share had durable written intent before implementation and how did both shares change over time?

Sourcedetectors.cs_sets; plans.plan_items; plans.goals; harness.sessions (pr_links, user_turns); pr_commits.pr_commits (first commit day); links.set_item_links.

FindingA third of the work was discovered while building (318 of 1,022 items, 31%, against 44% filed by the roadmap sync) and 85% of it was completed, yet open work grew from 30 items to 171 by the end of the study, with closures matching intake in only 6 of 11 weeks.

QuestionHow much work did the factory discover while building, how often was discovered work completed and did intake and closure keep pace well enough to prevent the open backlog from growing?

Sourceplans.plan_items; plans.item_edges (discovered_from); detectors.found_work; github.prs.

FindingA goal held a median 3 work items and finished in a median 2 days (80th percentile 7 days); 20 of 123 goals (16%) took on work after they started; and from 18th of September a merged pull request carried a mean 2.69 shipped items, up from 1.02, a calendar split rather than a causal estimate.

QuestionHow did goals change the relationship among work items, change sets and pull requests, and what do the distributions of goal size, duration and scope growth show beyond the medians?

Sourceplans.plan_items; plans.goals; plans.item_logs; github.prs; git.commits.

FindingOf 1,554 pull requests merged in 2026, 99% were reviewed, but a person reviewed 7 (0.5%); one-way-door changes had a person involved more often (25% against 3%), yet independent human review reached 2 of 326 against 5 of 1,228.

QuestionWho actually reviewed merged changes, how often was the reviewer independent of the builder and did consequential changes receive materially different human or independent review?

Sourcegithub.prs; github.pr_reviews; harness.prompts (agent commands); git.commit_files (one-way-door paths).

Finding in the chapterThat sounds like an unqualified success until it is placed beside the backlog. Open work grew from 30 items to 171 at the end of the study. Closures kept pace with newly opened work in only six of the eleven weeks measured.

Method (backlog-trend)Count items created and items ended (shipped, delivered, dropped) each week since the .plans store began on 23rd of July, and the items still open at each week's end.

Finding in the chapterThe commit "budget" attached to a goal (an upfront allowance of how many commits the goal should take, based on its tasks) exposed a related mistake. Sixty-six of the 76 goals with a commit budget used more commits than their nominal budget, with a median of 1.7 times the budget, yet only four exceeded their final budget_max. The checkpoint may have been useful, but commits were the wrong unit. A budget only means something in the same unit as the plan. If the plan counts features or change sets, the budget should count features or change sets, not the commits used to save intermediate states.

Method (goal-budgets)Each goal records budget (commits expected) and budget_max (the checkpoint). Compare with the original commits in its merged goal PR.

Finding in the chapterPermission prompts are a poor substitute for this design. An agent in this study stopped to ask roughly three times per working hour, while the owner refused 0.75 tool calls per hour. Neither rate fell during the 38 days measured, although incomplete logging means those figures should be treated cautiously. If an agent repeatedly presents a person with low-context questions, the person will approve most of them. Safety then depends on attention at precisely the moment the process has trained attention away.

Method (agent-stops-per-hour)AskUserQuestion calls and turned-down tool calls per working hour, by week from 25th of August.

Finding in the chapterTo understand the autonomous end of how much I can trust the factory, during this study, I deliberately leaned toward the agents’ recommendations. I accepted the recommended option in 80 percent of the 353 asks where a recommendation could be identified. I approved most plans unless I saw a clear problem. I allowed models to identify improvements and usually told them to proceed. Corrections and redirects fell from 0.54 per change set before July to 0.27 from July through September. Those figures do not show that the agents were right. They describe the posture of the experiment: I spent more of my time on what to build and why, while the system carried more of the verification.

Method (recommendation-uptake)For each one-question ask with a recommended option, compare the owner's answer with the recommendation.

Finding in the chapterSecurity offers a more specific example. Of 176 security change sets, 39 originated in a scanner or alert, 31 in audits, four in review and one with the owner. For 101 (57 percent) there was no record of what found the problem. Review write-ups named a security weakness in only five of the 367 pull requests whose reviews contained more than “approved.” A security label appeared more often, but it usually described the subject of the work rather than a flaw discovered during review.

Method (security-finders)Give each security change set a finder from its work item (origin, discovered_from), a security review finding on its PR, or an advisory in its label.

Method (review-security-flags)Label every review body with content (Copilot, agent self-review, the judge's change requests) for a security weakness in the PR's own change, and compare with the existing security-topic tags.

Finding in the chapterHowever, finding a vulnerability doesn't establish its present consequence. A scanner may identify a flaw that is unreachable; an audit may identify one that already shipped. Whether an attacker can use it today may depend on credentials, network exposure and running configuration outside the repository. None of the 66 recorded infrastructure risks had a live check represented in the data. Perhaps evidence needs to be collected continuously, both to avoid missing the risks and to catch potential regressions.

Method (infra-security-risks)Label every work item for an infrastructure risk (credentials, boot secrets, exposure, revocation) and follow each to fixed or open.

Finding in the chapterOnce a ready item begins, the build is usually short. A work item shipped in a median of one day, with 66 percent shipping within a day. The slow fifth became slower, however: the 80th percentile rose from 1.2 days in July to three days in September.

Method (item-lead-time)Days from an item's first log line to the log line that ships it, grouped by the week it shipped.

Finding in the chapterThe main delay was not always work in progress. For pull requests covered by session logs, no session was open during 55 percent of their open time. Agents were working, or running a tool or subagent, for 28 percent; I was reading or away for 7 percent and being waited on for 6 percent; usage limits took 4 percent. Seventy-six percent of pull requests used one session, but pauses between sessions lasted a median of 8.9 hours; 32 of 123 such pauses followed a usage limit. Perhaps I was asleep. I'd like to blame token limits, but they explain only about a quarter of those pauses.

Method (pr-time-breakdown)Minute by minute inside the open window of each PR opened and merged between 25th of August and 1st of October (655 PRs with a linked session, 1,512 open hours), what the sessions working on it were doing, and whether CI was running. Each gap between logged turns takes its state from the session-clock classifier (Chapter 6): a gap with a tool or subagent still running counts as agent working, not idle, and a gap ending in a typed prompt is waiting on me when the agent's last message asked a question, otherwise reading or away. Shares of all PR minutes: no session open 55%, agent working 28%, reading or away 7%, waiting on me 6%, usage limit 4%, CI running with no session activity under 1%. An earlier rule counted every long gap without an ask or a limit as idle (19%) and left usage limits and CI (7%) out of the split.