diff --git a/docs/v1/experiments/findings/measurement-defects.md b/docs/v1/experiments/findings/measurement-defects.md index a9eec13..c038ea8 100644 --- a/docs/v1/experiments/findings/measurement-defects.md +++ b/docs/v1/experiments/findings/measurement-defects.md @@ -1,7 +1,9 @@ # Defects in my own measurement -Recorded because the pattern is the point: **four of these, all the same shape — -searching by name or scope instead of by the operation itself.** Each was caught +Recorded because the pattern is the point: **five of these, and every one is the same shape — +reading a proxy instead of the thing.** A file instead of the operation, a +variable name instead of the shape, a scope instead of the whole, a pipe's exit +instead of the program's, a line count instead of the object identity. Each was caught by running something, never by reading. ### 1. Scoped the search to one file @@ -32,6 +34,25 @@ timeout 10 /tmp/leakrun 2>&1 | head -2; echo "exit=$?" # reports head's exit Reported `exit=0` for a program that was returning **139 (SIGSEGV)**. I nearly recorded a segfault as a clean run. +### 5. Read a count that was not counting + +Comparing the three promoted branches: + +```bash +for pair in "dev stage" ...; do set -- $pair + n=$(git diff --stat origin/$1 origin/$2 | wc -l) # git errored to STDERR + ... # wc counted empty STDOUT +``` + +`git diff` failed on a malformed revision, wrote its error to stderr, and `wc -l` +counted zero lines of stdout. Three confident `IDENTICAL` results, all +meaningless. **Had the trees actually differed, I would have reported the +promotion clean.** + +Redone correctly, the three trees share one hash — `2acd9374` — which is the +check that should have been run first: not "how many files differ" but "is the +tree object the same object". + ### And one that was not a measurement defect but a method defect One cycle was run **without committing predictions first** — see