1500 lines of new tests written, 20 bugs found, codebase sans tests is 1700 lines (2200 with the aliases.) The first 800 lines of tests uncovered 6 bugs, the next 700 found 14. Apparently practice does make you better at writing tests.
I'm still around a thousand test lines short of done, so at the current rate of one bug per 50 lines of tests I might expect to find 20 more bugs. Which'd put prelude.ml's pre-testing bugs/kSLOC at 24 bugs per thousand lines of source code (bug every 42 lines.) Prelude.ml contains around 700 top-level function definitions, giving a ratio of one bug every 17.5 functions.
Furthermore, I might expect the current set of tests to contain 36 bugs (1500 * 24/1000) and the finished tests to contain 60 bugs. A bug in a test would be omitting some relevant test case or expecting wrong behaviour.
A few questions arise: How to find more defects with less effort? How to assert the correctness of the tests? How to write code with fewer defects?
 My counting methodology was: Misbehaviour is a bug. Changing code and tests to change expected behaviour counts as a bug. Solving multiple bugs with a single search&replace counts as one bug. Changing performance characteristics of a function (e.g. making a function tail recursive) counts as a bug.
art with code
- ► 2013 (26)
- ► 2011 (20)
- ► 2010 (94)
- Two things to make Ubuntu 8.10 less nerve-wrecking...
- Performance debugging with R
- Programming books that I've liked
- Revised quickcheck.ml syntax
- Low-boilerplate testing in OCaml
- QuickCheck in make_suite.rb
- PreString tests done
- PreArray testing stats
- Prelude.ml: first pass of array tests complete
- Multi-threaded qsort in C
- Test generation / measuring code
- And even more testing
- Current testing stats
- Random thought on automated testing
- Prelude.ml testing
- ▼ January (15)
- Built art installations, web sites, graphics libraries, web browsers, mobile apps, desktop apps, media player themes, many nutty prototypes, much bad code, much bad art.Have freelanced for Verizon, Google, Mozilla, Warner Bros, Sony Pictures, Yahoo!, Microsoft, Valve Software, TDK Electronics.Ex-Chrome Developer Relations.
- Filezoo - Minimalistic zoomable file manager
- Missile Fleet - A game written with Cake.js
- Gitbug - In-repo bug tracker for Git
- Prelude.ml - OCaml stdlib replacement with a Haskellish flavour
- Metadata - File metadata extraction tool and Ruby library
- Thumbnailer - File thumbnailing tool and Ruby library
- Random canvas demos