art with code

2009-01-04

Current testing stats

1500 lines of new tests written, 20 bugs found, codebase sans tests is 1700 lines (2200 with the aliases.) The first 800 lines of tests uncovered 6 bugs, the next 700 found 14.[1] Apparently practice does make you better at writing tests.

I'm still around a thousand test lines short of done, so at the current rate of one bug per 50 lines of tests I might expect to find 20 more bugs. Which'd put prelude.ml's pre-testing bugs/kSLOC at 24 bugs per thousand lines of source code (bug every 42 lines.) Prelude.ml contains around 700 top-level function definitions, giving a ratio of one bug every 17.5 functions.

Furthermore, I might expect the current set of tests to contain 36 bugs (1500 * 24/1000) and the finished tests to contain 60 bugs. A bug in a test would be omitting some relevant test case or expecting wrong behaviour.

A few questions arise: How to find more defects with less effort? How to assert the correctness of the tests? How to write code with fewer defects?


[1] My counting methodology was: Misbehaviour is a bug. Changing code and tests to change expected behaviour counts as a bug. Solving multiple bugs with a single search&replace counts as one bug. Changing performance characteristics of a function (e.g. making a function tail recursive) counts as a bug.
Post a Comment

Blog Archive

About Me

My photo

Built art installations, web sites, graphics libraries, web browsers, mobile apps, desktop apps, media player themes, many nutty prototypes