Re: Suggestion: One QC, one patch file

"Serge Dosyukov (Dragon Soft)" wrote:
>
> About QA - with ALM and SDO? one click - called automated regression testing
> procedures.

Not quite so simple. Say there are 8 individual patches out there. They can be
applied or not applied in whatever order you want (under Max's suggestion you
could apply 1, 4 and 7 and ignore the rest). Even though they SHOULD be
independent you have to go back an retest every combination of patch results
that might result. This will quickly get into the millions of combinations.
The current process there is only one end result to test by QA and Borland fully
controls the patching order. You can't apply patch 1 and 3, you must apply
patch 1, 2 and 3.

Even with automated test suites there is time involved in running each
combination (you have to run all combinations in case one patch inadvertently
has a side effect on another or that one patch accidentally needs another to
actually work right). Lets say that that the regression test suite takes 1 hour
to run (a ridiculously low estimate). At just 16 individual patches as
described you have 64K worth of combinations to test (2666 man days of computer
time). At 32 it is 4 billion (490293 man years worth of computer time). There
just isn't enough time to do this type of testing.

Without fully testing each of the combinations you potentially are opening the
door for a much less stable product and also a much harder product to track down
what is going wrong because you have to find out exactly what combinations of
these singleton patches have been applied.

Now if you follow your suggestion of one a month and you add on the condition
that you have to apply them in order that is more doable, but it used to take 3
weeks to do a full regression test of Delphi (this was back in the D5 days). QA
also needs to be working on testing the features in the next release and other
things. Even one a month isn't practically feasible. One a quarter is probably
the time frame that is practible.

--
Jeff Overcash (TeamB) On waves of silver I dreamed of gold
(Please do not email 'Till I lost the peace that dreaming gives
me directly unless I dreamed of the moment of my own death
asked. Thank You) That no one ever dreams and lives (Marillion)
.

Relevant Pages