BDS needs more bugs fixed

I am reposting this on the recommendation of Chris Burrows. It was originally posted in the "Stop the Negativism" thread, but he felt it probably got lost in the flurry of traffic there.

This post was my reply to several other posts responding to my original reply to Nick Hodges regarding product quality. My comments are below.

You, and others at Borland, continually preach about the importance
of quality, as above. What we are not seeing is any demonstrated
improvement in quality.

[snip...]

Until I see some concrete demonstration of your efforts to fix the
quality issues in BDS 2006 I won't even consider buying another
version. I suspect there are a more than a few others out there who
feel the same way.

Several people replied that there had already been a couple of updates and nine hotfixes released, and that this should be a suitable demonstration of there intent to attend to BDS 2006 quality issues.

The following is a repost of my reply to them (slightly edited to correct typos).

---------------------

This is going to be a long post. It needs to be to detail some of the issues that others are glossing over with "they have released two updates and nine hotfixes". Yes they have fixed a few things, but there is a very long way to go.

Lets start with the last update, Update 2.

When I open the file readme_upd2.htm included with that update, and scroll to the section Resolved Defects, I am told to go to the site http://bdn.borland.com/article/0,1410,33463,00.html for a list of resolved defects. So I dutifully click on that link to see the list of resolved defects. What do I get, another page with almost identical content. In fact it contains the same Resolved Defects section and points at the same web page. So apparently there is no actual list of resolved defects, at least I can't get a look at it. :-( It then suggests going to QC and checking the resolution of your bug reports to see if they have bee fixed. So far, none of the reports I have been interested in have ever been fixed.

But, lets continue. Near the top of this page they have a section "Higher Performance and Better Quality" that says "Over 500 bug reports tracked by our internal system have been fixed in this release. Of the reports submitted by the community into QualityCentral, over 150 have been addressed in this update." That sounds promising, so I go to QC to see what was fixed. I run a query to return all the issues resolved in build 10.0.2288.42451 (i.e. BDS 2006 Update 2). I get 199 issues, 33 users reports to QC and 166 pulled from Borland's internal tracking system. This total is significantly less than the 650 fixes claimed on the web site, but lets continue. I started scanning the user reported issues and discovered that most of them have a resolution other than Fixed. The resolutions of the 33 user reports are summarized below:

13 Duplicate
8 Can't Reproduce
3 Test Case Error
1 Need More Info
1 As Designed
1 Won't Do

So all but 6 the reports were closed without any changes to the code. The remaining 6 were marked as Fixed. So lets look at those six in more detail to see what was fixed.

QC 32789 was reported as a bug in BCB 6. It was reproduced in BCB 6, but since the problem didn't exist in BDS 2006 it was marked as fixed. They didn't change anything in the update, they just closed the report because it was fixed in that build. Furthermore, the user who reported the problem was still using BCB 6, and he didn't get an update or hotfix, so this issue is not resolved for him unless he pays for an update to BDS 2006 and installs update 2.

QQ 31990 requested a change to the CPU view; "The CPU view should become two new debug windows. One window should consist of the disassembly view, registers view and stack. The other should consist of the memory pane. Multiple memory panes should be allowed." This was also marked as Fixed, but I haven't seen any sign of this change in BDS 2006.

QC 30402 reported "Object Inspector dropdown list takes ages to be populated on forms having a lot of objects". In the comments The problem is reproduced, but the severity is downgraded to "infrequently encountered" since the form has lots of objects (just like the original description said). There is nothing else in the comments to indicate if it was actually fixed or not. I'll give Borland the benefit of the doubt and assume this was corrected. Hooray, one infrequently encountered bug fixed (maybe). Assuming it was, it is a fix for a cosmetic issue only.

QC 29214 reports "Tooltip evaluation access violation". It claims to be fixed in the same build that it was reported in. I can only sympathize with the reporter when he states "Oops. I didn't see that it is fixed in the same build that I found the problem. Where does that leave me, I wonder?" It turns out this one really was fixed by hotfix 8 below.

QC 28548 is similar to QC 32789 in that it was reported in D 2005, but since it was already fixed in BDS 2006 it was marked as fixed. Again no change to the code. Furthermore, the original poster found the problem and a fix for D 2005 and posted that, but I doubt it was offered as a patch to anyone with D 2005. Borland wants them to buy an upgrade to BDS 2006 to get the fix if needed.

QC 28053 was again reported in BCB 6 and marked fixed in BDS 2006 update 2 because the new compiler generated an error message instead of an assertion failure. This is of no use to the original poster who needs to support BCB 6 for another two years. He will be forced to buy an upgrade to BDS 2006 so he can get the required error messages to fix the code.

So of the 33 user reported bugs marked Fixed possibly one actually involved changes to the code.

I did not go through the 166 reports pulled from the internal tracking system in detail. It looks like the majority of them are related to .NET and/or ECO. There are no comments or other useful information with all of the pulled reports, and they don't say what was changed to fix the problems. I really don't know what, if anything, that I might care about, was fixed. I'm sure there were some real issues that are generally relevant that were fixed, I just don't know how many.

The Resolved Defects section of the Update 2 web site is followed by a section of Known Issues.

Simon is going to love the first issue. :-) It is a General change to the operation of WideChar (and a hint at a similar impending change to AnsiChar) that may break existing Win32 code. This change was made so the new behavior is consistent with .NET. It is obvious from this alone that .NET behavior is more important to Borland than breaking its customers existing Win32 code.

On to Hotfixes.

Hotfix 1 was important to C++ users (like me). As the readme says, "This affected the Update 2 release of the C++ RTL only." This is a fix for the kind of thing that should never be released in the first place.

Hotfix 2 was an update for Caliber users. It did nothing for me or, I suspect, the vast majority of users.

Hotfix 3 was a fix for a bug introduced by Update 2. Again from the readme, "This hotfix contains a fix for customers who were unable to run the BDS2006 IDE after installing Update 2." It did not provide any benefit to me or, again I suspect, the vast majority of users.

Hotfix 4 was a fix for those using international characters in the HTML editor. Again I saw no benefit, and I suspect most users didn't see any benefit from this.

Hotfix 5 fixed a performance issue in the IDE. From the readme, "This hotfix addresses the performance issue described in QC 22372. When the product is configured with a large number of component packages, there is a significant delay when switching to the form designer view." So this speed up may have helped some users (not me, I don't have a large number of component packages), it corrected what is principally a cosmetic issue (i.e. nothing crashed, hung, or performed incorrectly).

Hotfix 6 fixed another minor issue. From the readme, "Accented characters or far east characters in source code cause errors in C++ rename refactoring." Again, I feel this is a very minor fix since almost no one uses refactoring in C++, let alone with extended characters in the variable or function names.

Hotfix 7 is another fix for C++ users like me. From the read me, "This hotfix contains a fix for C++ projects that include Delphi units with 'class virtual' methods. In Update 2 the generated HPP file may not compile, specially if the methods are used as a property setter or getter." I suspect the C++ users who ran into this know who they are and are thankful they can now use the problematic Delphi units. This was reported in 2006-04 and a fix wasn't available until 2006-09, five months later.

Hotfix 8 is the fix for the problem reported an QC 29214 above. "This hotfix addresses the evaluator issue described in QC 29214. When trying to evaluate a dataset.fieldbyname('fieldname').as... expression, an access violation is returned rather than the expected value." Again this was reported in 2006-05 and the fix was released with the hotfix rollup in 2006-09, four months later.

Hotfix 9 is a fix for a memory leak issue. From the read me, "This hotfix addresses the memory issue described in QC 22481. Memory use climbs steadily when switching between editor tabs." This problem was reported in 2005-12 and finally fixed in the hotfix rollup nine months later.

I may have to soften some of my earlier statements about a lack of bug fixes given the fact that last 3 hotfixes actually address issues relevant to a large percentage of users. :-) Still it took an inordinate amount of time to get these issues resolved. The earlier fixes were only relevant to a small number of users (who I suspect are members of that "other" group of large users who are not active in QC or these newsgroups but have Nicks ear by other means).

So it seems that update 2 and the nine hotfixes addressed 4 real customer reported problems and one issue that should never have been released, as well as some number of internally detected bugs (mostly in the .NET and ECO areas). This is all since BDS was released in 2005-11. That amounts to one user reported bug fixed every couple of months or so, or more likely one user reported bug fixed in the first 9 months and three in the last couple of months prior to the release of the hotfix rollup.

There are still an awful lot of outstanding bug reports that need to be addressed.

For example (one of my pet peeves), Borland has an obvious bug in the editor that triggers an assertion (see QC 33917). It has existed in all versions since D 8 (i.e. D 8, D 2005, BDS 2006), and it has been reported by many different users in the newsgroups. It is being dismissed as unreproducible, and therefore won't be worked on, in QC. These people have no reason to make this stuff up. This is an assertion that gives the source file and line number where the code fails. Fix it.

The fact they are now releasing hotfixes may be a good sign. To be honest, I never worried much about them when using BCB 6, I plugged away until I noticed an update came out, then installed it. I never really noticed any change after installing one either. Its only been since my "upgrade" to BDS 2006 that I have been messing with QC, trying to report bugs in order to get them fixed and patches released. It hasn't been a pleasant experience. I also started monitoring the newsgroups for reports of problems similar to mine, and noticed a lot of other people reporting pretty basic usability problems as well. It seems to me that the last few revisions along the way to BDS 2006 have had similar problems, and they are not being resolved.

The help system is a good example. It's broken. I don't think anyone disputes that. It has been broken for a very long time. And I noticed today (in the newsgroups) that Borland is only now looking to hire a technical writer to address this. So it unlikely that this fundamental problem will be fixed anytime soon. In fact I'd be surprised to see it by the release after Highlander. Also, I doubt the new, usable, help will be offered as an update to existing BDS 2006 (or earlier) users when it is released. We will get the same old mantra, "you have to upgrade to get those fixes".

Finally, to make matters worse the focus of development at Borland seems to be adding new features (i.e. .NET) rather than fixing the problems with the existing product. You really can't ride two horses at the same time.

I still feel that a lot more effort needs to go into fixing bugs and releasing those fixes. Given the number of outstanding bugs and the number of developers at Borland, I would expect to see fixes being released on a weekly basis (if not several per week) until the number of bug is reduced substantially. That would demonstrate that you are taking the quality issues in BDS 2006 seriously. You would have a better product base for the next release and customers would feel a lot more confident in upgrading.

Dennis Cote



P.S. In a later posting Nick Hodges said that there are a couple more hotfixes for BDS 2006 that will be released in the near term.

.