Re: Delphi makes it to digg!
- From: "m. Th." <a@xxxxx>
- Date: Wed, 18 Jul 2007 10:33:49 +0300
Chris Pattinson (CodeGear) wrote:
Patrick Moloney wrote:
Chris Pattinson (CodeGear) wrote:
BDS2006 was developed using the old waterfall methodology, forChris, I assumed you were using some form of Agile for a while -
D2007, C++Builder and now Highlander we use a hybrid of agile.
although I sensed that bugs were getting fixed (those that did)
immediately before release. They seemed to sit in QC for 18-24 months
at least. It would be hard to create a new release (or two) every
year under a waterfall system.
We did use a hybrid of waterfall. Multiple integration builds provided
to QA to test and a continual improvement process with a tightly
integrated a test and develop cycle. We had several formal field tests
that would let us revisit product quality and gauge customer feedback.
Currently we aim for weekly builds to the field test, and smaller
increments between changes/test. There is a huge benefit to catching a
bug as soon as it's introduced - just knowing what changed, and
minimizing any dependancies on the change, makes a big difference.
Of course. Agreed 100% here. Very good news, Chris! Some things to note:
1. WHO are the 'field test'?
No, I don't mean here 'names', but from our small experience we found that the best 'field test' is the one who have users from all user levels:
- yourselves (for white box testing)
- the ones who knows a little from internals (gray box testing)
- the ones who can do only 'black box' testing
2. What are the DYNAMICS of the 'field test'?
It's well known that the same men (tend to) think in the same way, with the same virtues and the same weak points, thus making the same mistakes, tending to leave the same areas uncovered. Did you consider a rotation between areas and/or users? Sometimes, imho, a small guidance is necessary. For ex. someone will send you good reports from graphics area. You can temporary 'rotate' him a bit telling 'thank you very much, we are eager to know more from you in this area, but, because we really appreciate your work, can you give us some hints about how VCL is painting, in your opinion?'
3. Are you aware of http://www.satisfice.com/info_rst.shtml ?
The quality central system seems misunderstood - it is/was a vehicle
for the community to identify and promote issues into the internal bug
tracking system. This is done via sysops who mark a report 'needs
attention'. The 'Reported' ones need to be looked at by Sysops. QC does
not provide visibility into the internal bug tracking system though,
except for reports promoted into it directly. This also leads to cases
where reported issues that never made it to Open were fixed in a
release, but never refleced in QC.
....and this leads to the _impression_ that there isn't activity. In order to community to use QC, it needs to clearly know that they (the community) will have something to gain from there.
That being said - the QA team is now spending a few hours every week to
go through QC. We're pretty good at keeping up during field tests, it
gets trickier to deal with field test, internal AND the public backlog.
("Hire more people!" is often the reply to that, I usually think
"Promote more SysOps!")
Yes, but just an aside, why do you think about HR only? There are also other resources:
- time: When someone gets stuck somewhere, switch off from there. Leave it in the 'background thread' of his mind. It's much more productive, imho. (Of course, here is the danger of hyper-multitasking, isn't it?) He can ask/pass it to someone else. Also, it seems here that the 'field test' is done by QA team (CIIW). In order to minimize the pressure upon you, perhaps is better to give also the weekly builds to others (CodeGear tech. partners, selected individuals aso. - as it seems to fit to you).
- know-how: A very interesting thing here, is, from our little experience, a program who does Remote Desktop Connection (for the record, we use GoverLAN for that purpose) and the possibility to quick build-and-push. When a certain user encounters a problem, usually he cannot spot neither the exact _nature_ of the problem, neither the exact _cause_. So, entering in his screen and seeing what's there it was a truly life-savior for us. Also, when we cannot see the exact cause we put some spies in the dangerous area, do a quick build and push the resulted .exe to him, telling him what's happened and asking him to do it again, and if happens then call us and FREEZE, till we look at the results of our agents (and at his screen). We fixed many problems, both technical (bugs) and human (usability, UI misunderstandings aso.).
- technical support: QC enhancement (you speak very nice about this). Faster boxes for builds, automatic testing, *better* and *more reliable* web servers for pushing the builds to the field testers (hint, hint)
Regardless, now we're doing what we can since the volume of backlog is(perhaps) D) The reports weren't grouped in order to have a better fixing coverage. When a QA guy is working in an area, he must know (at least theoretically) the exact image of all the nasty reptiles which are lurking (or might be) in the swamp were he is.
huge. That backlog is large because:
A) Reports could not be reproduced but were never closed
B) A report was in an area no sysop was looking
C) The report was fixed in a release, but never promoted and never
retested
Several 'QC Heroes' put a huge effort into keeping QC clean. Sysop
stats are informative : http://qc.codegear.com/stats/sysop.aspx
David Dean did such a great job on C++ side of QC that when we had an
opening, we recruited him for a senior QA position. It's working out
great.
Yep, all my praises. Only to not became proud. ;-)
We're looking to address the QC backlog in several ways. The most major
step is centralizing both bug tracking systems into one (and right now,
we're persuing upgrading QC to be that one).
100% agreed. Approved. Accepted. Logical. Kudos. Bravo. Way to go! etc. etc.
Personally I feel the
community feedback and issues are the most valuable feedback we can get
- as long as we can ensure the reports are reproducible and contain the
information the team needs to fix a problem. The field tests seem to be
getting better recently.
And, fixing the QC reports is good PR anyways ;) It's the most visible
place for when we do fixes, the community can see just how busy we are.
Fixing hundreds of reports that only made the internal system really
means very little when QC has hundreds which apparently have no
activity.
Finally! For God sake, I hope to keep this trend and to fulfill it.
Your posts:
The good: nice!
The bad: rare!
;-)
my 2c,
--
m. th.
.
- Follow-Ups:
- Re: Delphi makes it to digg!
- From: Chris Pattinson (CodeGear)
- Re: Delphi makes it to digg!
- From: David Dean [CodeGear]
- Re: Delphi makes it to digg!
- References:
- Delphi makes it to digg!
- From: paladin
- Re: Delphi makes it to digg!
- From: Fons Neelen
- Re: Delphi makes it to digg!
- From: Patrick Moloney
- Re: Delphi makes it to digg!
- From: Nick Hodges (CodeGear)
- Re: Delphi makes it to digg!
- From: jana . vasseru
- Re: Delphi makes it to digg!
- From: Chris Pattinson (CodeGear)
- Re: Delphi makes it to digg!
- From: Patrick Moloney
- Re: Delphi makes it to digg!
- From: Chris Pattinson (CodeGear)
- Delphi makes it to digg!
- Prev by Date: Re: Here we go again -- problems
- Next by Date: Re: International Pricing Policy
- Previous by thread: Re: Delphi makes it to digg!
- Next by thread: Re: Delphi makes it to digg!
- Index(es):
Relevant Pages
|
