Re: The curse of constant fields

On Sun, 14 Dec 2008, Arved Sandstrom wrote:

"Tom Anderson" <twic@xxxxxxxxxxxxxxx> wrote in message
news:Pine.LNX.4.64.0812141226390.27581@xxxxxxxxxxxxxxxxxx
On Sun, 14 Dec 2008, Arved Sandstrom wrote:

As the current official builder (all changes to the Subversion production branch, EAR construction etc) for a reasonably important J2EE app (let's just say that if I don't include what I should include, all motor vehicle administrative activities grind to a halt in a Canadian province until we deploy the previous working EAR) I am pretty fanatic about sanitizing everything prior to an official build.

After bad experiences with the reliability of IDEs in regards to builds (unless one is very careful) I no longer use them in any stage of an official build. All the SVN ops are command line (and a fresh checkout is involved at some stage), and I use an Ant build file for the EAR construction. No clean is required since I do a fresh checkout.

This sounds like excellent policy.

After the classes are built I run JUnit tests, and after the EAR is
built I deploy it locally using the server's admin console for some
sanity tests through the application web interface.

By hand? No automated testing of the complete app through its web interface?

The idea of using automated tests for the web GUI has been bruited, but in
our environment it would be unrealistic.

Interesting. Can i ask why?

Also, presumably, before you deploy to production, you deploy to a staging
or pre-production environment which replicates the production environment,
and where you do some more thorough (manual or automatic) testing, right?
Or rather, you have someone else - a QA or client team - do it?

We have a team of dedicated testers (they've come up through the ranks on
the business side and know exactly what should happen), and quite a few
others that do it part-time as required (again, business-side people who
know the business rules to a T). By the time a production build is to be
produced, a test build which is identical except for some configuration has
been deployed in a similar environment, and tested for weeks (or months).

It might be more accurate to say that I am quite concerned to make sure that the test builds are correct. Because a lot of time can be wasted if a tester reports that an error is still there, and then it's a question of is the error still there because the build is flawed, or because the developer who "fixed" it only fixed it for a different use case or in his own development environment.

Yes, i can see that if your testing process is as large-calibre as yours is, you want to make sure you're not wasting its time on avoidable mistakes.

I now always clean everything associated with an EAR build/deploy in
development, because I've seen partial cleans mess stuff up too often.
And in any case even with monster J2EE apps the clean takes very little
time. We have Hudson set up to do pretty much all of the above - fresh
checkout, compile, JUnit test suite, build - after each developer
commit...this is more for convenience than anything else.

That also sounds like an excellent idea. We have a nightly build (and
actually, we don't at the moment, because the thing we're working on takes
a lot of manual intervention to build, and we haven't invested the time in
automating it yet), but i'd love to have a build and test after every
checkin. With a klaxon and flashing light that goes off if it's broken.
Hell yes, a set of traffic lights - green if the current version in CVS is
good, amber if there's just been a checkin and it's currently undergoing
testing, and red if it's broken!

We pretty much have the klaxon and flashing lights. :-) If a build breaks then Hudson emails every developer, and the email tells you what the base event was (whose commit), and what broke (if JUnit tests then you follow the emailed link and drill down). Hudson itself is quite easy to configure, and jobs (say one for trunk, one for each feature branch etc) are also very easy to configure. What's nice too is that the Ant build.xml (if such you choose to use) that you point Hudson at is exactly the same thing that you'd use in your dev environment, or test or production.

That all sounds good. I have no idea what we're using for nightly builds - knowing the guy who's our semi-official buildmaster, a bunch of perl scripts. Not necessarily a bad thing, mind!

In fact, i fantasise about having a build and test running on a pre-commit hook, so that if you try to check in code that doesn't build and run, it gets rejected! [...] Obviously you'd also need a fairly fast set of unit tests - you'd need to flag anything slow as not to be run on checkin.

This is one thing we don't enforce. It has not been an issue to date.

With the exception of slow tests (slow by their nature), I haven't seen that large test suites take so long to run that a developer couldn't execute them on each commit. On the specific app I refer to we have probably close to 3000 JUnit tests, and it's on the order of a minute on my local box to run them all. They're typically not trivial tests either.

I've had mixed experiences. Mostly, the same as you - tests are fast. Three projects i've worked on have been quite different, though. Two are web apps where we do heavy testing through the web interface, so every test is making multiple roundtrips up and down the whole stack, from web server down to RDBMS via a zillion layers in between. The third is HtmlUnit, which has about 3000 tests (IIRC), and which take quite a while to run, because a lot of them involve running gigantic amounts of javascript (the point being to test HtmlUnit's javascript handling). All of those test suites ran in <5 minutes, though, which would still be doable on a checkin hook, just about. It might be kinder to only run a subset, though.

Indeed, we do run the complete test set before a commit - but it's up to the developers to remember to do it. Enforcing it by machine would be preferable!

Speaking of tests and Hudson, one handy other thing to hook in is test coverage, like Emma. It just gets added to the script that Hudson is provided...you end up with nice graphs of coverage at various levels. IMO this is indispensable (even in a TDD environrment) for staying on top of whether your tests are sufficiently blanketing the codebase.

Yes, we haven't been using any coverage tools. It'd be something i'd like to try. I'd really like to have ways of knowing whether our tests are any good. Quis custodiet ipsos custodes?

tom

--
KEEP CALM and CARRY ON
.

Relevant Pages

  • Re: The curse of constant fields
    ... EAR construction etc) for a reasonably important J2EE ... No automated testing of the complete app through its web ... Also, presumably, before you deploy to production, you deploy to a staging ... or pre-production environment which replicates the production environment, ...
    (comp.lang.java.programmer)
  • Re: why administrator refuse to give permission on PLUSTRACE
    ... and looking at the metrics created by the DBA. ... A developer rummaging around a production instance trying to ... that your test environment isn't just a joke. ...
    (comp.databases.oracle.server)
  • How to keep developers out of Production?
    ... With app roles an application has to know the role's password, ... the developer of the app has to know the password. ... but not in our certification or production ... different passwords in the three environments, ...
    (microsoft.public.sqlserver.security)
  • Re: VS.NET 2003 and VS2005 co-existence or changeover
    ... > still beta and no serious developer would put it on to his developing ... > production machine with the risk of re-formatting to remove beta software ... > configure your .NET app to use .NET1.0 features only, ...
    (microsoft.public.vsnet.general)
  • Two databases - wrong data being fetched in report
    ... I'm using two databases (developer and production). ... I have modifications to do to a couple of reports (these are working ... perfectly fine with the VB.NET app we have). ...
    (microsoft.public.vb.crystal)