Two letters prompted this article. The first was to TNT, which though complementary did ask us to do more "Hi-Fi Choice" type group tests. The second was to another publication and berated a reviewer for praising a component whilst describing it as bright - the gist being that any serious hi-fi component should be neutral.

Now it might seem that these are unconnected but nothing could be further from the truth. Here I intend to explain why I don't do group tests and hopefully give readers some idea of where I am coming from when I review a component.

The idea of a group test is superficially attractive. Get in a group of "golden eared" reviewers and sit them down over a couple of days to listen to a series of components and rank them against each other - simple.
Lot's of magazines do this and readers seem to prefer this technique, not least because it gives the reviewers less opportunity to "hedge" and produces "best buys" so readers don't have to bother with auditions. It also brings in advertising from all the manufacturers involved. There is one snag - at best this type of review is pretty worthless, at worse hopelessly misleading.

Before "Hi-Fi Choice" fans cry "foul!" let me state here that I was an avid reader over the years and if you must read such tests then "Hi-Fi Choice" produces probably the best of them. So accepting that I randomly pick (top of the pile) one of my old back copies and find the "group test". This particular issue pits 14 different speakers against each other, ranging in price from 300 to 700 pounds.

A wonderful cross section of the middle market I hear you cry, but if you read the small print - the methodology, the cracks start to appear. Let's start putting the "nails in the coffin"...

Nail One - The listening panel comprised 5 experienced individuals from the hi-fi industry, nothing wrong with that, but why no professional musicians? The presentation of each speaker took 1/2 an hour and the test spread over two days. There were "a number of repeat presentations".
So that is at least seven hours of listening, if each speaker is presented twice that makes a least 14 hours. Plus the time taken to swap speakers - ten minutes minimum? Plus time changing records, talking, eating, having a pee, writing notes etc. I'd guess on 20 hours. Do you think that the panel worked two ten-hour-days? No - me neither...
Could you really relate the sound of a speaker meaningfully to one you heard briefly seven hours earlier with an intervening 12 presentations? No, I'd be totally confused and so would you.

Nail Two - The presentation consisted almost entirely of "modern" music, from Joni Mitchell to Massive Attack, though the Cambridge Singers put in an appearance with "There is Sweet Music" (sheesh!).
So no classical orchestra, no jazz - worrying at best, but understandable with only 30 minutes to play with. I'd also add that after three goes of the Cambridge Singers not only wouldn't I care what speaker I was listening to I'd be ready for the nut house, the poor panel had to go through it 20-30 times ARGHHHHH!

Nail Three - All speakers were played at the same volume, verified by measurement! We're not told what that volume was - background, easy listening or window rattling. I'm sure most of us play "Massive Attack" a hell of a lot louder than Van Morrison for example.
Many speakers are wonderful at low levels only to lose the plot as the volume rises and equally others need watts to "come alive". Ditto any amp used for the review will be effected by the speaker used. Some speakers may be efficient enough to draw little from the amps, others to drive them to the limit.
I imagine a Lowther horn driven by the Naims might draw a couple of watts, a big panel might make them clip at the same volume.

Here the issue of neutrality comes in...

Nail Four - Bare with me for a moment... Get up from your computer. Clap your hands. Now go to the bathroom and repeat the experiment, now the kitchen, now the living room, now outside. Each sounds very different doesn't it? Now tell me which is the "neutral" clap:) The tests were done in a larger-than-average room.
What that room is like I don't know and neither do you. I've heard speakers boom hopelessly in one room only to sound dry in another - the speaker/room interaction is critical.

Nail Five - The test was done using an all Naim CD/Amp system with an LP12 for vinyl. As an ex-Naim user and fan and member of the Naim forum I can state with certainty that some speakers will sing with this combination and others be aweful.

Nail Six - No experimentation was allowed re stands, cabling or positioning, thus throwing out everything you've ever been told about setting up speakers.

To be fair, the "head tester" then played with the speakers for a couple of weeks, but all the "best buys" and "recommended" speakers earned their "swing tags" during the panel's testing.

I won't bore you by going on, but I hope you understand why I have no time for group tests. Recently Steve Davey and I got together for a day and tried three pairs of speakers against each other. It took all day.
We didn't have time to fiddle with cables/stands etc and all it proved was that the pair I use myself, and which the system is optimised for, was hugely superior - big surprise. Remember also that speakers do sound very different from each other, using the same system for CD players or a bunch of amps is frankly ludicrous

So what do I do? Firstly I don't have the resources/space to do group tests even if I wanted to. Right now I have six pairs of speakers here and divorce is being threatened... When I get a component, for arguments sake a pair of speakers, the first thing I do is just wire them up and listen over a week.
I (unlike any panel) know my listening room intimately and so can take much of its effect into account, but by now the speakers will have run in and I'll have some idea of their characteristics. Now the fun starts... We're back to balance again. Every component from front-end to speakers, including all connections will alter the sound to a certain extent.
The only component that can be reliably identified as neutral is the pre-amp and then only if your power amp has a volume control as mine has so you can use it "direct". Thus I can say with some certainty that my Audion Pre-amp is essentially neutral. So given that every other component is not neutral but only balanced one way or other compared to an alternative, I begin to juggle things to produce the best possible sound in my own - unneutral - room.

An example- My main speakers are the IPL S3's. Putting Cabasse Sloops in their place produces a forced mid-forward sound with a detached bass and hard treble - horrible. To get the best from them I change the speaker cable from FFRR to Cabasses own stranded cable.
The interconnects from silver multicore to copper solid core. I change the Dynavector step-up for the warmer and more forgiving Audion step-up transformer. I reduce the damping on the SME IV and move the speakers a foot back towards the rear wall.
The result is not that the Sloops sound like the IPL's, but that they sound totally different but arguably as good! They also tend to draw me towards different parts of my record collection (1400 LP's), those which they suit better than the IPL's. The review then concentrates on the characteristics of the speakers with advice as to what they work with and what sort of music they prefer.
At this time I invite a friend and fellow audiophile over for a listen. He happens to have played flute with top international orchestras for the past 30 years and he gives me an insight into the correctness of the orchestral sound, something I have no knowledge of. To find the new balance takes a couple of weeks with fine tuning to follow.
This is the opposite of the "group test" approach. In a group test using my standard system the Cabasses would have "bombed". With care and time they could be a "Best Buy":) This is where dealer knowledge will come in because they should know what works with what.

The problem remains that readers like to know a "winner", and to pontificate on "balance" looks like evasion. But it isn't.
Read my reviews and you'll see specific criticisms for just about everything I review. But I also try to find what the component does well and who it might appeal to.
Luckily nothing (bar one turntable mat) has completely failed, though sooner or later I'll come across something that has IMHO very little merit no matter what I do. When that happens you'll know that it has had a bad review despite my best efforts, not because I didn't have the time to try different cables or put on something other than the "Cambridge Singers"...

© Copyright 2000 Geoff Husband - http://www.tnt-audio.com