Remove bias in gear reviews

Discussion in 'The Sound Hound Lounge' started by Rock72, Dec 20, 2009.

  1. Rock72

    Rock72 Member

    Messages:
    1,644
    Joined:
    Jul 22, 2007
    Maybe this is just another useless thread which will quickly fade. But, I was thinking about how to remove bias in gear testing by magazines etc. I know this is NOT THE SAME, but therapies, including drugs, needs well controlled double-blind studies in order to prove their use. This would of course be impossible with gear, but what if the reviewer initially was blinded to what was tested and had to make judgement purely based on sound, feel and response. When that was finished, the gear tested could be revealed and the discussion of features and build quality start. Of course they would need the b*lls to publish their judgements without going back and polishing it after knowing what brand it was and all the hype surrounding it) :p

    Definitely would be more fun to read guitar magazines....

    Let the discussing begin
     
  2. Pietro

    Pietro 2-Voice Guitar Junkie and All-Around Awesome Guy

    Messages:
    16,413
    Joined:
    Feb 17, 2007
    Location:
    Maineville, OH
    The problem also is that the guitar magazines' revenue comes from advertising products, so they have to be careful which they "hate".

    It's why I go to Consumer Reports for info. No advertising. Nobody to please. So if a Chevy sucks, they can say it sucks.
     
  3. Rock72

    Rock72 Member

    Messages:
    1,644
    Joined:
    Jul 22, 2007

    They should do gear reviews. Pronto :)
     
  4. yucatown

    yucatown Theory-free noodling enthusiast Gold Supporting Member

    Messages:
    2,754
    Joined:
    Nov 16, 2008
    Location:
    Tampa, FL
    I agree with the blind test approach. I'd love to read about side by side blind tests of pedals and amps.
     
  5. daddyo

    daddyo Guest

    Messages:
    11,808
    Joined:
    May 19, 2003
    You want the best gear reviews? It's here, man. Here and gearmandude:)
     
  6. tbp0701

    tbp0701 Member

    Messages:
    442
    Joined:
    Jul 22, 2006
    It's be tough to do unbiased reviews, even without the pressures most publications have. Anything other than the guitar could be blind tested, except the reviewer wouldn't be able to test different settings and controls.

    Maybe there should be an "independent auditor" to evaluate how much extra time the reviewers spent with the equipment when they didn't have to anymore, and how much they were smiling when they were done.
     
  7. Tone_Terrific

    Tone_Terrific Member

    Messages:
    25,250
    Joined:
    Dec 26, 2004
    Location:
    Canada-GTA
    One problem is that no matter what is being tested it is being tried in conjunction with other gear (if we are taking electric guitar). Some stuff works better in certain applications (or rigs) and environments than others.
    There is still the highly subjective aspect that only the player can judge, and this leads to a string of personal experience and expectations.

    However, reviews could be done with more comparisons.
    For instance, if a pedal can generate OD tones can it really replace or compete with the od channel of an amp in side by side comparison? This could be blind tested, etc.
     
  8. waxbytes

    waxbytes Member

    Messages:
    249
    Joined:
    Apr 13, 2008
    Location:
    Canada
    The gear reviews in magazines are there to sell the products being reviewed.
     
  9. Kappy

    Kappy Member

    Messages:
    14,044
    Joined:
    Jan 26, 2005
    Location:
    West Village, NYC
    You're kidding right? At least a magazine can justify its bias with the fact that it needs to make a living from advertizements, here there is no (monetary) reason for 95% of the bias in the reviews/recommendations/rants, etc.
     
  10. loudboy

    loudboy Member

    Messages:
    27,418
    Joined:
    Nov 3, 2003
    Location:
    Sedona, AZ
    Tone is so subjective that this will never happen.

    You can take two guitars, and two people will love one and hate the other, because they've got different requirements for what they like in a guitar.

    I've played a bunch of Tom Anderson guitars, and while they feature spectacular workmanship and playability, they don't speak to me. The few Groshes I've played, however, I would own in a minute. Which one is better? They're both amazing instruments, just different takes...

    I hate TS-9s, and anything that sounds like one. How am I going to review one and give it a good rating?

    Other than a laundry list of features (which you can get from the mfg.) every review will be biased.
     
  11. ABKB

    ABKB Member

    Messages:
    3,167
    Joined:
    Jul 15, 2005
    Location:
    Chicago
    Like the idea of blind testing, but those mags NEVER hate anything, so it wouldn't work anyway. At best in one of their reviews you might catch a line like "we did notice a small problem in that the neck of the guitar fell off while testing" :huh Quickly followed by "but was easily fixed with some super glue" :thud I take ALL their reviews with a grain of salt about the size of the grand canyon. :puh
     
  12. daddyo

    daddyo Guest

    Messages:
    11,808
    Joined:
    May 19, 2003
    No. I'm not kidding. Maybe you'll get half a dozen or more guys jumping on the band wagon of the latest hot amp, but you'll also get guys that will expose the warts as well as touring pros who keep specific amps in their touring rigs despite having the resources to play anything. I much rather have 10 - 20 experienced guys reviewing the latest amp than 1 editor at GP. Plus guys here are into really good clips. Where else can you hear a real Dumble plus 3 or 4 Two Rocks plus a Fuchs playing the same riff for comparison.
     
  13. rh

    rh Robo Sapien Noise Maker Gold Supporting Member

    Messages:
    6,447
    Joined:
    Oct 31, 2003
    Location:
    New York, USA
    The are good reviewers here, and then there are a few serial reviewers here who seem motivated to increase orders for whatever item they're reviewing.
     
  14. Julia343

    Julia343 Member

    Messages:
    7,618
    Joined:
    Oct 31, 2007
    +1 Here and gearmanndude with his white coily cable from the good folks at Bullet Cable. I just wish he did amps, but I guess that's asking for too much.
     
  15. FlyingVBlues

    FlyingVBlues Gold Supporting Member

    Messages:
    4,512
    Joined:
    Mar 20, 2002
    Location:
    Virginia & Cortona Italia
    I hosted a tone fest 2 years ago where we did a blind comparison of a Blueverb, 1966 Pro Reverb, Fargen Blackbird and Bruno Cowtipper 45. We had 18 people who listened to all 4 amps fill out a fairly detailed evaluation form after hearing each of the amps. Here's the thread, and my first and second posts describe the test method we used:

    http://www.thegearpage.net/board/showthread.php?t=236061

    The results are on the last post of this page:

    http://www.thegearpage.net/board/showthread.php?t=236061&page=5

    This was an interesting blind comparison, and the results weren't exactly what I expected. Ben Fargen abd David Smith lent us the Blackbird and Blueverb for the comparison. I don't own either amp or any other Fargen or Blueverb product, and I don't have any sort of affiliation with either company. I do own the Pro Reverb and Cowtipper we used in the blind comparison. This wasn't a perfect way to do an amp comparison (I discuss this in the results post) but it was fairly objective.

    FVB
     
  16. iamdavea

    iamdavea Silver Supporting Member

    Messages:
    7,451
    Joined:
    Sep 10, 2008
    Location:
    Bay Area, CA.
    The magazines would say that the VAST majority of their reviews are generally positive because they don't have the space or inclination to test crappy gear.
     
  17. Flyin' Brian

    Flyin' Brian Member

    Messages:
    29,366
    Joined:
    Jul 29, 2005
    Location:
    K7C 4K8
    There's no reviewer that's going to substitute for your own eyes, ears and hands.

    I read about the specs of something and decide if it's something that I can use. But anything mentioned other than specs is subject to opinion, and that opinion belongs to someone else, no matter how "legit" the reviewer is.
     
  18. iamdavea

    iamdavea Silver Supporting Member

    Messages:
    7,451
    Joined:
    Sep 10, 2008
    Location:
    Bay Area, CA.
    Joe Gore, editor of GP through most of the '90s, was the most thorough and informative reviewer they've ever had. You learned a lot from him.
     
  19. SUBmariner

    SUBmariner Member

    Messages:
    773
    Joined:
    May 13, 2008
    Location:
    Kingman AZ
    It's all subjective. Reviews of gear require reading between the lines and the proverbial grain of salt to garner any useful information. I love those HC user reviews where a given piece of gear gets 10s all the way across from one guy, only to be pronounced a useless POS by the next guy.
     

Share This Page