Happy Metal GuyI guess you could say this is Happy Metal Guy’s version of Angry Metal Guy’s well-known article on objectivity mixed with Steel Druhm’s rant about the trials of a music reviewer.

In research methodology, there is a phenomenon called “carry-over effects”. This refers to the problem of a previous experimental treatment’s effects on research subjects carrying over to the next experiment the subjects are participating in, which is likely to confound the results of that experiment.

This problem typically arises because of the desire to stay within a certain budget while conducting experimental research. You see, some researchers try to save as much money as they can, and one way of doing it is using the same sample of research subjects for multiple experimental treatments. This is called the “repeated-measures” design. There are two well-known examples of such a design:

I. Experimental research involving a single group of research subjects, which are administered either different food products or health supplements one after another within a single sitting. This is because the researchers involved in this type of experiment typically try to find out which food product or health supplement results in the greatest benefit to human cognitive aspects such as astuteness, memory strength, speed of learning etc. Examples of such food products or health supplements include energy drink Red Bull, various brands of canned coffee drinks and extracts of ginkgo biloba leaves (a traditional Chinese medicinal herb).

II. Experimental research involving a single group of research subjects seated in an auditorium, which are administered segments of songs from different music genres, one after another within a single sitting. The researchers involved in this type of experiment are typically hired by record labels or music industry figures to find out which genre of music is the “best” or most popular with the general public, and they then make their decision based on the scores that the research subjects assign to the song segments from various music genres.

As you can probably already tell, II is most relevant to my discussion here, but I will elaborate further on how it relates to music reviewers in the context of music reviewing later on. For now, let me explain why researchers utilize the “repeated-measures” design.

(1) Experimental research involving different groups of subjects is very common and always faces the problem of variability: the quality of being subject to slight differences in a certain type of measurable quantity. Such an experimentClockwork typically involves two different groups of subjects, and one group is given the experimental treatment while the other is given the placebo treatment. On one hand, let’s say that Group A people got the real health supplement/food product and display results that seem to indicate that the health supplement/food product has a positive effect on humans. But for all we know, they could just happen to be having a good day and appear more alert or healthier than usual due to a perfect hormone balance. On the other hand, Group B people, who received the placebo treatment, could just happen to be experiencing hormone imbalance on that day and hence, appear less alert or healthier. Hence, researchers cannot confidently conclude that the health supplement/food product administered to Group A has real benefits for humans.

The “repeated-measures” design, however, solves this problem. By using a single group of subjects, it eliminates the need to account for variability between different groups of subjects involved in an experiment. This allows researchers to spend their time in a productive manner as they can then focus more precisely on treatment effects.

(2) The “repeated-measures” design is more economical since the same subjects are subjected to multiple treatments, which means fewer subjects needed and money saved.

As you can see, this particular area of research methodology is akin to the practice of reviewing music. Every active music reviewer is basically being subjected to a continual experiment based on the “repeated-measures” design. Now, allow me to explain it in the context of music reviewing.

testingThe Black Dahlia Murder is generally considered to be a “melodic death metal” band, but I always found it hard to hear the “melodic” in their brand of “melodic death metal”. But this could possibly be due to the fact that I heard Finnish “melodic death metal” way before I heard the American version of it; Children of Bodom and Kalmah were some of the first “melodic death metal” bands I ever heard. We all know how melodic the Finnish brand of “melodic death metal” can be due to the prominence of the synthesizer and expressive guitar melodies, and the carry-over effects from my listening to this version of “melodic death metal” must have made American “melodic death metal”, which mainly focuses on the electric guitar instrument, sound not as melodic by comparison.

Thinking of each instance of reviewing a particular record as a unique experimental treatment then, whenever music reviewers review a particular album, they are never truly assessing it based solely on its ‘own merit’. Unless you are someone who has only ever listened to and reviewed one record, you are sure to have a history with previous records, and this becomes an extraneous variable that confounds the result of the reviewer’s latest assessment of the subject album. For example, I could have been listening to Jay-Z before listening to the new record by Amon Amarth in preparation for reviewing the latter. Even though both artists belong to entirely different genres, the effects of the previous treatment of listening to Jay-Z would not have worn off by the time the treatment of listening to Amon Amarth’s new record is administered. If I happen to really enjoy listening to rap music and nothing else at this point in time, my assessment of Amon Amarth’s new album is going to be negative. If I happen to really hate listening to rap music and like just about any other music genre at this point in time, my assessment of Amon Amarth’s new album is going to be positive. But of course, this is assuming that I would be listening to that new Amon Amarth record immediately after listening to Jay-Z.

So you see, carry-over effect occurs when a treatment is administered before the effects of the previous treatment have worn off. Researchers usually avoid this problem by allowing sufficient time, aka a break, between different treatments.Experiment Theoretically, I could avoid this problem by taking a breather after listening to Jay-Z before I start listening to the new Amon Amarth record.

But in practice, this is very unlikely to happen, especially for those music reviewers who either review music albums for a living or as a passionate hobby. Based on personal experience, I’d say that the right amount of “sufficient time” needed to avoid the problem of carry-over effect in practice would have to be at least one week. This is, however, too much time wasted for the aforementioned music reviewers. Even for music reviewers who do it for fun rather than a career, they are already reviewing multiple albums per week. So can you imagine how many albums the paid music reviewers are typically reviewing within a normal working day/week? They certainly cannot afford to take a breather for one day/week if reviewing albums is their job.

Additionally, if you really want to interpret the argument presented earlier in a strict manner, you could go on to say that no amount of break-time is ever going to eliminate the problem of carry-over effects in music reviewing. This is because as long as you possess memories of your listening experience with other music whenever you are reviewing a particular album, those memories will always interfere and influence your judgment of the record you endeavor to review. The effects of the previous treatments, aka listening to other records in the past, are never going to go away! Also, the latest treatment, aka whichever album you are reviewing, will instantly be inducted into the Hall of Previous Treatments when you move on to the next album, and this vicious cycle of un-objectivity continues until you smash the computer keyboard into eleven pieces, adopt the philosophy of an ethical subjectivist and point out to everyone and anyone that everything is very subjective and there is no such thing as objectivity! Hah!

robot attackI express such verbal frustration because herein lies the logical consequence of the carry-over effects influencing the process of music-reviewing: the general population’s trust in the objectivity of paid/unpaid-but-busy music reviewers is misplaced. Even if I was listening to an old Kalmah album before listening to and reviewing the new Kalmah album, I would still not be judging the latter by its ‘own merit’ because I would be subconsciously comparing it’s qualities to the former’s. (Yeah, I’m looking at all you reviewers out there who always compare a band’s latest material to their old material). [That’s just how it’s done, sonSteel Druhm].

Welp, there you have it. This is basically one of the many other long-winded explanations of why objectivity doesn’t exist in the practice of reviewing music.

Share →
  • PropsToOrpheus

    Nice article! More to the point though, evaluating anything without context is meaningless. And in terms of reviewing music, context is defined in large part by how the music makes you feel—something deeply personal. We all know that immaculate technique or pristine production does not guarantee a great album. Nothing can be evaluated in a vacuum. Carry-over effects, recency effects, familiarity effects, personal bias, etc. all contribute to a ***good*** review, I believe. The caveat is, you need to read several reviews to infer these “confounds” and take them into account when consuming reviews. Each author who writes for this site has unique tastes, and that is expressed in their reviews—it cannot be any other way. Anyone who calls for objectivity in the review process is, I believe, missing the point.

    • “The caveat is, you need to read several reviews to infer these “confounds” and take them into account when consuming reviews.” It’s interesting how this seems to imply that multiple viewpoints on the same piece of musical work are really just fractions of the bigger picture, eh?

      I’ve always wondered this: if we get every single human being alive right now to review the same piece of musical work, and we, as a collective, read every single one of the written reviews to discover the common point(s) brought up by everyone, would that/those common point(s) that everyone brought up be the true judgment of said piece of musical work?

      • PropsToOrpheus

        Interesting! I meant to say “several reviews *by the same person, of different albums*”, to infer a particular reviewers tastes and proclivities. In response to the “look for the intersection among thousands of reviews of one album to find the truth” idea… I don’t think that there necessarily think that there is a truth to be found, and that this will be prone to the same problems we currently face, albiet washed out a bit.

        This is how I am thinking about it: if a million people from around the world were randomly sampled and asked to review a metal album, there are probably going to be two attractors in this “opinion space”. Some people don’t like death metal vox, and find the music abrasive. They are going to rate it low, and comment with thing like “not real music”, “noise”, “painful” etc. On the other hand, there are people who appreciate and regularly consume metal, and they are going to have a different vocabulary, and may appreciate harsher styles. Neither is any closer to the truth… although, one might argue that the impression that death metal has on the uninitiated might tell us something about human response to certain sounds prior to “conditioning”.

        So, lets randomly sample metal connoisseurs, and weed out the plebeians who don’t have a pallet for all things metal. Here, you may find some attractor states like “boring” or “uninspired”, but that is going to be quite related to how much prior exposure one has to metal. There was once a time that I was impressed by the compositional savvy of some truly underwhelming acts; there was once a time, too, where I would have rated Deathspell Omega’s “Kenose” unfavorably, and was unable to stomach Anaal Nathrakh’s “In the constellation of the black widow”, which I now really enjoy and find quite accessible… reviews might still stratify as a function of exposure ***to a particular region of metal-space***. No amount of Rhapsody will prepare a listener for Dodecahedron.

        The things people agree on will be things like “this has a crisp production” or “this musician is talented”. But, among successful metal bands, the production and technical wizardry utilized are part of the art form—and are NOT necessarily a valid means of evaluating the album. It may be true, but it is not informative about the “goodness” of the album.

        Sorry… I’m enjoying this lol.

        • David Rosales

          *nod* *nod* *nod*
          I would love to read articles on music written by you!

          • PropsToOrpheus

            I appreciate the sentiment! I think the world will have to make due with the occasional rambling comment. As much as I love music and reading my own words, I don’t think I have it in me to do what bloggers like the AMG Team do.

      • David Rosales

        I believe not…
        The people reviewing should be at least somewhat knowledgeable on the subject (music) and specific genre… Do you figure anyone who has NEVER listened to Black Metal is going to find Immortal’s third album charming during the first few listens? I’m pretty sure the answer is a big, resounding ‘NO’. Does that mean the album is the worst piece of shit a musician can ever come up with? I don’t think so. The general public prefers a lot of bland and pedestrian stuff just because it does not strain their underused brains.

  • Shahir Chagan

    Yeah, the idea of objectivity in reviewing has always been a sketchy thing with me. After months of reading through a few philosophy books (for debating competitions) and reading different reviews on different things, I can agree with the notion that not one person can make a COMPLETELY ‘objective’ review.
    I would write a whole freaking essay down now, but I feel you and AMG pretty much covered everything I would say on a broader scale and I don’t feel like nitpicking. (In short, I feel too lazy to analyse and dissect different points to argue with)
    Really great read, Happy Metal Guy!

  • Zadion

    This article explores the negatives of the “carry-over effect” while ignoring the positives: it is because we have extended experience with music as a whole that we are able to give weight to our opinions on music. That guy who listened to one album and NOTHING ELSE is likely to enjoy that album, regardless of how terrible it is, because it’s all he truly knows. Because we have some sort of experience with a scene of music, we can balance the positives/negatives of an album compared to the positives/negatives in another album in the same scene, and a reviewer can use comparison to tell you what makes the album he’s reviewing good or bad.

    Additionally, it is because of our experience within a specific scene that we hold relevance to our opinions. If I’m listening to a hip hop album, a genre I’m not all too familiar with, I cannot really weigh its influence and relevance to the genre as a whole. If I hate it, it could just be because I don’t have a taste for hip hop, not because the album lacks qualities other hip hop albums contain that make it bad. The same logic does not apply to a melodeath or progressive metal album. (As a subnote, it could just as easily apply to an atmospheric black or gothic metal album, as I am about as familiar with those as I am with hip hop. It shows the diversity of metal as a whole!)

    Essentially, our opinions will always be just that: opinions. However, carry-over effects allow us experienced, well-informed opinions, I think. This is why I don’t tend to take the opinions of people who don’t make music an active hobby (you know, the people who basically just listen to what’s on the radio and thats it) seriously.

    • Vesal

      Exactly! For all AMG’s rants against objectivity, if his taste wasn’t better “informed” than my own, what would be the point of reading his reviews? It is, of course, completely subjective how a piece of music will hit a person, but there are many objective aspects about the review process. For example, AMG wrote that Amaranthe’s vocals are handled by “a woman, who sounds like a pop singer (think E Type) and who I guarantee you cannot name a Slayer record.” This is a completely objective statement that can be empirically verified by simply interrogating her. Either she can name a Slayer record or she cannot; where is the subjectivity you all talk about?

      • Ah, in this case, the argument would be that the Amaranthe vocalist is arguably not a woman because it is tricky to define “woman” and asking this person to name a Slayer record is pointless because calling it a “Slayer record” implies that the record was a product of group effort, but it could be argued that every piece of art that is supposedly the result of a team effort is really ultimately attributed to one guy’s effort only (see auteur theory). For Slayer, it could be argued to be either Kerry King or Jeff Hanneman.

      • Also, to take subjectivity to the very extreme, I could even argue that a statement such as “This record has eleven tracks” is not objective at all. This is because the concept of numbers is a human creation and it has been traditionally accepted by virtually every human being who ever lived that a certain collection of things should be defined as “one track”, “two tracks” or “ten tracks”, simply because we were taught that those words/numbers ought to represent those particular collection of things.

        • Vesal

          That’s a valid critique if your goal is to capture the ultimate truth about the album. Fortunately, you don’t need to do that because your reviews are not a substitute for the real experience.

          I read your reviews because you (originally AMG and Steel Druhm) have consistently managed to communicate key qualities of an album such that I have come to trust the descriptions on this site. If we accept that truth is more a matter of coherence than correspondence, we should see objectivity as relative to the purpose at hand: to effectively communicate about an album. In these terms, HMG, your reviews are quite rich in objective descriptions of the music.

          (To be clear, objectivity is not necessary for a good review. I quite liked Zed’s review of the baby sausage, though if you recall, many people called it fan-girly. Still, it was a good review because it communicated her first-person experience very elegantly, and it got me interested in the album. However, that’s very different from attempting to describe it in such a way that I can imagine my own experience of the album.)

  • This is far too much egghead yackety yack! Start talking about metal, riffs and boobs!!

  • David Rosales

    “one true form of music”
    Sounds like something classical musicians (the assholes) would say. “We make universal music, everything else is pop — laughable and naive child’s play.”
    Just reminded me of this, I’m sure that is not what you are saying.