Hattogate, or, The Music You Hear is Not What You Think

Fans of classical music know that there are great performers and there are the rest. But those with any experience listening to music also know that a lot of what you hear depends on context: if you think you’re hearing a great performer, you’re likely to appreciate their performance even more.

In addition to my activities writing about Macs and iPods, I also review CDs for MusicWeb, an independent British web site that publishes reviews of classical CDs. So I was quite amused this morning when I read an article on the web site of the British classical music magazine Gramophone, entitled Masterpieces Or Fakes? The Joyce Hatto Scandal.

Joyce Hatto is a late pianist who had recorded a handful of good discs then suffered from cancer, from which she died in June 2006. Somewhere in the past year of her life, recordings started spewing out from a small label, run by her husband, showing this woman to have a surprising range of talents, and Gramophone, along with other publications, began championing these recordings. But some months ago, posters to the rec.music.classical.recordings newsgroup began questioning the possibility that this woman could have played all these works, with styles that sounded so different. One poster said:

“After hearing so much about Joyce Hatto, I started purchasing some of her recordings. While nothing I have heard is bad (in fact, I am glad I bought these CDs), I have noticed something eerie: that the pianist playing the Mozart sonatas cannot be the pianist playing Prokofiev
or the pianist playing Albeniz. I have the distinct feeling of being the victim of some sort of hoax. Does anyone else share these feelings?”

Well, where I come from, you might say, “them’s fightin’ words”, but they incited some people to start looking more closely at this phenomenon. The results seem to be clear (as shown in the Gramophone article linked above): not only was this a hoax, but a purely monetarily-driven one, which simply took copies of some works, fiddled with others, and released them to a world of people who fawn after the latest sensation.All this raises many questions, of course. First, you have to feel bad for the professional critics who, hearing something they liked, not only lauded it, but created the context to fulfill their wishes with each subsequent recording from this pianist. Second, it shows that there are, perhaps, some recordings by lesser-known musicians that had been “pirated” and branded with the Joyce Hatto name which merit further attention. Had these same critics panned the discs that were the actual sources of the Hatto recordings?

Finally, and perhaps more important, it shows the futility of any kind of criticism. Well, you can’t copy books or movies, but for classical music where critics review not so much the music as the interpretation and performance, how much criticism is truly objective? Perhaps it is time for critics to work blindly, getting nothing but blank discs (or digital files) and reviewing these, then, only after the reviews are filed, finding out who the performers are. This would, of course, not be to the liking of the major record labels, for whom marketing is often more important than actual performances. (Granted, this is only really valid for instrumental performances; it is relatively simple to recognize a familiar voice in an opera or other vocal recording.)

There has always been criticism of critics, but nowhere other than the classical music arena does the concept of “great performances” or “reference performances” hold sway. These are the benchmarks against which other performances are measured, and they can be self-fulfilling: the more familiar you are with your benchmark, the more you will like it and reinforce its validity.

I tend to be somewhat obsessive about music, and, for some composers, actively seek out different versions of works I like in order to have a variety of performances, because no one performance can be considered final or perfect. I have never succumbed to unfailing appreciation for a specific artist (though Dietrich Fischer-Dieskau is by far my preferred singer of German lieder), and tend to search broadly. It’s a shame for those who do think they found the new musical messiah, and many music publications–including Gramophone, who will have to make a serious mea culpa–will suffer from this type of hoax.

While it’s almost surprising this hasn’t happened before, there are actually a few reasons that make this case different from others. Joyce Hatto had not performed anywhere for a long time, so no one would have been able to compare her performance style with her recordings. Also, this was a very small label, and, while Hatto-mania may have blossomed, it certainly never went far enough to generate large sales. It seems that the greed behind this hoax was limited to a single person, the late pianist’s husband.

But with digital technology so prevalent, such that anyone can copy a CD and release it as their own, no one has time to check all the recordings that are released to make sure they are what they say. (Kudos to Andrew Rose of Pristine Audio for taking the time to analyze these recordings down to their waveforms; check this link for examples, both audio and visual, proving that the Hatto recordings are not indeed Hatto recordings.) While it is unlikely that there are many unscrupulous record labels who would consider perpetrating such a hoax, the cat’s out of the bag, and this may give ideas to others. Caveat emptor, right?

18 thoughts on “Hattogate, or, The Music You Hear is Not What You Think

  1. I’ve never listened to a Joyce Hatto recording. And this may very well be a
    hoax — it probably is. However, I’d suggest caution rather than rushing to
    judgement, because a good deal of the comment regarding this scandal
    seems to be technically ill-informed.

    The most persuasive case is made here:
    <http://www.charm.rhul.ac.uk/content/contact/hatto_article.html&gt;

    But other articles (Gramophone, NYT, IHT, Classics Today, etc) are very short
    or misleading on crucial technical elements. For instance, the scandal was
    supposed to have started with a critic and former Hatto fan inserting one of
    her CDs in his computer and having iTunes identify the performer as Laszlo
    Simon instead of Joyce Hatto. But anyone who understands how iTunes
    identifies audio CD tracks, and is familiar with the character of the Gracenote
    CDDB and its limitations with respect to classical music, will immediately
    realise that iTunes misidentifying the performer is proof of nothing.

    The issue becomes even more complicated when we find out that, according
    to Andrew Rose <http://www.pristineclassical.com/HattoHoax.html&gt;,
    of the twelve tracks on the Hatto CD, ten were actually by Laszlo Simon, and
    two by Minoru Nojima. Such an audio CD should not have been identified by
    iTunes either as a performance by Simon or one by Nojima. Moreover,
    Andrew Rose argues that the Simon recordings attributed to Hatto were time-
    shrunk by 0.02%, which would have altered the track duration enough, once
    more, to prevent iTunes from recognising the CD as one by Laszlo Simon.

    Gramophone states that Andrew Rose "scientifically checked the sound-
    waves"; but a visual comparison in Adobe Audition of two different sound-
    waves of the same music is anything but "scientific".

    It is entirely possible, perhaps even likely, that all this is true: Joyce Hatto’s
    recordings were fakes, Andrew Rose’s comparisons were valid, and so on. But
    the contradictions and muddled (or lack of) methodological and technical
    details so far should give one pause. It seems to me that, just as critics
    rushed to proclaim Joyce Hatto an unrecognised genius, they are now rushing
    to proclaim her a hoax, without really understanding the technology involved.
    These critics need not only to update their skills for the digital age, but also
    to reflect a little on Webster’s definition of "critical": "[…] to exercise careful
    judgement and selection […]".

    • I’ve asked the same question about the iTunes thing, because I know how it
      works with Gracenote – it compares frames and offsets, so anything that is
      not _exactly_ the same will not turn up a disc which is a (partial) copy.

      However, this is the disc it turned up:

      http://www.gracenote.com/music/album.html/genclassical/
      865115b0574fa1ba9388bfa349a9038e.html

      Laszlo Simon / Franz Liszt -12 Etudes d’execution transcendante
      Label: Concert Artists

      My thought is that someone who was in on the scam got this info onto the
      Gracenote CDDB in the hopes that it would eventually be discovered; if the
      performer were listed as Hatto, or the label as Bis, then I would accept some
      sort of (wildly improbable) coincidence, but this looks too much like a plant.

      Second, regarding Andrew Rose, he’s a very talented engineer, having worked
      with the BBC, and working now restoring old recordings. You can listen to
      exceprts on his web site – it’s obvious that he discovered the "originals" for
      the Hatto tracks he has examined.

      • But which disc is it? The Simon CD, with the wrong label? The Hatto CD with
        the wrong artist? (Remember, according to Rose, two of ten tracks are
        Nojima’s, not Simon’s.) And, when iTunes came up with "Simon" as the
        performer on the Hatto CD, why did the critic mutter darkly, "Something is
        rotten in the state of Denmark", instead of dismissing it as yet another
        example of the countless errors in the classical music section of CDDB? And
        why is it that none of the critics and bloggers thought to ask such basic
        questions?

        As to Andrew Rose, the issue is neither his expertise nor his probity (both of
        which are beyond question). The issue is the description of his conclusions as
        "scientific" (Gramophone, NYT, etc.) Visual examination of two waveforms is
        no more scientific than a critic listening to two different recordings and
        pronouncing them to be of the same performer.

        The CHARM article is closer to the "scientific" label (and thus more
        persuasive), but it, too, falls short. It contains no reference and no description
        of the methodology of building what they call "timescapes"; where they use a
        well-established parameter (the Pearson correlation coefficient), they do not
        explain what data was used to calculate it and how it was acquired. Worse,
        some of the results seem to be contradictory: the Indjic/Rubinstein 1939
        comparison yields r < 0.8 for Opp. 17 and 68, but an amazingly high value
        (0.954) for the "entire set".

        Again, why is it that none of the critics/bloggers have raised these issues?

        • My guess is that someone who had info about the scam managed to get the
          disc’s info on the Gracenote CDDB. It is not Gracenote that spotted the
          performer as being Simon, but rather the disc as matching a record in its
          database. So someone had to add that record (which lists Simon as the
          performer of the Hatto disc, and not on Bis but on Concert Artists). I don’t
          know if it’s possible to find out when that info was added. But it was clearly a
          clue planted by someone.

          Why haven’t others examined this? Because they don’t understand it. They
          don’t realize that the Gracenote CDDB merely matches frames and offsets to
          find a record that corresponds to a disc ID it uses. It doesn’t "analyze" the
          music (as a New Scientist article suggests). Most journalists are merely
          reporting what they’ve read elsewhere. However, the Gramophone article is
          wrong, when it says that Distler discovered this. Actually, it was a reader of
          Classics Today who sent the info to that site (where Distler writes reviews)
          who then passed the info on to Gramophone who went public before Classics
          Today. Sigh.

          As for scientific, I think that waveforms are pretty unique; you’d have a match
          for a short passage, perhaps, but for entire pieces? Doubtful. It will most
          likely be the courts that decide which approach is sufficient to inculpate
          Barrington-Coupe, but in the mean time it’s interesting to see how more and
          more discs are indentified, and how little-known pianists will benefit from
          this.

          • "My guess is that someone…" Perhaps it was a Freudian slip of Barrington-
            Coupe’s. Or perhaps it was Barrington-Coupe, racked by guilt and remorse,
            driven by an unconscious urge to confess. Or — and I find this most likely —
            it was an Albanian dwarf named Cyrano, kept chained to a dank wall in the
            basement of the Concert Artist studio. At any rate, we can guess as much as
            we like, but guesses aren’t proof of anything, especially not of journalists and
            music critics doing the jobs they’re paid for.

            "Most journalists are merely reporting what they’ve read elsewhere." A telling
            indictment of the profession, and the Hatto stories do much to prove it. No
            wonder journalists rate almost as low as politicians in public confidence polls.

            "As for scientific, I think that waveforms are pretty unique". The symptom of
            the problem in a nutshell. It is not the OBJECT, but the METHOD which makes
            a study scientific or not. Astronomy and astrology study the same objects —
            one is a science, the other is not. Palaeontology and "creation science" study
            the same objects — one is a science, the other is not. Waveforms being
            "pretty unique" does not make a visual comparison of two waveforms
            "scientific". As a matter of fact, if you look carefully at the Hatto and Simon
            waveform pics posted by Rose, you will notice, even at low rez, that they are
            not identical. Do the differences matter? Based on his experience, Rose says
            they don’t, and I believe him; but that doesn’t make his opinion "scientific".
            He may be wrong, and I may be wrong to believe him, just as wrong as the
            music lovers who believed Gramophone when it proclaimed Joyce Hatto "the
            greatest living pianist that no one has heard of" — the same Gramophone
            who now proclaims Joyce Hatto a hoax.

            This whole story shows that the journalists, the critics, the bloggers, the
            pundits, etc, have learned absolutely nothing. Yesterday they rushed to
            proclaim Hatto a genius — without even the minimal common sense of
            asking how it was possible for a cancer-afflicted elderly woman who hadn’t
            performed in public for two decades suddenly to start producing a massive
            volume of recordings, all of top technical quality, covering the entire gamut
            of the piano repertoire. Today, the same people, lemming-like, are rushing to
            proclaim her a fraud — without even the minimal common sense to ask a few
            basic questions. (How exactly did iTunes do what it is supposed to have
            done? What exactly is a "scientific" study of waveforms? How valid is a
            "scientific" study which posts a value indicating almost complete statistical
            correlation between Eugen Indjic of 1988 and Rubinstein of 1939?)

            The moral of the story is that journalists, music critics, bloggers, etc — all
            those whom we assume to have a broader education, a finer sensibility, a
            more discerning intellect, and a more inquiring mind than the rest of us hoi
            polloi — turn out to be no more than a flock of sheep, baahing mindlessly
            while they rush hither and yon as some bellwether leads them. The real
            problem is that, after all, it’s us, not them, who end up being fleeced.

        • > Again, why is it that none of the critics/bloggers
          > have raised these issues?
          

          Because sensationalism sells, and rational thought puts the public
          to sleep. Anyway, it is finally occurring here we noticed.

          > but it, too, falls short. It contains no reference
          > and no description of the methodology of building
          > what they call "timescapes";
          

          I am a co-author of the CHARM
          report
          . We had been preparing the report as an article in response
          to Gramophone’s March 2006 challenge for compelling evidence that Hatto
          was not genuine; however, the scandal surfaced independently, so we just
          deposited the current state of our report on our website, since problems
          dealing with libel would be the responsibility of the news breaker. Thus,
          a full analysis was not yet prepared in writing. That is our excuse for
          it falling short. In any case, it was indended to only point out the
          amazing conincidences between the two sets of performances, nothing else.
          Why they are so similar is still a complete mystery to us…

          Timescapes show correlations at different timescales, but they do not show
          probabilities (they do show probability in a weak sense). So timescapes
          can not be used to scientifically or legally prove that the two mazurka
          sets were the same. The importance of timescapes is that they were
          instrumental in the discovery of the striking similarity between the
          two mazurka sets, and they can thus place the debate of similarity on
          a level that even a three-year-old can comprehend (I am not sure about
          adults, though). Examine the timescape pictures related to the report
          which are found on the webpage:
          http://mazurka.org.uk/ana/pcor
          Look at the various pictures, and you will notice that most contain
          structures that indicate either random or real similarities between
          multiple performances of the same mazurka (there are two pairs of
          timescapes where I can actually see that one performer owns the
          recorings of another pianists on CD).

          However, there are a few exceptions to the norm in the plots. This is
          when the entire plot show a solid field of color. This indicates that
          the best correlation at all timescales always matches to a single
          other performance (out of typically 30 performances per work). This is
          expected to occur when the plots represent different re-releases of the
          same performance, but it is very extremely unlikely to occur between two
          independent performances. For example, there are multiple Rubinstein
          performances of the same mazurka on the timescape pages. He is most
          similar to himself in other performances, but there are other closer
          similarities that break the picture up into multiple regions.

          > Worse, some of the results seem to be contradictory:
          > the Indjic/Rubinstein 1939 comparison yields r < 0.8
          > for Opp. 17 and 68, but an amazingly high value
          > (0.954) for the "entire set".
          

          The information is not contradictory. Correlation does not imply
          causation. Or in other words, correlation does not imply probability.
          For example, correlation between sets of pairs of numbers can be very
          high just due to random processes.

          What you wanted to see in the article are probabilities, which cannot
          really be calculated from the correlations. However, we have calculated
          probabilities related to the Hatto/Indjic mazurka sets which are
          outlined below:

          Probability that Hatto track times matched Indjic track times by
          random coincidence.
          All track times match between the two within about +/- 5
          seconds (due to time stretching of one of the sets and different amounts of silence padding). Assume that
          the probability that any one mazurka performance can be matched at random
          to within 5 seconds of another independent performance is about 75% likely. This is a very
          rough assumption, but it should be fair and conservative. Therefore the
          probability that all 49 mazurkas which I was interested in (there are
          about 10 more on the two disks) can be calculated by this equation:

               1 / 0.75^49 = 1 in 1,324,336
          
          

          Thus, the probability that the the track times match so closely is
          approximately one in a million. This cannot be caluculated from the
          r-value for the correlation between Indjic and Hatto track times directly. This is not scientific proof that the two
          recordings are the same, and it is not legal-level proof either, since
          one in a million does still can leave a reasonable amount of doubt.

          Probability that a single Hatto recording is the same as the matching
          Indjic recording
          . I use Mazurka in A minor Op. 17, No. 4 as an example
          in this case. I measured the downbeat locations in an independent
          manner. The accuracy of my timing measurement is 10 milliseconds.
          I then subtracted the difference between the two beat-time sets.
          There was a linear increase in the differences due to time shifting in
          one of the performances. This was removed. What resulted was a plot
          of timing differences between about 400 beats in the piece. There was a
          region of beat timing differences which was about 200 beats long, where
          the average beat timings were all less than 11 milliseconds. If a human
          were to try to tap to a constant beat, they would be expected to get less
          than 1/2 of the taps within 25 milliseconds (note that mazurkas are not
          performed at a steady tempo, so it actually should be a larger deviation).
          Being able to hit the beat timings at less than a 25 millisecond deviation
          for 200 consecutive beats is equivalent to flipping a coin and having it
          come up heads 200 consecutive times. The probability of that occuring
          is one in 1^200 or 1 in 10^59.

          This proves beyond a reasonable doubt that the two recordings are the
          same. 1 in 10^59 is approximately the same as one atom of hydrogen out of
          all other atoms of hydrogen in a star. And this is only the probability
          for a single performance on the CDs. Additionaly, if you were to examine
          the waveform for acoustic similarities (they have different durations
          and filtering, so are not exact copies), the probability that they are
          the same performance skyrockets much, much higher.

          • > The probability of that occuring is
            > one in 1^200 or 1 in 10^59.

            Ack, I meant:
            1 in 2^200, or 1 in 10^59

            1^200 is still 1. Anyway it was a typo, and I don’t want
            anyone pointing out the numerous spelling mistakes I made…

          • > Thus, a full analysis was not yet prepared in writing. That is our
            > excuse for it falling short.

            That a "full analysis" hasn’t been prepared yet is understandable. What is not
            understandable is the complete silence regarding data and methodology.
            What exactly was the raw data used? How was it acquired and processed?
            What was the algorithm employed to generate "timescapes"? This shouldn’t
            require a lot of time to deal with: if it has been published all you need to do is
            add the refs. If it’s a novel method, somebody’s on-going thesis work, then
            this needs to be said, and, once again, it shouldn’t take too much time or
            space to do so. Moreover, on your web page you don’t have the stylistic and
            space constraints of a journal article, so you don’t have to worry about
            formatting and you can post all the raw data you wish.

            > The importance of timescapes is that they were instrumental in the
            > discovery of the striking similarity between the two mazurka sets, and
            > they can thus place the debate of similarity on a level that even a
            > three-year-old can comprehend […]

            The importance of "timescapes" cannot be determined without understanding
            how they were generated, just as you cannot understand what an X-ray
            shows if you don’t know how it was generated. I said (and I still believe) that
            your article made the "most persuasive case", but my belief was based on a
            fundamentally unscientific assumption. Without the process, timescapes are
            really just pretty pictures, no doubt highly gratifying to a three-year-old, but
            falling somewhat short of that for an adult with minimal scientific literacy.

            > The information is not contradictory.

            It may not be, but I need a better explanation for Indjic/Rubinstein 1939 r
            values of 0.616 (Op. 17/4), 0.664 (Op. 68/3), and 0.954 (entire set).

            > Correlation does not imply causation.

            You cannot have it both ways. Either r says something, in which case you
            have to explain not only the values for Hatto/Indjic, but also those for Indjic/
            Rubinstein; or it does not, in which case, why was it featured at all? In fact,
            one could make a good case that r should not have been included in the
            paper to begin with, because we already know the variables are correlated —
            they are performances of the same musical piece! But it was your choice to do
            so.

            > What you wanted to see in the article are probabilities […]

            Actually, no, that was not what I wanted. To recapitulate, what I felt was
            missing was, (a) methods of data acquisition and processing, (b) methods of
            building timescapes, (c) discussion of apparent inconsistencies in presented
            data, (d) tables of all values used to generate timescapes and calculate
            Pearson’s coefficient.

            • > Moreover, on your web page you don’t have the
              > stylistic and space constraints of a journal article,
              > so you don’t have to worry about formatting and you
              > can post all the raw data you wish.

              The timescape pictures were generated from beat timing data found on
              the webpage:
              http://mazurka.org.uk/info/excel/beat
              (In MS Excel format, and a simpler text file containing same data).

              > if it has been published all you need to do is add the refs.
              > If it’s a novel method, somebody’s on-going thesis work, then
              > this needs to be said

              (1) It has not been published yet :-), and (2) it is a novel method.
              (3) William Barrington-Coupe has admitted to fraud, so no need to dot
              anymore t’s or cross anymore eyes in the aborted article.

              If you are interested, there is a description of a predecesor method
              call "keyscapes" which I described two years ago in this article:

              Sapp, Craig. "Visual Hierarchical Key Analysis" in Computers in
              Entertainment [http://www.acm.org/pubs/cie.html], volume 3, no. 4
              (Oct 2005)

              Also an earlier description is found in the Proceedings of the
              International Computer Music Conference 2001. I adapted the method for
              performance analysis this summer with only one minor modification in
              the methodology.

              The application for performance analysis will be presented next month
              in the CHARM Symposium 4: Methods for analysing recordings which will
              be held just outside of London on 12-14 April:
              http://www.charm.rhul.ac.uk/content/events/symp4.html

              An article that I am writing on the method will appear in an issue of
              Musicae Scientiae towards the end of this year:
              http://musicweb.hmt-hannover.de/escom/english/MusicScE/MSstart.htm

              Also, I will probably submit an article related to timescapes to the
              ISMIR 2007 conference, so you might be able to find it on their website
              in the Fall: http://ismir2007.ismir.net

              —————————————————————–

              Timescape methodology in brief: Take a 1-dimensional string of numbers
              (beat tempo or beat durations in this case) and perform pair-wise
              correlation (Pearson) between a reference performance and the other
              performances’s data. The correlations are done in increasingly larger
              sub-sequences of data, until the entire data set pairs are correlated to
              each other.

              For example, if there are 6 numbers (A,B,C,D,E,F) in the performance
              sequence, these sub-sequence pairings are used to calculate correlation
              values:
              AB, BC, CD, DE, EF, ABC, BCD, CDE, DEF, ABCD, BCDE, CDEF, ABCDE,
              BCDEF, ABCDEF
              These correlations are done between the reference performance and the
              other performances being compared to the reference performance.

              Next, a 2-dimensional plotting diagram is filled from small-scale
              subsequences at the bottom to the complete sequence correlations at
              the top. For the example sequence, this is approximately how the 2d
              plotting area is arranged:
              ABCDEF
              ABCDE, BCDEF
              ABCD, BCDE, CDEF
              ABC, BCD, CDE, DEF
              AB, BC, CD, DE, EF
              (Imagine each line being centered, since the spacing is
              altered due to web formatting)

              For correlation, the sub-sequences A, B, C, D, E, and F are not
              interesting, but for other sub-sequence calculations (such as average
              tempo instead of correlation), I might add them to the bottom row.

              Now, each cell in the plotting area is assigned a color. The color
              is given by which other performance has the highest correlation value
              to the reference performance. For example, if there were 100 performances
              and 1 of them is the reference performance for the plot, then there are
              99 other performances that were correlated with the reference at
              sub-sequence BCDE. I would look at all 99 of those correlation
              values and place a color in the BCDE slot in the timescape which
              represents that performance.

              In the Hatto/Indjic case, nearly all sub-sequences of their independent
              beat duration/tempos correlated better than any of the other 30
              or so performances by other pianists. The only ones which didn’t
              match were the short sub-sequence correlations at the bottom of the
              pictures which can randomly generate high correlation values by chance
              since they are short sequences:
              http://mazurka.org.uk/ana/pcor
              Also, for those interested, I have generated "dynascapes" this week
              which compare beat loudness (dynamics) between performances:
              http://mazurka.org.uk/ana/pcor-gbdyn
              Source data for these plots are found at:
              http://mazurka.org.uk/info/excel/dyn/gbdyn
              My focus group of three-year-olds can still pick out the Hatto/Indjic
              pairing in these pictures :-).

              Various slide shows which include some nicer figures can be found at
              http://mazurka.org.uk/info/present

              > timescapes are really … pretty pictures
              Thanks, I agree 🙂

              > > Correlation does not imply causation.

              There can be many reasons why the r-value is high, and it is all the
              more confusing since they are related to the same composition, as
              you point out. The probability calculations give in my previous entry
              are a more stronger basis for demonstrating plagiarism, since one of
              them demonstrates that it is not humanly possible to imitate another
              performance to such a high degree of accuracy. In other words, p-value
              calculations are difficult for this type of performance analysis, and
              I am interested in anyone who has suggestions…

              > (a) methods of data acquisition and processing,

              The data acquisition methodology is also novel and would take
              too long to describe here in the margins :-), but basically
              I am writing my own plugins to this free audio analysis editor:
              http://www.sonicvisualiser.org
              My plugins:
              http://sv.mazurka.org.uk/download
              If you own a copy of Chopin mazurkas for which I have extracted
              data, you can load one of those performances into the editor,
              and then download one of the annotation layer files from
              the webpage (Sonic Visualiser can load a file directly
              from a web address):
              http://mazurka.org.uk/ana/markup
              Such as the beat times in Joyce Hatto’s performance of Mazurka 17/4:
              http://mazurka.org.uk/ana/markup/pid9073-15/contents.html
              http://mazurka.org.uk/ana/markup/pid9073-15/pid9073-15-avgtap-sv.txt

  2. I’ve never listened to a Joyce Hatto recording. And this may very well be a
    hoax — it probably is. However, I’d suggest caution rather than rushing to
    judgement, because a good deal of the comment regarding this scandal
    seems to be technically ill-informed.

    The most persuasive case is made here:
    <http://www.charm.rhul.ac.uk/content/contact/hatto_article.html&gt;

    But other articles (Gramophone, NYT, IHT, Classics Today, etc) are very short
    or misleading on crucial technical elements. For instance, the scandal was
    supposed to have started with a critic and former Hatto fan inserting one of
    her CDs in his computer and having iTunes identify the performer as Laszlo
    Simon instead of Joyce Hatto. But anyone who understands how iTunes
    identifies audio CD tracks, and is familiar with the character of the Gracenote
    CDDB and its limitations with respect to classical music, will immediately
    realise that iTunes misidentifying the performer is proof of nothing.

    The issue becomes even more complicated when we find out that, according
    to Andrew Rose <http://www.pristineclassical.com/HattoHoax.html&gt;,
    of the twelve tracks on the Hatto CD, ten were actually by Laszlo Simon, and
    two by Minoru Nojima. Such an audio CD should not have been identified by
    iTunes either as a performance by Simon or one by Nojima. Moreover,
    Andrew Rose argues that the Simon recordings attributed to Hatto were time-
    shrunk by 0.02%, which would have altered the track duration enough, once
    more, to prevent iTunes from recognising the CD as one by Laszlo Simon.

    Gramophone states that Andrew Rose "scientifically checked the sound-
    waves"; but a visual comparison in Adobe Audition of two different sound-
    waves of the same music is anything but "scientific".

    It is entirely possible, perhaps even likely, that all this is true: Joyce Hatto’s
    recordings were fakes, Andrew Rose’s comparisons were valid, and so on. But
    the contradictions and muddled (or lack of) methodological and technical
    details so far should give one pause. It seems to me that, just as critics
    rushed to proclaim Joyce Hatto an unrecognised genius, they are now rushing
    to proclaim her a hoax, without really understanding the technology involved.
    These critics need not only to update their skills for the digital age, but also
    to reflect a little on Webster’s definition of "critical": "[…] to exercise careful
    judgement and selection […]".

    • I’ve asked the same question about the iTunes thing, because I know how it
      works with Gracenote – it compares frames and offsets, so anything that is
      not _exactly_ the same will not turn up a disc which is a (partial) copy.

      However, this is the disc it turned up:

      http://www.gracenote.com/music/album.html/genclassical/
      865115b0574fa1ba9388bfa349a9038e.html

      Laszlo Simon / Franz Liszt -12 Etudes d’execution transcendante
      Label: Concert Artists

      My thought is that someone who was in on the scam got this info onto the
      Gracenote CDDB in the hopes that it would eventually be discovered; if the
      performer were listed as Hatto, or the label as Bis, then I would accept some
      sort of (wildly improbable) coincidence, but this looks too much like a plant.

      Second, regarding Andrew Rose, he’s a very talented engineer, having worked
      with the BBC, and working now restoring old recordings. You can listen to
      exceprts on his web site – it’s obvious that he discovered the "originals" for
      the Hatto tracks he has examined.

      • But which disc is it? The Simon CD, with the wrong label? The Hatto CD with
        the wrong artist? (Remember, according to Rose, two of ten tracks are
        Nojima’s, not Simon’s.) And, when iTunes came up with "Simon" as the
        performer on the Hatto CD, why did the critic mutter darkly, "Something is
        rotten in the state of Denmark", instead of dismissing it as yet another
        example of the countless errors in the classical music section of CDDB? And
        why is it that none of the critics and bloggers thought to ask such basic
        questions?

        As to Andrew Rose, the issue is neither his expertise nor his probity (both of
        which are beyond question). The issue is the description of his conclusions as
        "scientific" (Gramophone, NYT, etc.) Visual examination of two waveforms is
        no more scientific than a critic listening to two different recordings and
        pronouncing them to be of the same performer.

        The CHARM article is closer to the "scientific" label (and thus more
        persuasive), but it, too, falls short. It contains no reference and no description
        of the methodology of building what they call "timescapes"; where they use a
        well-established parameter (the Pearson correlation coefficient), they do not
        explain what data was used to calculate it and how it was acquired. Worse,
        some of the results seem to be contradictory: the Indjic/Rubinstein 1939
        comparison yields r < 0.8 for Opp. 17 and 68, but an amazingly high value
        (0.954) for the "entire set".

        Again, why is it that none of the critics/bloggers have raised these issues?

        • My guess is that someone who had info about the scam managed to get the
          disc’s info on the Gracenote CDDB. It is not Gracenote that spotted the
          performer as being Simon, but rather the disc as matching a record in its
          database. So someone had to add that record (which lists Simon as the
          performer of the Hatto disc, and not on Bis but on Concert Artists). I don’t
          know if it’s possible to find out when that info was added. But it was clearly a
          clue planted by someone.

          Why haven’t others examined this? Because they don’t understand it. They
          don’t realize that the Gracenote CDDB merely matches frames and offsets to
          find a record that corresponds to a disc ID it uses. It doesn’t "analyze" the
          music (as a New Scientist article suggests). Most journalists are merely
          reporting what they’ve read elsewhere. However, the Gramophone article is
          wrong, when it says that Distler discovered this. Actually, it was a reader of
          Classics Today who sent the info to that site (where Distler writes reviews)
          who then passed the info on to Gramophone who went public before Classics
          Today. Sigh.

          As for scientific, I think that waveforms are pretty unique; you’d have a match
          for a short passage, perhaps, but for entire pieces? Doubtful. It will most
          likely be the courts that decide which approach is sufficient to inculpate
          Barrington-Coupe, but in the mean time it’s interesting to see how more and
          more discs are indentified, and how little-known pianists will benefit from
          this.

          • "My guess is that someone…" Perhaps it was a Freudian slip of Barrington-
            Coupe’s. Or perhaps it was Barrington-Coupe, racked by guilt and remorse,
            driven by an unconscious urge to confess. Or — and I find this most likely —
            it was an Albanian dwarf named Cyrano, kept chained to a dank wall in the
            basement of the Concert Artist studio. At any rate, we can guess as much as
            we like, but guesses aren’t proof of anything, especially not of journalists and
            music critics doing the jobs they’re paid for.

            "Most journalists are merely reporting what they’ve read elsewhere." A telling
            indictment of the profession, and the Hatto stories do much to prove it. No
            wonder journalists rate almost as low as politicians in public confidence polls.

            "As for scientific, I think that waveforms are pretty unique". The symptom of
            the problem in a nutshell. It is not the OBJECT, but the METHOD which makes
            a study scientific or not. Astronomy and astrology study the same objects —
            one is a science, the other is not. Palaeontology and "creation science" study
            the same objects — one is a science, the other is not. Waveforms being
            "pretty unique" does not make a visual comparison of two waveforms
            "scientific". As a matter of fact, if you look carefully at the Hatto and Simon
            waveform pics posted by Rose, you will notice, even at low rez, that they are
            not identical. Do the differences matter? Based on his experience, Rose says
            they don’t, and I believe him; but that doesn’t make his opinion "scientific".
            He may be wrong, and I may be wrong to believe him, just as wrong as the
            music lovers who believed Gramophone when it proclaimed Joyce Hatto "the
            greatest living pianist that no one has heard of" — the same Gramophone
            who now proclaims Joyce Hatto a hoax.

            This whole story shows that the journalists, the critics, the bloggers, the
            pundits, etc, have learned absolutely nothing. Yesterday they rushed to
            proclaim Hatto a genius — without even the minimal common sense of
            asking how it was possible for a cancer-afflicted elderly woman who hadn’t
            performed in public for two decades suddenly to start producing a massive
            volume of recordings, all of top technical quality, covering the entire gamut
            of the piano repertoire. Today, the same people, lemming-like, are rushing to
            proclaim her a fraud — without even the minimal common sense to ask a few
            basic questions. (How exactly did iTunes do what it is supposed to have
            done? What exactly is a "scientific" study of waveforms? How valid is a
            "scientific" study which posts a value indicating almost complete statistical
            correlation between Eugen Indjic of 1988 and Rubinstein of 1939?)

            The moral of the story is that journalists, music critics, bloggers, etc — all
            those whom we assume to have a broader education, a finer sensibility, a
            more discerning intellect, and a more inquiring mind than the rest of us hoi
            polloi — turn out to be no more than a flock of sheep, baahing mindlessly
            while they rush hither and yon as some bellwether leads them. The real
            problem is that, after all, it’s us, not them, who end up being fleeced.

        • > Again, why is it that none of the critics/bloggers
          > have raised these issues?
          

          Because sensationalism sells, and rational thought puts the public
          to sleep. Anyway, it is finally occurring here we noticed.

          > but it, too, falls short. It contains no reference
          > and no description of the methodology of building
          > what they call "timescapes";
          

          I am a co-author of the CHARM
          report
          . We had been preparing the report as an article in response
          to Gramophone’s March 2006 challenge for compelling evidence that Hatto
          was not genuine; however, the scandal surfaced independently, so we just
          deposited the current state of our report on our website, since problems
          dealing with libel would be the responsibility of the news breaker. Thus,
          a full analysis was not yet prepared in writing. That is our excuse for
          it falling short. In any case, it was indended to only point out the
          amazing conincidences between the two sets of performances, nothing else.
          Why they are so similar is still a complete mystery to us…

          Timescapes show correlations at different timescales, but they do not show
          probabilities (they do show probability in a weak sense). So timescapes
          can not be used to scientifically or legally prove that the two mazurka
          sets were the same. The importance of timescapes is that they were
          instrumental in the discovery of the striking similarity between the
          two mazurka sets, and they can thus place the debate of similarity on
          a level that even a three-year-old can comprehend (I am not sure about
          adults, though). Examine the timescape pictures related to the report
          which are found on the webpage:
          http://mazurka.org.uk/ana/pcor
          Look at the various pictures, and you will notice that most contain
          structures that indicate either random or real similarities between
          multiple performances of the same mazurka (there are two pairs of
          timescapes where I can actually see that one performer owns the
          recorings of another pianists on CD).

          However, there are a few exceptions to the norm in the plots. This is
          when the entire plot show a solid field of color. This indicates that
          the best correlation at all timescales always matches to a single
          other performance (out of typically 30 performances per work). This is
          expected to occur when the plots represent different re-releases of the
          same performance, but it is very extremely unlikely to occur between two
          independent performances. For example, there are multiple Rubinstein
          performances of the same mazurka on the timescape pages. He is most
          similar to himself in other performances, but there are other closer
          similarities that break the picture up into multiple regions.

          > Worse, some of the results seem to be contradictory:
          > the Indjic/Rubinstein 1939 comparison yields r < 0.8
          > for Opp. 17 and 68, but an amazingly high value
          > (0.954) for the "entire set".
          

          The information is not contradictory. Correlation does not imply
          causation. Or in other words, correlation does not imply probability.
          For example, correlation between sets of pairs of numbers can be very
          high just due to random processes.

          What you wanted to see in the article are probabilities, which cannot
          really be calculated from the correlations. However, we have calculated
          probabilities related to the Hatto/Indjic mazurka sets which are
          outlined below:

          Probability that Hatto track times matched Indjic track times by
          random coincidence.
          All track times match between the two within about +/- 5
          seconds (due to time stretching of one of the sets and different amounts of silence padding). Assume that
          the probability that any one mazurka performance can be matched at random
          to within 5 seconds of another independent performance is about 75% likely. This is a very
          rough assumption, but it should be fair and conservative. Therefore the
          probability that all 49 mazurkas which I was interested in (there are
          about 10 more on the two disks) can be calculated by this equation:

               1 / 0.75^49 = 1 in 1,324,336
          
          

          Thus, the probability that the the track times match so closely is
          approximately one in a million. This cannot be caluculated from the
          r-value for the correlation between Indjic and Hatto track times directly. This is not scientific proof that the two
          recordings are the same, and it is not legal-level proof either, since
          one in a million does still can leave a reasonable amount of doubt.

          Probability that a single Hatto recording is the same as the matching
          Indjic recording
          . I use Mazurka in A minor Op. 17, No. 4 as an example
          in this case. I measured the downbeat locations in an independent
          manner. The accuracy of my timing measurement is 10 milliseconds.
          I then subtracted the difference between the two beat-time sets.
          There was a linear increase in the differences due to time shifting in
          one of the performances. This was removed. What resulted was a plot
          of timing differences between about 400 beats in the piece. There was a
          region of beat timing differences which was about 200 beats long, where
          the average beat timings were all less than 11 milliseconds. If a human
          were to try to tap to a constant beat, they would be expected to get less
          than 1/2 of the taps within 25 milliseconds (note that mazurkas are not
          performed at a steady tempo, so it actually should be a larger deviation).
          Being able to hit the beat timings at less than a 25 millisecond deviation
          for 200 consecutive beats is equivalent to flipping a coin and having it
          come up heads 200 consecutive times. The probability of that occuring
          is one in 1^200 or 1 in 10^59.

          This proves beyond a reasonable doubt that the two recordings are the
          same. 1 in 10^59 is approximately the same as one atom of hydrogen out of
          all other atoms of hydrogen in a star. And this is only the probability
          for a single performance on the CDs. Additionaly, if you were to examine
          the waveform for acoustic similarities (they have different durations
          and filtering, so are not exact copies), the probability that they are
          the same performance skyrockets much, much higher.

          • > The probability of that occuring is
            > one in 1^200 or 1 in 10^59.

            Ack, I meant:
            1 in 2^200, or 1 in 10^59

            1^200 is still 1. Anyway it was a typo, and I don’t want
            anyone pointing out the numerous spelling mistakes I made…

          • > Thus, a full analysis was not yet prepared in writing. That is our
            > excuse for it falling short.

            That a "full analysis" hasn’t been prepared yet is understandable. What is not
            understandable is the complete silence regarding data and methodology.
            What exactly was the raw data used? How was it acquired and processed?
            What was the algorithm employed to generate "timescapes"? This shouldn’t
            require a lot of time to deal with: if it has been published all you need to do is
            add the refs. If it’s a novel method, somebody’s on-going thesis work, then
            this needs to be said, and, once again, it shouldn’t take too much time or
            space to do so. Moreover, on your web page you don’t have the stylistic and
            space constraints of a journal article, so you don’t have to worry about
            formatting and you can post all the raw data you wish.

            > The importance of timescapes is that they were instrumental in the
            > discovery of the striking similarity between the two mazurka sets, and
            > they can thus place the debate of similarity on a level that even a
            > three-year-old can comprehend […]

            The importance of "timescapes" cannot be determined without understanding
            how they were generated, just as you cannot understand what an X-ray
            shows if you don’t know how it was generated. I said (and I still believe) that
            your article made the "most persuasive case", but my belief was based on a
            fundamentally unscientific assumption. Without the process, timescapes are
            really just pretty pictures, no doubt highly gratifying to a three-year-old, but
            falling somewhat short of that for an adult with minimal scientific literacy.

            > The information is not contradictory.

            It may not be, but I need a better explanation for Indjic/Rubinstein 1939 r
            values of 0.616 (Op. 17/4), 0.664 (Op. 68/3), and 0.954 (entire set).

            > Correlation does not imply causation.

            You cannot have it both ways. Either r says something, in which case you
            have to explain not only the values for Hatto/Indjic, but also those for Indjic/
            Rubinstein; or it does not, in which case, why was it featured at all? In fact,
            one could make a good case that r should not have been included in the
            paper to begin with, because we already know the variables are correlated —
            they are performances of the same musical piece! But it was your choice to do
            so.

            > What you wanted to see in the article are probabilities […]

            Actually, no, that was not what I wanted. To recapitulate, what I felt was
            missing was, (a) methods of data acquisition and processing, (b) methods of
            building timescapes, (c) discussion of apparent inconsistencies in presented
            data, (d) tables of all values used to generate timescapes and calculate
            Pearson’s coefficient.

            • > Moreover, on your web page you don’t have the
              > stylistic and space constraints of a journal article,
              > so you don’t have to worry about formatting and you
              > can post all the raw data you wish.

              The timescape pictures were generated from beat timing data found on
              the webpage:
              http://mazurka.org.uk/info/excel/beat
              (In MS Excel format, and a simpler text file containing same data).

              > if it has been published all you need to do is add the refs.
              > If it’s a novel method, somebody’s on-going thesis work, then
              > this needs to be said

              (1) It has not been published yet :-), and (2) it is a novel method.
              (3) William Barrington-Coupe has admitted to fraud, so no need to dot
              anymore t’s or cross anymore eyes in the aborted article.

              If you are interested, there is a description of a predecesor method
              call "keyscapes" which I described two years ago in this article:

              Sapp, Craig. "Visual Hierarchical Key Analysis" in Computers in
              Entertainment [http://www.acm.org/pubs/cie.html], volume 3, no. 4
              (Oct 2005)

              Also an earlier description is found in the Proceedings of the
              International Computer Music Conference 2001. I adapted the method for
              performance analysis this summer with only one minor modification in
              the methodology.

              The application for performance analysis will be presented next month
              in the CHARM Symposium 4: Methods for analysing recordings which will
              be held just outside of London on 12-14 April:
              http://www.charm.rhul.ac.uk/content/events/symp4.html

              An article that I am writing on the method will appear in an issue of
              Musicae Scientiae towards the end of this year:
              http://musicweb.hmt-hannover.de/escom/english/MusicScE/MSstart.htm

              Also, I will probably submit an article related to timescapes to the
              ISMIR 2007 conference, so you might be able to find it on their website
              in the Fall: http://ismir2007.ismir.net

              —————————————————————–

              Timescape methodology in brief: Take a 1-dimensional string of numbers
              (beat tempo or beat durations in this case) and perform pair-wise
              correlation (Pearson) between a reference performance and the other
              performances’s data. The correlations are done in increasingly larger
              sub-sequences of data, until the entire data set pairs are correlated to
              each other.

              For example, if there are 6 numbers (A,B,C,D,E,F) in the performance
              sequence, these sub-sequence pairings are used to calculate correlation
              values:
              AB, BC, CD, DE, EF, ABC, BCD, CDE, DEF, ABCD, BCDE, CDEF, ABCDE,
              BCDEF, ABCDEF
              These correlations are done between the reference performance and the
              other performances being compared to the reference performance.

              Next, a 2-dimensional plotting diagram is filled from small-scale
              subsequences at the bottom to the complete sequence correlations at
              the top. For the example sequence, this is approximately how the 2d
              plotting area is arranged:
              ABCDEF
              ABCDE, BCDEF
              ABCD, BCDE, CDEF
              ABC, BCD, CDE, DEF
              AB, BC, CD, DE, EF
              (Imagine each line being centered, since the spacing is
              altered due to web formatting)

              For correlation, the sub-sequences A, B, C, D, E, and F are not
              interesting, but for other sub-sequence calculations (such as average
              tempo instead of correlation), I might add them to the bottom row.

              Now, each cell in the plotting area is assigned a color. The color
              is given by which other performance has the highest correlation value
              to the reference performance. For example, if there were 100 performances
              and 1 of them is the reference performance for the plot, then there are
              99 other performances that were correlated with the reference at
              sub-sequence BCDE. I would look at all 99 of those correlation
              values and place a color in the BCDE slot in the timescape which
              represents that performance.

              In the Hatto/Indjic case, nearly all sub-sequences of their independent
              beat duration/tempos correlated better than any of the other 30
              or so performances by other pianists. The only ones which didn’t
              match were the short sub-sequence correlations at the bottom of the
              pictures which can randomly generate high correlation values by chance
              since they are short sequences:
              http://mazurka.org.uk/ana/pcor
              Also, for those interested, I have generated "dynascapes" this week
              which compare beat loudness (dynamics) between performances:
              http://mazurka.org.uk/ana/pcor-gbdyn
              Source data for these plots are found at:
              http://mazurka.org.uk/info/excel/dyn/gbdyn
              My focus group of three-year-olds can still pick out the Hatto/Indjic
              pairing in these pictures :-).

              Various slide shows which include some nicer figures can be found at
              http://mazurka.org.uk/info/present

              > timescapes are really … pretty pictures
              Thanks, I agree 🙂

              > > Correlation does not imply causation.

              There can be many reasons why the r-value is high, and it is all the
              more confusing since they are related to the same composition, as
              you point out. The probability calculations give in my previous entry
              are a more stronger basis for demonstrating plagiarism, since one of
              them demonstrates that it is not humanly possible to imitate another
              performance to such a high degree of accuracy. In other words, p-value
              calculations are difficult for this type of performance analysis, and
              I am interested in anyone who has suggestions…

              > (a) methods of data acquisition and processing,

              The data acquisition methodology is also novel and would take
              too long to describe here in the margins :-), but basically
              I am writing my own plugins to this free audio analysis editor:
              http://www.sonicvisualiser.org
              My plugins:
              http://sv.mazurka.org.uk/download
              If you own a copy of Chopin mazurkas for which I have extracted
              data, you can load one of those performances into the editor,
              and then download one of the annotation layer files from
              the webpage (Sonic Visualiser can load a file directly
              from a web address):
              http://mazurka.org.uk/ana/markup
              Such as the beat times in Joyce Hatto’s performance of Mazurka 17/4:
              http://mazurka.org.uk/ana/markup/pid9073-15/contents.html
              http://mazurka.org.uk/ana/markup/pid9073-15/pid9073-15-avgtap-sv.txt

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.