I have to say, I absolutely loved this story. It completely exceeded any expectations I may have had going in, and I'm really looking forward to reading more or Kiernan's work.
The story-crafting was excellent, especially with the way the two threads came together at the end, and the surprises just kept coming (I admit it didn't occur to me that the violin had been made from the remains of the collector's victims). As a former paleontology enthusiast I really appreciated the digression on ammonites, which, turned out to be one of many Lovecraftian references/homages. This is weird fiction exactly as it should be!
I think my one criticism is that I would have liked to get more about the connection both characters (and especially the collector) had to the sea. The violinist's description of the black-eyed things lurking in the depths was maybe my favourite part of the story, and I was hoping for more. But perhaps that would have made it too Lovecraftian, and Kiernan certainly did a great job of forging her own story while evoking the classics of the genre.
I suppose I don't have much to say about the details of the story, which mostly seemed self-explanatory (dare I say disappointingly so?) - apart from Captain Kangaroo, who as a Brit I had to google, and the musical elements (ballad in the title and the choice of song to play), which I'm not knowledgeable enough to comment on. The podcast episode covered things very well!
I did have a different interpretation of the evidence of the collector's murders though. I didn't read it as he had purposefully left the evidence for the police to find: I understood the "evidence" as part of the collection of strangulations. Not speaking from personal experience, but I suppose once you have strangled someone you don't have anything to keep (apart from the memories). So the body parts are to the murder collection as the fossils are to the ammonite collection - the physical element that can be held onto and (possibly) displayed. The journals detailing the murders are the documentation of the collection. Even though the collector says that he can remember the details of each and every murder, a true collector would have this kind of record-keeping. He probably has something similar for the ammonites (for which he says he can't remember the specific details of each). If there was supposed to be this kind of parallel, it would have helped cement it to know more about how/where everything relating to the murders was kept. But perhaps that means the author didn't intend this parallel.
I also thought the collector had intended to murder the violinist, shooting her somewhere non-fatal to weaken her before the strangling. but then shot himself when he realised he had "created" something so "perfect" that he would never top it. He had nothing left to achieve in his life.
Well, I'm talking in a disturbingly dispassionate way about murder and suicide (albeit hypothetical), so I'll stop now before I get reported for being some sort of sociopath myself.