I have to admit that when I read the story, I was a bit puzzled what was expected of me as a reader - I don't like stories with too much allusions (seems a bit elitist), and to me the schism between thoughts/narration and the action of and focus on Moore's story was a bit confusing. Furthermore, I missed a sense of atmosphere (like I also missed it in Amber and for example in stories like those of Isaac Asimov) - like most sf and speculative fiction they are rather experiments in social constellations. In some way Ray Bradbury is able to combine sf and a more personal, atmospheric type of story, though.
But I know now I'm not doing justice to these sf and speculative fiction stories. After listening to the two extensive podcast episodes on The Graveyard Heart, I have to admit I just should have read the story multiple times to understand it better. Your discussion showed there is a good reason Zelazny uses this bombardment of allusions, there is a reason he confuses the reader with the aforementioned schism.
Though I don't think this story is weird fiction, I had to think of two weird fiction writers (according to Lovecraft, that is) when you discussed the allusions to mythology (gods with a minor g) and religion (Catholicism).
Firstly, I had to think of Unger as a sort of Arthur Machen when he is lamenting about the hollowness of humanity when they become gods and art becoming empty. In his essay Hieroglyphics, Machen says: 'Man is a sacrament, soul manifested under the form of body, and art has to deal with each and both and to show their interaction and interdependence.' Together with Machens notion of 'ecstacy' (a vague notion, but some sort of required 'soul' in literature and art, maybe the Sublime), I think you can view Unger and Machen as persons who wanted to unify Catholicism with the Dionysian, a sort of mysticism as opposed to the 'clean' dogmatic and pragmatic Apollonian version.
Secondly, when you mention gods with a minor g in a weird fiction podcast, you almost automatically say Lord Dunsany. His world of gods is (superficially) a far more romantic one than for example Zelazny's worlds of Amber, but both writers use these gods as a frightening mirror of humans turning into gods or meddling with divinity - as S.T. Joshi calls it (about Dunsany's stories) it is 'Nietzsche in a fairy tale'. As you said in the podcast, those gods are perfect on the outside but also 'soulless' when it comes to ethics, and the humans in The Graveyard Heart are heading the same way, according to Unger.