Great podcast on this short story. I'm new to the weird and speculative fiction genre and am enjoying discovering these stories as you cover them.
Your points about the unreliable narrator are all very good. I still like it despite its flaws. For me it adds to the creepy factor as the narrator struggles to explain what he doesn't understand. The scene descriptions are my favorite part of this story, what great language.
I hadn't made the Ponce de Leon connection either, I thought the place was much older, say from the time when the continents were all one giant landmass.
A side note about the fountain of youth. Ponce de Leon wasn't looking for it, the term appears nowhere in the official documents for his expedition which were very explicit as to his goals. The fountain of youth story came out later, but would be the one Lovecraft knew.