When I reported this to Pono, they said:
"... try emptying the Trash. On Macs files are cached in a .Trash folder on storage devices, so they are not completely removed from that device until the Trash on the computer is emptied."
Interesting. Not a solution for my Windows installation though. To make things more interesting, this only happens to two of the three albums I've purchased, and one of them fails to re-download every time.
Looking at this further, it may be because I fixed the release dates on all my tracks. Originally the date fields were empty. I changed them all to the actual release date, 1963, and now the software is downloading duplicates with 2014 release dates. Maybe if I delete the old duplicated tracks with the 1963 date, this problem will go away.
However: (a) the first time I touched the tags, some bug caused every track and album in my library to get the same title, and (b) one album's tracks don't have duplicates because it fails to re-download every time. Possibly there are more problems with other tags in that one.
I spent two months trying to get help or maybe a credit for replacement downloads from Pono, receiving dozens of clueless disconnected responses, before I finally just called their office. In the end, a support supervisor just gave me the tag info and told me to input it all manually. Good thing I have only three albums in there. Not looking forward to dealing with this again.