It just decided to work and they are really removed. I did not have to clear the cache or restart the phone or anything else.
So I went ahead and individually deleted about 20 downloaded podcasts and it worked as it should with all the deleted podcasts disappearing from my downloaded list. I tried again this morning and for some reason it decided to work, giving me the 'are you sure' choice. This morning all those podcasts were still on the device. I tried to remove several played downloads yesterday, and as you described, nothing happened. This problem exists for me exactly as it does for you. Unfortunately the answer about just disabling the offline availability and clearing the cache is incorrect. Can't remove downloaded podcasts from device or de.