This problem was finally resolved with the generous help of Bryn M. Reeves on the inux-lvm@redhat.com list.

The difficulty was that the utility kpartx had the lvs opened via their mappings. Further, the mappings were created with the -pp option of kpartx and that option had to be provided to the delete action in order for that to work:

# kpartx -d -pp /path/to/logical/volume/name

Once the mappings were removed then the lvremove also worked and the volumes were removed.

A small point, failure to provide the -pp option to the kpartx -d action does not raise an error. Nor does it remove the mapping however.