Here goes the sad story with happy end about VDP 6.1.2.19. One unlucky day I have fed a couple of OLAP VMs to VDP and it choked failing to deduplicate properly. It have had ended up with nodes full at 98, 97 and 98 % respectively.
I do not know what exactly helped but here is a full list of my actions (add reboots as needed):
1. Delete big backups, run manual checkpoint, integrity check and garbage collection. Everything failed with MSG_DISK_FULL.
2. Rollback to earlier checkpoint, run manual checkpoint, integrity check and garbage collection. Everything failed with MSG_DISK_FULL.
3. Modify configuration threshold amounts to allow garbage collection run (as described above). Same errors when trying to set values to 99%.
4. Expand storage! Wizard went successfully. Only node1(/dev/sdc1) expanded . Run manual checkpoint, integrity check and garbage collection. Only checkpoint succeeded, hfscheck and gc failed with MSG_DISK_FULL.
5. At this point I have up and let the system run for a weekend.
6. On Monday it has magically repaired itself. Thare was a good checkpoint, good hfscheck and good gc. Admin mode persisted although.
7. Several times rebooting and running manual checkpoint including unmounting disks and running xfs_check helped at last. Fullaccess mode was there.
8. The last bit was using xfs_growfs on /dev/sdb1 and /dev/sdd1 to fix wrong size of nodes.
Edit: I think that checkpoint rollback and waiting was enough...