Most people recon impact would be very small. See links below. You might need to test it out and see whether it is worth doing that. But for the sake of backup, it is definitely worthy. Unless there are users on VMs 24/7, I doubt anyone would notice the performance impact when they are being backed up at night.
By markachtemichuk - http://virtualizationeh.ca/2010/06/04/performance-penalty-for-change-block-tracking-cbt/
http://www.yellow-bricks.com/2009/12/21/changed-block-tracking/