How does Aggregate Deduplication affect SnapMirror?

Applies to

• Snapmirror destination DP volumes
• Aggregate-level Cross-Volume-Deduplication savings

Answer

• Cross-Volume-Deduplication at the aggregate-level eliminates duplicate blocks across multiple volumes hosted at the same aggregate
  • These savings are achieved by an aggregate-level background efficiency-scanner that triggers automatically
• Cross-Volume-Deduplication savings on a Snapmirror source volume (R/W) aggregate are are not retained by Snapmirror transfers on the Destination volume (DP) aggregate by default in ONTAP when the relationship was established
  • Snapmirror maintains inner-volume storage efficiency deduplication savings from the source to the destination over the wire via the LRSE replication engine.
    • This does not require the State of volume efficiency to report Enabled on the DP volume at all (vol efficiency show -fields state can report Enabled or Disabled on the DP volume and it does not matter for the topic of this KB article)
  • After each Snapmirror update with the LRSE engine, an efficiency operation triggers on the DP volume by default. This gains volume level deduplication savings that had not been processed background on source yet. 
    • No dedicated efficiency configuration is required on the destination DP-volume for this to happen
  • The LRSE replication engine is in use by Snapmirror
    • When the destination DP volume has no dedicated efficiency configuration, or
    • When the efficiency configuration of the DP volume uses identical compression settings as the source R/W volume
• SnapMirror destination clusters can use aggregate deduplication with type-DP Snapmirror destination volumes optionally
  • This requires manual changes for efficiency configuration on the Snapmirror destination DP volumes, because cross-volume-deduplication is disabled by default on individual- or svmdr-DP volumes
  • A one-time manual volume-level deduplication full-scan (-scan-old-data true) is required on a DP volume, when Cross-Volume-Deduplication on a DP volume is enabled AFTER the snapmirror initialize has already completed or the destination DP volume is member of a svmdr relationship
  • Enabling Cross-Volume-Deduplication explicitly on Snapmirror destination DP volumes has no impact to Snapmirror LRSE engine to maintain volume-level storage efficiency savings from the source over the wire and does not require an  efficiency-schedule or -policy on the DP volume
  • To gain extra deduplication savings from Cross-Volume-Deduplication on Snapmirror DP volumes it is not require to set an auto- or scheduled- or threshold-based background operation efficiency policy on the DP volume, as long as the DP volume does not become R/W (snapmirror break).

Additional Information

Parent topic: SnapMirror storage efficiency configurations and behavior

To achieve cross-volume savings on a DP (Snapmirror destination) volume:

1. Enable cross-volume-efficiency on the DP volume prior snapmirror initialize

• Enable efficiency on the DP volume:
  volume efficiency on -vserver svm-destination -volume volume_dp
• Enable cross-volume-background-dedupe on the DP volume:
  volume efficiency modify -vserver svm-destination -volume volume_dp -cross-volume-background-dedupe true
• Initialize the Snapmirror relationship:
  snapmirror initialize -destination-vserver svm-destination -destination-volume volume_dp

2. If Snapmirror re-initialize is not possible, a full-scan is required on the DP volume to initialize its block-fingerprint database:

• This will take longer, because a full-scan takes a long time and can only consume limited CPU, so if possible prefer option 1
• Enable efficiency on the DP volume:
  volume efficiency on -vserver svm-destination -volume volume_dp
• Enable cross-volume-background-dedupe on the DP volume:
  volume efficiency modify -vserver svm-destination -volume volume_dp -cross-volume-background-dedupe true
• Run a full-scan on the DP volume:
  volume efficiency start -vserver svm-destination -volume volume_dp -scan-old-data true
• Wait for the full-scan to complete:
  volume efficiency show -vserver svm-destination -volume volume_dp -scan-old-data true -fields op-status

3.  For SVM-DR, option 1 is not possible, because the volumes are only created during snapmirror initialize.

• Use the same steps as in option 2, but make use of advanced command vserver config override for the non-show command:

set advanced

vserver config override -command "volume efficiency on -vserver svm-destination -volume volume_dp"

vserver config override -command "volume efficiency modify -vserver svm-destination -volume volume_dp -cross-volume-background-dedupe true"

vserver config override -command "volume efficiency start -vserver svm-destination -volume volume_dp -scan-old-data true"

With either option:

• Cross-volume-savings are achieved after next aggregate efficiency background operation had triggered.
  • Check output from aggregate efficiency cross-volume-dedupe show to determine last operation end or current operational state
  • An aggregate efficiency background operation is triggered automatically upon a certain percentage of block-changes within the aggregate
  • An aggreate efficiency background operation can get trigegred manually using: aggregate efficiency cross-volume-dedupe start
• An efficiency operation schedule for the DP volume efficiency operation is not required when LRSE (logical replication with storage efficiency) is in use.
• With LRSE in use, every snapmirror update will trigger an efficiency background operation (only visible in sis.log) to achive cross volume savings.
• The per-volume savings from the source-volume are maintained over from primary to secondary with XDP relatioonships using storage efficiency when there is no secondary compression enabled on the DP volume. Additional details:
  • What kind of SnapMirror transfer avoids the Data Warehouse rebuild?
  • Understanding SnapMirror storage efficiency