Verification methods have been redesigned in the 9.0 release resulting in even more assurance of data integrity and completeness, while verification is even more tightly integrated into protection plan execution, copy, and purge processes.
The compelling strategy not only offers data and referential integrity checks at newly implemented levels, but stages new, permanent indexes used by DATASTOR tasks. Along with new improvements in store tasks in their own right, tight integration with data verification actually boosts task performance significantly.
Partial list of verification's impact
- Protection Plans use verification results to update plan indexes at startup time to remove any problem references identified during verification.
- Copy task execution benefits from new restore point indexes built by the verify process to quickly identify items to synchronize, then multithreads the copy processes.
- Purge task execution benefits from new restore point indexes built by the verify process for marked improvement in identifying items eligible for permanent deletion.
- Each restore point displayed in the calendar in the Restore context now displays restore point status as 'Available' or 'Available, Incomplete' with detail about any problem files in each restore point.
- The new Verify process improves the self-healing strategy of the software. In addition to the self healing process that primes protection plans to place good versions of quarantined items in the store, the verification process re-verifies all items placed in quarantine and restores items back to the store if verified successfully.
- Verification builds a semi-permanent global index of the store's common content that improves item identification in verify, copy, and purge. New protection plans load the global index to improve the first plan runtime.
Three Verification Levels
Verification of data content (data integrity) and verification that all referenced files are present (referential integrity) assures the consistency and health of restore points in a store. You can configure a store verify task through the Create Store Task menu and specify three levels of verification. The three levels of verification handle differently the verification of store Common Content, individual restore points, and plan indexes (that are used during plan runtime). See the chart below for the verification types performed at each level.
Verification types performed at each verify level
The verification levels are presented to the end user as radio buttons during the wizard to create a store verify task, or after completing the wizard by selecting the verify task's 'task settings'. Level 1 and 2 are available for selection during the 'add verify task' wizard, level 3 is only available from 'task settings' after wizard completion.
Backward Compatibility
Verification improvements take effect immediately upon upgrade to release 9.0. Existing stores created in the 8.0 release of the software will continue to operate without issue. However, to fully benefit from the new release we recommend converting stores in the 8.0 store layout to the 9.0 store layout.
Best Practices
- Create and schedule a level 1 verify task daily after protection plans complete. The first time the verify process executes, it will perform a data integrity check of the entire, latest restore point, and thereafter, just the new and changed data in the latest restore point for a reduced runtime. Level 1 verify builds the new restore point catalog.index file that helps prepare for an efficient purge and an efficient copy. Purge and Copy will build the required catalog.index file for each restore point if one doesn't exist.
- Run the level 1 verify task before any copy task executes. If you cannot run a verify task prior to the copy task, our new best practice would be to create and run the copy task on the source store, not the destination store, to avoid verification of items across a network which significantly degrades performance.
- Create and schedule a level 2 verify task weekly, bi-weekly or monthly. A level 2 verify is a re-verification of referential integrity to ensure all referenced files are present on disk. A level 2 verify will build catalog indexes for any restore point that does not have one yet, which benefits the purge and copy process, which requires them and will build them if not present. You might time a level 2 verify to run midpoint between scheduled purges. (It is a good idea to run a level 2 verify just after store conversion from build 8.0 to build 9.0 to ensure a consistent store.)
- Run a level 3 verify only if an inconsistent store is indicated, due to file system corruption, for example, due to the extensive time it would take to complete.
- Additional best practices include creating a single store per volume, sizing the volume according to storage considerations. This allows the store to be scanned in fast mode, which scans the file system's master file table. The store must also be converted to the 9.0 store layout or be created new in the 9.0 release to benefit from fast scan mode. Scan time can be 10 times faster in this configuration than when multiple stores reside on a single volume, greatly reducing verification scan time as integrated in the multiple DATASTOR tasks.
We'd love to hear your experience with the 9.0 release!