Best Practices for Migration of Archived Data
November 19, 2013 No CommentsFeatured article by Sam Elbeck, Sr. Director of Business Development and Client Services, Globanet
Industry Overview
Over the last decade, we have witnessed the inception and evolution of the archiving industry. The year 1999 marked the unofficial start of the archiving industry, when just a few software companies began competing to provide these solutions. The development of the archiving industry was spurred by some notable events: compliance regulations that required organizations to keep electronic data for mandatory durations; explosive growth in electronic data; the need for controlled and predictable storage costs, and the need to accurately discover against electronically stored data.
Over the years, additional market forces were exerted, new requirements emerged, the archiving industry matured, while Enterprise Class offerings were being developed. Now that archiving has reached maturity phase, client mindsets have changed as they have become more familiar with the industry offerings. Clients are no longer trying to grasp the concept of archiving; they are now trying to find the right technology to meet their requirements now and for years to come.
Why Organizations Migrate
Early adopters of archiving technologies are finding that their current systems no longer meet their needs, so they choose to migrate. There are several factors that might drive an organization to migrate its data:
– Mergers and acquisitions often necessitate the integration of incompatible archiving applications.
– WORM devices have become a costly, inefficient way to archive data, so they are being replaced by faster, more flexible and less expensive technologies.
– Enterprise Class Cloud-based storage offers scalability, data redundancy and disaster recovery support, while reducing hardware and personnel costs.
– The desire to retire legacy systems and retain old data.
Data Migration Best Practices
Regardless of the reason for migration, to ensure a successful process, an organization must follow industry best practices.
Migration of data archives requires a significant amount of planning and expertise to ensure it is done accurately, with data integrity (chain-of-custody) and quickly. There are steps that an organization can take before, during and after a migration to reduce risk, lower costs and shorten the time frame.
Pre-Migration steps
Project Planning. Consider three factors in the planning process:
- * Dependencies: Is the data migration dependent on any other efforts or are other efforts dependent on the data migration?
- * Retention: Have the current retention parameters been maintained?
- * Pre-requisites: Were any dependencies identified and explored?
– Communication Plan. Determine how you will inform stakeholders of the migration, its temporary impact and long term benefits.
– Testing Plan. It is vital to develop a testing plan to validate reports, message meta-data integrity, and time frames.
– Special Circumstances. Find out from folks in Legal and Compliance how to handle special circumstances such as items on legal hold, items belonging to terminated users, and items that cannot be migrated.
– Legacy Archive Health Check. If possible, conduct a health check on the existing archive system to identify problems with the source.
– Cut-Over. Define a cutover date, after which the legacy systems will no longer receive data, and then perform a full backup of the environment to protect against a system failure.
During the Migration
– Stick to the Plans. Closely follow the migration design and the communication plans.
– Vendor Reports. Expect regular reports from the vendor including issues found, issues resolved, migration performance, and end user experiences. Suggest regular status meetings if the migration vendor does not.
– Backups. Allow time for regular backups of the environments, which may require migrations to be paused.
– Change Control. Inform the migration vendor of any regular maintenance windows, emergency outages or unplanned changes that impact the migration.
– Expect Failures. Archived data tends to have items that can no longer be retrieved in the current system for various reasons, such as corruption. Most migrations have some failures – make sure the migration vendor explains how failures will be handled.
– Expect Road Bumps. Variables in archiving technologies, hardware configuration, network security, backup method, database health, etc. make migrations unique. Therefore, the archiving vendor might make slight adjustments during the process.
Post-Migration
– Review Reports. Review all chain of custody reports with special attention to items that failed to migrate. Work with the migration vendor to determine whether these failed items can be captured or migrated by some other means, and document them for Compliance and Legal departments, as to what is missing, and why it could not be migrated. Save these documents and reports until the retention of the failed items has expired.
– Migration Acceptance. Document the acceptance of or additional requirements for project completion.
– Delete Legacy Data: Delete the original data, as well as any backups of the original data, unless provided instruction to the contrary by the organization’s Legal department.
Large-scale data migrations can be challenging but with the appropriate planning and careful execution, organizations can greatly reduce the risks and costs associated with these projects.
The rewards of having all your archives on one platform outweigh the cost. Beyond the obvious benefits of standardizing on a state-of-the-art platform that will be around for many years, clients most frequently cite streamlined management of just one system instead of several and an enhanced ability to perform efficient searches against for all archived data simultaneously as the main benefits.
Sam Elbeck is the Sr. Director of Business Development and Client Services at Globanet, where he is responsible for all projects delivered by Globanet’s highly skilled technical resources; including designs, implementations, data migrations and many other services that Globanet provides. Sam is also responsible for developing new service offerings based on industry requirements. Prior to joining Globanet Sam spent three years at Symantec as a Lead Architect designing archiving, compliance and data management solutions for some of the largest organizations globally. Sam moved into a Solution Strategist role where he also spent a considerable amount of time discussing customers’ business issues and solving them using the latest technologies. Prior to Symantec, Sam spent nine years at IBM where he started his technical career in administration and architecture of messaging infrastructures. He then went on to build a team of engineers that specialized in data management and archiving solutions. Sam carries an MBA with a concentration in Management Information Systems and holds an undergraduate degree in International Business as well as several other technical certifications. Sam can be reached at selbeck@globanet.com or through www.globanet.com.