Best Practice Guide for Email Archive Migration

United Kingdom | Best Practice Guide for Email Archive Migration
United Kingdom | Best Practice Guide for Email Archive Migration

Best Practice Guide for Email Archive Migration

PREFACE

Email archiving solutions are a mainstay of the enterprise environment, particularly those in regulated industries or subject to ongoing eDiscovery requirements.

These environments have been subject to the same changes that have rippled through other facets of the online business world. In particular, the maturation of cloud-based archival and messaging platforms has seen organisations move to take advantage of their enhanced capabilities.

Email archive migrations are complex undertakings and there are numerous risks associated with a poorly planned migration. For instance, manual migrations are often hampered by the native tools of legacy archive platforms with out-dated APIs, making extraction slow and difficult. As such, Insentra has prepared this paper to illustrate in greater detail the challenges that may arise in such migrations.

Some common causes of data migration failure which can be avoided with a little forward planning include:

  • The appearance of unanticipated or unknown data formats
  • Insufficient or absent documentation for legacy systems
  • Corruption of legacy data
  • Breaking of Chain of Custody
  • Legacy data that does not mesh well with the new platform
  • Dramatic underestimation of migration time and cost
  • Poor user experience
United Kingdom | Best Practice Guide for Email Archive Migration

In this overview, we offer key recommendations on preparing for the task, the risks to be aware of and how best to measure the success of your migration project.

We trust this information is of value for your organisation as it embraces the benefits of a well orchestrated transition to your target platform.

REASONS YOU MAY NEED TO PERFORM AN EMAIL ARCHIVE MIGRATION

Simply put, an email archive migration is the process of copying the content of an existing email archive platform to a new target platform. Despite sounding straightforward in concept, email archive migrations are often complex, involved and lengthy. Each organisation often has different requirements which define the parameters and outcomes of the task, and where the complexity lies is in these nuances.

Mobile Application Management (MAM) app protection policies allow you to control your company data within your applications. Many Microsoft and third-party apps are supported and can be managed by Intune MAM. The official list can be found here: https://docs.microsoft.com/en-us/mem/intune/apps/apps-supported-intune-apps

Many factors create the need for migrations, including:

  • Changing the underlying storage platform of the current archive. Perhaps the current platform has reached End-of-Life (EOL) and is no longer supported. It may have unresolvable recurring issues or its feature-set may no longer meet your organisation’s needs.
  • Consolidation of environments in the event of a merger or acquisition. When organisations merge there is often an archive solution needing to be consolidated.
  • Maintaining multiple instances of the same archiving platform (or a few different archiving platforms) results in high management costs and inefficient use of resources – not to mention a poor user experience.
  • Splitting a subset of the content to another location (de-merger) OR,
  • Migrating to the cloud. Software as a Service (SaaS) email archive solutions (i.e. Google, Mimecast, Microsoft365, Enterprise EV.Cloud, etc.) present organisations with the option of no longer deploying and managing their own archive solutions. This can result in cost-efficiencies including the benefits of unlimited storage often at a fixed per-user price.

The proprietary nature of legacy archive platforms means that moving between different platforms is not as simple as just copying files.

Email archive migrations can be completed either manually or can be automated via the use of specialist third-party software tools.

MIGRATION INGESTION PROCESS TO M365

Microsoft MDM vs Intune

United Kingdom | Best Practice Guide for Email Archive Migration

We will be looking at the value of automated solutions shortly, but for now, let’s tackle the essential aspects you’ll need to consider before you delve too deep into your data.

WHAT SHOULD I PLAN FOR WHEN MIGRATING?

Ensuring a smooth migration relies on your organisation getting the details right. The following factors will have a direct impact on the success of the process:

United Kingdom | Best Practice Guide for Email Archive Migration

KNOW WHERE YOU’RE COMING FROM AND WHERE YOU’RE GOING

It’s crucial to have an in-depth understanding of your legacy archive environment. How far does the archive go back? What types of data did it capture (i.e. emails, calendar appointments, attachments. etc.)? What types of metadata did it capture? How much of this data do you need to transfer?

Knowing this information will provide an insight into the data types you will be migrating and whether your archive infrastructure will be able to accommodate the extraction process. You need to understand your target environment as well:

  • Will it be cloud or on-premises?
  • What types of data formats will it accept?
  • What is the maximum rate of ingestion?
United Kingdom | Best Practice Guide for Email Archive Migration

ASSESSING THE PARTICULARS

Conducting an adequate pre-project assessment isn’t just a good idea, it’s a must if you want to effect a successful migration. It should provide you with a detailed picture of how much data exists in your archive, whether there is data-junk, which can be discarded rather than migrated, and whether there are PSTs requiring consolidation.

Crucially, a pre-project assessment provides insight into any potential legal and compliance risks including whether Chain of Custody needs to be maintained (see box-out). Other questions it should resolve are the overall approach – (i.e. manual or automated migration, project costs and timeframes.)

United Kingdom | Best Practice Guide for Email Archive Migration
United Kingdom | Best Practice Guide for Email Archive Migration

CHOOSING YOUR MIGRATION PATHWAY

There are several different options when considering an archive migration, each with their own strengths and weaknesses.

PARALLEL SYSTEMS AND NO MIGRATION:

A parallel process keeping both old and new systems running concurrently allowing the old archived data to be expired over time and not be migrated.

Benefits: the new archive is available as soon as possible at minimum expense and the legacy archive is accessible and can be quickly reverted to in the event of problems.

Downsides: End-users have two systems to search for content, existing shortcuts will not always work and ongoing support and maintenance costs will need to be paid to maintain the legacy system.

SELECTIVE MIGRATION:

Also known as a partial migration where a smaller group of users or subset of data is migrated rather than the entire archive and the old archive is left in situ or removed.

Benefits: The ability to minimise the volume of data being migrated therefore reducing time and costs and the enabling of end-users to access their data from a single location.

Downsides: If access is required to ALL the data, two locations will still need to be managed, maintained and searched from (thus making eDiscovery cumbersome).

FULL MIGRATION:

All legacy data is migrated using a phased approach and the old system is decommissioned following verification of the migration.

Benefits: Old platform can be decommissioned and all data is located in a single place.

Downsides: Change for the organisation, upfront costs.

United Kingdom | Best Practice Guide for Email Archive Migration
United Kingdom | Best Practice Guide for Email Archive Migration

PLAN THE WORK, WORK THE PLAN

Once there is a basic understanding of the task ahead (how) it’s time to formulate an overview of what should happen, when, and who is responsible for its implementation. The plan should also feature a realistic breakdown of the estimated resources and work effort needed and a clear statement of purpose around why the migration is being undertaken.

Solicit advice from your compliance department and integrate their requirements into your plan. Additionally, to ensure everyone has a clear picture of what to expect when to expect it and who to contact should the unexpected occur, communicate plan to all parties likely to be affected.

United Kingdom | Best Practice Guide for Email Archive Migration

COVER YOUR BASES WITH A RECOVERY PLAN

Finally, have a robust contingency plan for those times when a problem emerges which cannot be readily addressed. Not having an option to roll back to a reliable ‘known state’ could prove costly and incredibly disruptive for your end-users.

WHAT SHOULD I PLAN FOR WHEN MIGRATING?

This term refers to the reliable recording of processes and procedures which occur while evidence (physical and/or electronic) is being captured, held, transferred or disposed of. For organisations with strict industry compliance regulations and internal data management policies, maintaining Chain of Custody is vital. When migrating, organisations must be able to demonstrate the data has not been altered in anyway as it transits from one archive platform to another.

Automated migration tools and services include a complete auditing of the migration process which can demonstrate complete ‘Chain of Custody’ for data subject to legal or regulatory requirements.

United Kingdom | Best Practice Guide for Email Archive Migration

COMMON QUESTIONS ABOUT EMAIL ARCHIVE MIGRATIONS

USER EXPERIENCE CONSIDERATIONS

Managing the impact to end-user experience should be a key consideration when migrating archive data. This is generally the greatest measurement for success or failure when undertaking a migration. Archives may be moved correctly, but if a user is not prepared for the changes to come, or has issues during cutover (such as accessing their emails or a policy change which was not communicated properly) then their perspective will be that the project was unsuccessful and provide feedback accordingly.

To ensure this does not happen, proper stakeholder management and communication is paramount. Ineffective communication during the planning phase could result in missing critical business processes from discovery. Similarly, not communicating upcoming changes with users means they may not know how to do their jobs in the new environment. The best way to do this is to build a communications plan which maps out the messages over the course of the migration. Here are some examples below to help get you started:

  • Communicate upcoming technology and features (i.e. What are the cool new things users can do in the new environment that enhances what they already know and love?)
  • Map out the communication frequency – a common one is
    • T-14
    • T-7
    • T-3
    • T-1
    • T-0
  • Along with communication frequencies and showing the users the cool new features, there are some unique advice that will need to be provided when it comes to archive migrations, these can include the following;
    • New archive location and name change (i.e. Your new archive will appear in Outlook underneath your usual mailbox on the left-hand side widget. The archive can be distinguished by the following: “Online Archive – ”
    • If there are stubbed emails/shortcuts (i.e.
      emails with the legacy archive environment
      icon assigned), properly communicating
      with the end-users about what will happen
      post-migration will undoubtedly minimize
      many calls/tickets to the IT Support team

BUT WHAT ABOUT PST FILES? CAN’T I JUST USE THEM TO MIGRATE TO A NEW PLATFORM?

There are many issues with a manual PST migration including, but not limited to:

SLOW AND MANUALLY INTENSIVE

Typically, native archive extraction and import tools are single-threaded and need to be manually overseen. Seldom can multiple mailboxes be scheduled to run and they certainly cannot be left unattended.

The impacts on resources such as time, staffing and budget are significant.

LITTLE TO NO ERROR MANAGEMENT

If a PST extract or import fails, the process generally stops with no indication of the underlying problem. Rarely is there the ability to either identify and skip the failed items or to resume where the process left off.

LITTLE TO NO LOGGING OR AUDITING

When extracting to PST there is seldom a record of which data has been moved. All checks must be done manually to ensure that all messages in the source have been extracted to the target which is extremely time-consuming and prone to human error.

PST’S RISK THE SECURITY AND INTEGRITY OF YOUR DATA

Coupled with the lack of built-in checks, the multi-step process extracting data to interim storage not only requires the storage to be available but also leaves the extracted data open to tampering or corruption. Chain of Custody is difficult to maintain, making a PST-driven migration unsuitable for organisations that have compliance or regulatory requirements.

LOSS OF COMPLIANCE DATA

When using a journal archive it is likely that ‘envelopes’ containing BCC and distribution list recipients will have been stored in the archive and will require recreation following extraction. Manual extraction to PST files does not allow for the recreation of this envelope information. Accordingly, BCC and distribution list data is lost and cannot be searched in the target archive.

LARGE PST FILES ARE PRONE TO CORRUPTION

The standard tools shipped with an archive product typically don’t allow for the split of a mailbox into multiple small-sized PST files. As such, when extracting large mailbox or journal archives, PST files can become quite large and prone to corruption. When this occurs, the source data will need to be extracted again and often the issue will recur.

REQUIRES INTERIM STORAGE

Most email archive platforms provide single instance storage, (i.e. only one copy of a message is stored in the database regardless of the number of recipients.) For example, if a mail message with a 10MB attachment was sent to 15 people, it will have been stored in the archive as one 10MB message. Depending on the source archive and/or storage platform, this message may also have been compressed. When extracting this data to PST, a copy of the message will need to be extracted into each recipient’s PST file. Therefore, the message that was previously 10MB will now become 150MB when extracted to 15 PST files. This loss of single-instance storage often means that the PST extraction is, at minimum, 2.5 times the size of the original legacy archive.

LIMITED OR NO FLEXIBILITY IN WHAT IS EXTRACTED

When extracting to PST files, typically all mailbox data will be extracted without an ability to filter or be selective over what is extracted and then ingested to the target.

United Kingdom | Best Practice Guide for Email Archive Migration

COMMON QUESTIONS ABOUT EMAIL ARCHIVE MIGRATIONS

WHY DO JOURNAL ARCHIVES REQUIRE SPECIAL CARE?

The majority of organisations capturing email into Exchange journal mailboxes tend to do so for compliance reasons and typically use the ‘Envelope Journaling’ feature. This feature was developed by Microsoft as a way to preserve vital header information including the BCC recipients and the expanded members of any distribution lists. As mentioned in the previous section, preservation of this data and making it available for access when performing eDiscovery is pivotal to providing a compliant and defensible archive migration.

Importantly, different archive platforms store the envelope information in different ways. Some may store the information separately in the archive index while others may store it in the archive store itself. This creates a requirement to recreate the envelope information at the time of migration – something which may not be (and often is not) supported by the native extraction tools of the source archive.

Additionally, journal archives tend to be very large data repositories – after all, they contain every mail sent to and from people in an organisation. This makes manual extraction to PST files very slow. Moreover, with many archive platforms, it is not actually possible to manually extract the envelope information nor to extract each message for each user. Migration software tools allow journals (and large mailboxes) to be split into a number of separately handled virtual mailboxes of a user-defined size for better allocation of computing resources and also address the expansion of the envelopes.

If the target platform is Exchange Online, an automated migration will ensure all the source objects are migrated (including the BCC and distribution list information contained in the envelopes) and are migration compliant with Microsoft’s requirements – user mailboxes can only contain the owner’s data. Migration software tools achieve this by allowing messages to be virtually split into “user” allocated containers and then migrated into Exchange Online.

United Kingdom | Best Practice Guide for Email Archive Migration

HOW WILL SHORTCUTS BE HANDLED? WILL THEY WORK IN THE NEW ENVIRONMENT?

Many archive platforms provide the ability to leave a “shortcut” or “stub” in the mailbox – typically a smaller footprint of the original mail object (perhaps the attachments are stripped). These items need to be addressed as their own “type” of message.

Importantly, if shortcuts are migrated as a part of the primary mailbox migration, there is a chance they may NOT work or will stop working after a period of time. Shortcuts need to be handled with “kid-gloves” > and either be removed or converted, the latter of which can only be done via an automated approach.

An automated migration service ensures shortcuts are seamlessly converted as part of the migration allowing them to work in the new archive environment. Alternatively, they can be removed if they are no longer needed.

United Kingdom | Best Practice Guide for Email Archive Migration

COMMON QUESTIONS ABOUT EMAIL ARCHIVE MIGRATIONS

HOW LONG WILL OUR MIGRATION TAKE?

This is one of the more difficult yet most common questions. Every environment is configured and performs differently with the main variables affecting migration performance being:

Available network bandwidth

  • Speed of the storage subsystem on which the legacy archive sits as well as the
    destination storage (and temporary storage where required)
  • The ingestion performance of the target archive system
  • The scheduling of other projects such as the commissioning of the target
    environment
  • The organisation’s ability to transform

When performing automated migrations, it is typical to refer to speed in terms of GB per hour. The average performance of an automated migration is approximately 20-30 GB per hour, per archive server, with speeds of up to 2TBs a day achieved. However, the speed at which migrations will take place is often not determined until the migration has commenced and core data has been gathered

HOW WILL I KNOW IF MY MIGRATION IS A SUCCESS?

The key indicators of a successful migration are as follows:

  • Little to no end-user impact during the migration (effective end-user management)
  • More than 99.9% of the readable data has been migrated
  • Chain of custody has been maintained
  • Timeframe to migrate data was reasonably met given the health of the environment
  • Internal resources have not been overly consumed by the project
  • Corrupted data has been eliminated via the migration
  • Project performed cost-effectively
  • Minimal impact on mail and data retrieval and the daily operations of staff
  • Archived data migrated into the new platform and support is in place for future data management and retrieval

It is important to note that a manual extraction cannot address the bulk of these success criteria – a significant reason why more organisations adopt an automated approach to email archive migrations.

COMMON QUESTIONS ABOUT EMAIL ARCHIVE MIGRATIONS

FAILED OR SKIPPED ITEMS

When migrating email archive data, there will more than likely be ‘failed’ or ‘skipped’ items. There are two failed data categories: temporary and permanent.

‘Temporary’ failures are usually due to environmental issues such as poor network bandwidth or high load on the legacy archive. In such cases, automated migration tools will automatically re-process the relevant item(s) a specified number of times and/or at different times of day until the object has been migrated.

‘Permanent’ failures are usually attributable to pre-existing problems in the source archive (i.e. not caused by the migration process). As such, it is likely these items would not have been readable by a user or any audit/eDiscovery process. This kind of failure happens fairly infrequently – typically about 0.01-0.1% of the overall archive content.

‘Skipped’ items typically occur when there are items in the source platform that are of a type or format not supported by the target and where these items cannot be altered to be ingested. These issues typically occur when migrating between Lotus Notes and Microsoft Exchange messaging platforms.

United Kingdom | Best Practice Guide for Email Archive Migration

CAN I PERFORM MY OWN MIGRATION, OR WOULD I BENEFIT FROM A MIGRATION PARTNER?

Theoretically it is possible to perform your own migration, however, experience has shown that even automated migrations require the involvement of an experienced and dedicated archive migration specialist. Such specialists are certified in the software tool being leveraged to complete the migration and know the traps to avoid. Thorough knowledge of both the source and the target is critical.

Each migration is different with many configuration permutations and combinations needing to be tested to ensure both the optimum migration throughput and outcomes. A migration specialist will have the necessary skills and experience to deliver a successful outcome, often providing an end-to-end service, ensuring that all source data has been migrated to the target.

Some organisations consider contracting a specialist firm to provide design, implementation and initial user migrations and seek to perform the balance of the migration themselves. Whilst this is a viable approach, it is not recommended. During migration, issues will inevitably arise that require expertise in the source and target platforms as well as in the tools to identify and isolate any problems. Troubleshooting these problems without the depth of experience and knowledge takes significantly longer than if the migration specialist provides an outcome-based service.

Often, the additional cost of having a partner perform the end-to-end migration versus doing a part yourself is marginal, with the majority of organisations preferring to leave the migration to specialists who can take the risk and own the project end to end (i.e. all readable data migrated successfully).

If the target platform is Exchange Online, an automated migration will ensure all the source objects are migrated (including the BCC and distribution list information contained in the envelopes) and are migration compliant with Microsoft’s requirements – user mailboxes can only contain the owner’s data. Migration software tools achieve this by allowing messages to be virtually split into “user” allocated containers and then migrated into Exchange Online.

United Kingdom | Best Practice Guide for Email Archive Migration

HELPING YOU MAKE THE RIGHT CALL

Archive migrations are complex processes and the decision on approach must be a considered one. All requirements must be well understood and all implications deliberated in order to capture business and technical requirements to facilitate a successful outcome.

As such, we recommend:

  • Adequate pre-planning, oversight and resource allocation for the task
  • In most cases an automated migration ensures Chain of Custody and a costeffective outcome, versus a slow, non-compliant and prone to error manual migration.
  • Partnering with an experienced migration specialist who can offer guidance and support, removing the difficulties of the process and minimizing risk to the organisation.

There is no need to get email archive migrations wrong and every reason to get them right. The business risks of a poorly conceived migration are simply not worth it.

KNOWING YOUR NEXT STEP

Archives are a necessity in helping to manage the explosion of data typical in today’s information-rich environment.

When properly implemented, cloud-based archival storage can provide a great boon for efficiency. As such, it pays to get it right. Always seek the best advice and ensure it gets done correctly from the get-go.

Know your reasons for migration. Know what data needs to be moved and how best to move it. Identify any limitations in your infrastructure and your compliance requirements.

Knowing where you’re coming from will make it that much easier to see where you are going with your migration.

Any questions? Insentra can answer them.

United Kingdom | Best Practice Guide for Email Archive Migration

DOWNLOAD BROCHURE

United Kingdom | Best Practice Guide for Email Archive Migration
United Kingdom | Best Practice Guide for Email Archive Migration
Loading Form...

THANK YOU FOR YOUR SUBMISSION!

United Kingdom | Best Practice Guide for Email Archive Migration

 We’ve sent a copy to your inbox. Remember to mark as a “safe sender”, and to check any junk or spam folders so you receive your copy.  

If you’re waiting for a sign, this is it.

We’re a certified amazing place to work, with an incredible team and fascinating projects – and we’re ready for you to join us! Go through our simple application process. Once you’re done, we will be in touch shortly!

Who is Insentra?

Imagine a business which exists to help IT Partners & Vendors grow and thrive.

Insentra is a 100% channel business. This means we provide a range of Advisory, Professional and Managed IT services exclusively for and through our Partners.

Our #PartnerObsessed business model achieves powerful results for our Partners and their Clients with our crew’s deep expertise and specialised knowledge.

We love what we do and are driven by a relentless determination to deliver exceptional service excellence.

United Kingdom | Best Practice Guide for Email Archive Migration

Insentra ISO 27001:2013 Certification

SYDNEY, WEDNESDAY 20TH APRIL 2022 – We are proud to announce that Insentra has achieved the  ISO 27001 Certification.