Migrate Your Website Without Affecting Search Engine Visibility
This entry was posted on December 11, 2019 by Nenad Mihajlović, SEO Specialist.
In order to explore the subject of this article, we have to define site migration first. Site migration is an event where the website undergoes substantial changes like changes in the site’s location, platform, structure, content, design or UX. Poorly implemented site migration will result in significant traffic and revenue loss, which can last from a few weeks to several months.
If site migration is done with temporary loss of traffic, or without losing traffic and revenue, it's considered to be a successful migration. Every other outcome is considered to be an unsuccessful migration.
How Search Engines Work
To understand the consequences of unsuccessful migration you need to know how search engines work. We will take the Google search engine as an example.
For the sake of simplicity, we will say that the Google search engine is just a pattern detection program. When you search for a specific term, the search engine shows all the pages that match the pattern that is related to your search term.
When you do a search on Google you are not searching the live web, you are searching Google’s index of a web. It is a stored copy of the website that Google has crawled.
Google crawls every web page of your website using programs called spiders. As a spider crawls your page, the copy of the page is stored on Google servers, which is Google index.
Then Google calculates the relevance and ranking. Every page is ranked for specific sets of keywords or long-tail keywords and gets its authority score.
Based on the rank, authority, relevance, and many other factors, the page will be presented on the search engine result page (SERP).
Fig.1 How a search engine works
How does all of this work practically?
If you are in eCommerce business and you are selling product A, this is how it works. When someone searches for “product A price” or “product A reviews”, Google will quickly pull all the data from its index related to the search term, and it will present them on SERP.
In the best-case scenario, your website may appear on the first page of SERP. In the worst-case scenario, your website may appear on the 56th page of SERP or not appear at all. Where you will be ranked, depends on how good your SEO is.
If your legacy website ranked well for certain keywords you should not ruin that rank with an unsuccessful migration. Successful migration can only be achieved if every single step has been well-planned and executed.
Before you decide to migrate your site, you need to identify a migration type. Based on the identification, you will apply a proper migration strategy.
Site Migration Types
- Site location change
- Platform changes
- Content changes
- Structural changes
- Design changes
Site Location Change
Basically, there are two types of site location change:
- With URL changes
- Without URL changes
Fig.2 URL Structure
The first one is when a protocol, subdomain, name or top-level domain changes. The second one may occur, for instance, when the platform has been changed, but URLs stay the same.
John Muller is the Webmaster Trends Analyst at Google. Here are his thoughts on migration:
“Moving sites is generally a lot easier because we can pass all of the signals one to one.”
Changing the platform implies moving to a new platform, upgrading the version of the platform or integrating different platforms.
Adding or removing pages, adding or removing content, and adding new languages.
“I think it’s just important to also make sure that… from a content point of view, you also make sure you have something that matches the old content. So that people who are explicitly looking for their old stuff they will have some place to go.” John Muller.
Structural changes include navigation changes, internal and external links changes, and website hierarchy changes.
The changes that apply to user experience (UX) and website design. Site performance changes, the look, and feel of the website, and media changes.
Before the Migration
Crawl your legacy website and get as much information and statistical data as you can. Use Google Search Console, Google Analytics and other free or paid crawling tools. The data will be later used as a starting point to compare and measure the performance of the new website. With the data, you can easily identify which pages underperform on the migrated website and fix them.
The data from Google Search Console that should be exported:
- Search analytics queries
- Search analytics pages
- Crawl errors
- Mobile usability issues
- Structured data errors
- Links to your site
- Internal links
- Index status
- Click-through rate (CTR)
The data from Google analytics:
- Organic visits during the last 12 months
- Pageviews during the last 12 months
- Bounce rate
Don't forget to make a copy of old robots.txt, as well as a sitemap file or sitemap files.
The crawled data should include:
- Page titles
- Meta descriptions
- Meta keywords
- Server status
- No-follow pages
- Internal links
- Structured data
- External links
- Any other information that you consider relevant
Analyze and save the data for later use. According to a Google developer Martin Splitt, “These measurements aren't perfect… You will always see basically some noise in that measurements”.
Performance audits should be done and saved both for desktop and mobile versions of your website.
Keywords Rank Tracking
Spend some time in keyword researching. Make a list of keywords that are most representative of the website’s organic search visibility. You should make lists of keywords both for mobile and desktop devices.
Because there are thousands of keywords and long-tail keywords combination you should focus on keywords that are driving traffic to your website.
If you do get traffic from both brand and non-brand keywords, you should also decide which type of keywords to focus on more. Keep in mind that non-branded keywords tend to be more competitive and they can drive more traffic.
Redirection is one of the most important parts of the migration process. Redirects help search engines and users to find a certain page. The page may be renamed, moved to another location or it may no longer exist. It’s extremely important to implement redirection correctly. The poorly implemented redirection will cause a higher bounce rate, and it will negatively affect CTR.
Types of redirections:
- 301 Permanent Redirect
- 302 Temporary redirect
- 404 Not found
- 410 Gone
The 301 status code means that a page has permanently moved to a new location. You should use 301 redirects when the URL between legacy and the new website is different.
These will tell the search engines to index the new URLs as well as forward any ranking signals from the old URLs to the new ones.
302 redirects are used in situations where a redirect does not need to live permanently and therefore indexing a new URL is not a priority.
404 means “content not found” and 410 is a more specific status code, it means “content deleted”. The status codes are the same for users, the end result is “no content”, but when it comes to search engines, the situation is slightly different. If a URL returns a 410, search engines know that you have removed the URL on purpose and it should be removed from its index much sooner.
John Muller on 404 and 410: ”We do treat 410s slightly differently than 404s. In particular, we’ll sometimes want to confirm a 404 before removing a URL from the index, and we tend to do that faster with a 410 HTTP result code…If you want to speed up the removal (and don’t want to use a noindex meta tag or the urgent URL removal tools), then a 410 might have a small time-advantage over a 404.”
Google tips on redirects.
“For content on the old site that will not be transferred to the new site, make sure those orphaned URLs correctly return an HTTP 404 or 410 error response code.”
“Don’t redirect many old URLs to one irrelevant destination, such as the home page of the new site. This can confuse users and might be treated as a soft 404 error. However, if you have consolidated content previously hosted on multiple pages to a new single page, it is acceptable to redirect the old URLs to the new, consolidated page.”
“Use HTTP 301 redirects. Although Googlebot supports several kinds of redirects, we recommend you use HTTP 301 redirects if possible.”
Before your new website goes live you need to do a series of testing, the goal is to find and fix technical errors. You should test the main pages, redirects, canonical tags, sitemap, images, HTML structure, speed, structured data, headings, meta tags, etc. Tests should be done on both mobile and desktop versions of the website.
According to Boehms and Basili’s “Software Defect Reduction Top List”, the cost of fixing a bug is 14 times more expensive after release. The deepsource is talking about the exponential rise of costs when fixing bugs on production. Bugs will not just affect the costs it will have a negative impact on SEO.
Before you make the new website available in a staging environment, you need to stop search engines from crawling and indexing your new website.
Preventing search engine to crawl your staging website can be accomplished by using robots.txt or making the staging website available only to specific IP addresses. If you decide to use robots.txt you need to add a few lines of code but keep in mind that you need to remove these lines when your website goes live:
- The content of robots.txt
- User-agent: *
- Disallow: /
The Release Day
The main question you have to ask yourself is: “When should my new website go live?” Don’t make a classic mistake “let's launch as soon as possible and fix it later” because that mistake will hit you so hard that you may never recover.
Perform site migration during the “slow” part of the year if there is some seasonality to your website performance. Don’t perform a site migration during or shortly before the holidays. If you are losing traffic, lose it when business is already slow.
While you replacing the new website with the legacy one, chances are that the live site is going to be temporarily down. Your server should respond to any URL request with a server code 503 (service unavailable). That will signal search engine crawlers that the site is temporarily down for maintenance and they need to come back later to crawl the website.
As soon as you launch your new website you need to take a quick look at:
- The robots.txt
- Top pages redirects
- Top pages canonical tags
- Top pages server responses
- Noindex/nofollow directives
Setup Google Search Console (GSC) and add a sitemap. Check the server logs. Test your new website more in-depth.
Check periodically Google Search Console for blocked resources, structured data errors, mobile usability errors, and fix them ASAP.
Measure website speed both mobile and desktop versions. Make your website as fast as you can, because site speed is one of the most important ranking factors.
Importance of site speed is confirmed by Akamai research which proves that:
- A 100-millisecond delay in website load time can hurt conversion rates by 7 percent
- A two-second delay in web page load time increase bounce rates by 103%
- 53% of mobile site visitors will leave a page that takes longer than 3 seconds to load
“...Make sure they load fast, for your users...”, John Muller.
Monitor and measure website performance. Compare the legacy website data with new website data. Use the new data to optimize your website performance.
After a couple of weeks or a few months, you will find out if your migration is successful or not. Why that long? Because that is the time Google needs to crawl, index and rank your new website. Statistically speaking, that is not a bad thing, because the data will be more accurate and the probability of error minimal.
Plan, test, choose wisely and don’t rush.
This entry was posted in SEO and tagged Web Development, SyncIt Group, Web, SEO, Website Migration, Search Engine Visibility, Search Engines, Site Migration Types, Site Location Change, Page Speed, Redirecting, Rank Tracking, Pre-launch Testing, Website Release, Google Search Console , SERP on December 11, 2019 by Nenad Mihajlović, SEO Specialist .