Behind the Scenes

Our website client came to us with a frustrating issue: a drop in qualified website traffic, especially from users overseas, and fewer quality leads. The site is hosted in Thailand, and even users in Thailand were starting to experience painfully slow page load times. That led us to investigate what was happening under the hood.

Using AI-assisted analysis and performance monitoring tools, we pinpointed the issue: the site was loading too slowly around the world. In some cases, it was taking up to 10 seconds just to start rendering. That kind of lag kills conversions and search performance.

So, we rolled up our sleeves and spent a day fixing the backend. The goal was to remove performance as a bottleneck, restore technical SEO hygiene, and ensure global accessibility.

Now that performance is no longer an issue, we can focus on improving content.

1. Switched to Cloudflare for Global Speed & Security

The client's website has been hosted in Thailand for many years, but now it runs through Cloudflare’s global edge network. This slashed our TTFB (Time to First Byte) across the globe from 8–10 seconds down to under 200 ms in key locations like Singapore, Europe, and USA. We:

• Updated DNS to use Cloudflare nameservers
• Enabled HTTPS and automatic redirect enforcement
• Configured caching rules for public content

2. Cleaned Up Robots.txt to Block AI and SEO Scrapers

We replaced WordPress’s default robots.txt with a custom version via WPCode, giving us full control over which bots can index the site. We:

• Blocked known AI crawlers (like GPTBot, Bytespider, ClaudeBot)
• Blocked SEO marketing scrapers (SemrushBot, AhrefsBot, MJ12bot, etc.)
• Allowed trusted search engines like Google, Bing, and Yandex
• Preserved SEO hygiene by disallowing WordPress clutter (e.g., tags, categories, feeds)

3. Hardened URL Structure with Canonical Redirects

All traffic — www, non-www, http, https — now cleanly redirects to the canonical version: https://www.clientdomain.com. We achieved this using Cloudflare’s native HTTPS enforcement and DNS + caching best practices, without requiring any extra plugins or messy code.

4. Optimized Cache Control Headers

Before today, the server was telling browsers not to cache anything (expires: 1970, pragma: no-cache). We fixed that by:

• Replacing outdated .htaccess rules with smart, modern cache-control settings
• Using longer cache durations for images, CSS, and JS (1 year)
• Using short-lived HTML caching (1 hour) to balance speed and content freshness

5. Set Up 301 Redirects for Updated URLs

After changing a few URLs for better SEO, we added clean 301 redirects directly in the site’s .htaccess file — no plugins, no added load. This ensures:

• Visitors (and Google) are redirected automatically from old URLs to the new SEO-friendly versions
• No more broken links in Google Search Console
• SEO value is preserved from past backlinks

6. Tested Everything (And It Works)

We validated our updates using:

• WebPageTest to check TTFB and Largest Contentful Paint
• KeyCDN performance test for global latency
• Chrome DevTools to inspect headers, redirect behavior, and CDN caching
• Google Search Console to monitor 404s and sitemap health

Final Thoughts

This wasn’t a redesign just smart backend cleanup. The results? A faster, cleaner, more SEO-ready site, with better global reach and less technical debt.

If you're running your website on WordPress, a day like this goes a long way.

Curious about how we did it or want help with your own site? Get in touch — we love this stuff. With the help of AI, Neat is providing better and faster solutions for our website clients. Check out our Website Consulting page to learn more.