How We Rebuilt Our Website in 2024

Remember when Gatsby was the hot new thing and everyone was talking about Jamstack? That was us back in 2019, proudly calling ourselves a Jamstack Agency with our shiny new Gatsby site. Fast forward to 2024, and wow, things have changed.

Our journey from that simple 20-page site to what we have now has been quite a ride. We've switched frameworks, changed hosts, and completely rethought how we build websites. It's been a mix of exciting wins and facepalm moments.

In this post, we're laying it all out there – the good, the bad, and the "why did we think that was a good idea?" We'll take you through our adventures with Gatsby, our short experiment with Next.js, and how we ended up with a custom setup that's working great for us (at least for now!).

We've dealt with everything from builds that took forever to hosting bills that shocked us with how high they were. We've learned a ton about what works, what doesn't, and why the "popular" solution isn't always the right one.

Whether you're a seasoned dev or just starting out, we think you'll find something useful here. Maybe you'll learn from our mistakes, or maybe you'll just feel better knowing that even "experts" sometimes get it wrong.

Let's dive into the story of how we rebuilt our website in 2024. It's been a wild ride.

Our Web Development Journey: 2019-2024

2019: The Gatsby Era

As we mentioned, our website journey began in 2019 with Gatsby. Our first site was pretty basic, but it did the job:

• About 20 pages
• A few landing pages and a blog
• Hosted as a static site on Netlify
• Hardly any visitors

This setup was cool at first. Our site looked fresh and was super fast. We felt like we were riding the wave of the latest web dev trends.

But as we grew, so did our problems:

• It got harder to update and manage the Gatsby code
• Builds started taking longer and longer
• Our site began to slow down, and Google wasn't happy about it

These issues got worse as our site expanded and we needed more from it.

2023: Moving to Next.js

By 2023, our Gatsby problems had become too big to ignore:

• Builds sometimes took more than 30 minutes on Gatsby Cloud, which was supposed to be the fastest option
• Our site wasn't scoring well on Web Vitals
• The Gatsby code was becoming a nightmare to work with

Then Netlify acquired Gatsby, shut down Gatsby Cloud, and laid off most of the Gatsby team. We knew it was time for a change. We decided to switch to Next.js. It took a couple of months. We didn't change how the site looked, just the code behind it.

We also moved our hosting from Netlify to Vercel. But this move brought its own set of challenges:

• We had to pay for each team member, even if they rarely used it
• Our monthly bills shot up to $200 to $400, mostly for team access
• Scaling efficiently became a problem, especially since our team size fluctuates a lot

We were paying for seats that barely got used. This didn't fit with our idea of only paying for what we actually use.

2024: Starting Over

By early 2024, we realized switching frameworks wasn't enough. A few things made us decide to rebuild from scratch:

• Jamstack had changed a lot since we started
• Google changed how it ranks websites, and it hurt our site (especially after the Helpful Content Update)
• We needed to rethink what our website was for and how we talk about our brand

The Google Helpful Content Update hit us particularly hard. It made us realize we needed to:

• Completely rethink our content strategy
• Make our content better and more relevant
• Change how we organize information on our site to focus more on what users want

All these issues showed us we needed to completely rebuild our site. Not just the tech stuff, but also the design, content, and overall strategy. We needed a new design, new structure, and a new way to show what our company is about.

Rethinking Our Approach

Hosting and Deployment

We figured out exactly what we wanted from our hosting:

• No paying for unused seats
• Only pay for the traffic we get
• Able to scale down to zero when not in use
• Serverless functions without crazy high costs

As an agency, we have lots of developers. Sometimes they work on our site when they're not busy with client work. It's usually small stuff - fixing bugs or adding tiny features. But to test these changes, they need access to our Vercel team. That costs $20 per month for each person.

We ended up paying over $200 a month for people who barely used Vercel. It felt like a waste. We could have given access and then taken it away each time, but that's a hassle to manage. It also feels a bit wrong, even though many companies do it.

We wanted to mostly pay for what we use. To do that, we needed to use basic cloud providers like AWS, Azure, or Cloudflare.

Is it really that hard to put a website on AWS in 2024? Is it some secret knowledge?

There are tools like SST that make it easier. SST helps you build full-stack apps on your own infrastructure without dealing with all the complex AWS stuff directly.

SST promises to do everything in one config file. With it, we can:

• Use Cloudflare for our domain, pointing to AWS Cloudfront
• Put a Remix app on AWS
• Have custom domains for testing (which can cost $200 on fancy hosting platforms)
• Run some of our code on Cloudflare Workers

Picking a Framework

If not Next.js, then what? We went with Remix, a React framework made by Ryan and Michael. Shopify bought it in 2022, and now it's joining up with React Router. Whatever they call it, Remix/React Router is still growing and has lots of fans.

Our bet: RRR (React Router Remix) will grow even more in the future.

It's a bit funny, but as we thought about what's best in Remix vs Next.js, we concluded that "Remix doesn't do weird things."

Remix is simple: someone visits your site, you load some data, and show a React component. That's all we want from a React framework. It has layouts, actions, meta stuff, and a few hooks. But these things make it simpler, not more complex.

Giving Up SSG (At Least for Now)

After years of deploying static sites, we know the pros, but we also learned the cons the hard way.

We get shivers down our spines when we remember Gatsby builds timing out after 50 minutes. The anxiety hits hard when someone asks about our potential Vercel bill, especially after recent changes where ISG (Incremental Static Generation) became pretty expensive. Our estimated bill was set to double!

Instead, we're now using SWR (stale-while-revalidate) caching. It's a different approach, but it solves our problems without breaking the bank.

The typical advice is to use ISG less. But for our comparison pages, we need regular updates. So we made a bold move: we decided to ditch SSG/ISG altogether.

In theory, server-side rendering (SSR) should make the first byte of data take longer to reach users because it's coming from one place (for us, it's AWS us-east-1). But in real life, we didn't see that. Our speed actually got a bit better for most users.

Is it because we switched from Next.js to Remix? Or because we're not using an extra layer on top of AWS? It's hard to know for sure because we changed a lot of things. But here we are, using SSR for everything and our site is faster.

A nice bonus: our build times are always quick now, no matter how many pages we have.

Will we ever go back to static? Maybe. We might try it again. Remix is working on a new way to load data (they call it single-fetch) that might let us do some static generation in Remix.

Static-only is still great if you have just a few pages or don't update often. But choosing static-only is becoming more about personal preference than actual performance benefits.

Content Management

We've been using Sanity CMS for our website for years. We have lots of different types of content and thousands of documents and images.

This year, we changed everything about our website. New design, new URL structure, new layout for our content hubs. We even changed our logo.

But here's the cool part: we barely touched our Sanity data structure.

Moving content from one system to another is usually a huge pain. It's hard to do automatically. But because we used a headless CMS, we could completely redesign our site without having to move our content around.

Here's a tip: don't tie your design too closely to your data. You'll thank yourself later.

Our New Setup

Here's a simple breakdown of how our new website works:

The main parts:

• AWS for hosting and running our code
• Cloudflare for managing DNS
• Remix as our main framework
• Sanity as our content management system

This setup gives us what we wanted: it's flexible, fast, and doesn't break the bank.

New Challenges We Faced

Our new approach fixed a lot of our old problems, but it also brought some new ones. One issue really stands out as a good lesson in managing websites that can grow and shrink as needed.

Unexpected Costs with Our Flexible Pages

We made our compare pages (like /compare/contentful-vs-sanity) super flexible. You could compare lots of different products in any combination. For example, with products A, B, C, D, E, and F, you could compare A-vs-B, or A-vs-B-vs-C, and so on.

We thought this flexibility was great, but we didn't see the problem coming. Google's bots started looking at every possible combination: a-vs-b-vs-c-vs-d, a-vs-c-vs-b-vs-d, and so on. Because our site creates pages when they're requested, each of these bot visits made a new page. This caused a huge spike in our usage.

The result? A shocking $579 bill.

We messed up. We didn't set limits or think about what would happen if someone (or something) tried all these combinations. It was a wake-up call that with great flexibility comes great responsibility, especially when you pay for what you use.

Luckily, AWS understood and cancelled the bill. But we learned an important lesson: we needed to be smarter about how we structure our content and URLs, and be more careful about what we let search engines see.

Fixing the Problem

To stop this from happening again, we did a few things:

1. We put a limit on how many items can be compared at once.
2. We updated our robots.txt file to tell search engines not to look at all these combinations.
3. We set up better monitoring to alert us if there's unusual traffic or spikes in usage.
4. We looked at our URL structure again to find a balance between flexibility and control.

This showed us that even with a better setup, new problems can pop up. It reminded us how important it is to keep an eye on things, test often, and always be ready to make improvements.

Results and What We Learned

After setting up our new website, we saw some big improvements:

Better Performance

All our speed measurements got better, even the tricky ones (hey INP) like how fast users can interact with the page. We were surprised to see that even our initial load time improved slightly, even though we switched to server-side rendering. This goes against what people usually think about server-side rendering being slower.

Lower Costs

Our hosting bill went way down, about 15-20 times less than before. By only paying for what we use instead of paying for each team member, we've cut our monthly costs a lot. We're not paying for seats that barely get used anymore.

To give you an idea, we now spend about $0.49 per day on AWS, which is about $14-15 per month. Before, we were paying $200 to $400 monthly. Here's what we're paying now:

• Total for the month: $14.57
• Average daily cost: $0.49
• We use 4 services: Lambda, CloudFront, Route 53, S3

Easier to Grow and Maintain

Our new setup is easier to manage and can grow with us. SST helps us automate a lot of the technical stuff, which means less manual work and fewer chances for mistakes. Being able to scale down to zero when traffic is low helps us save even more money.

Faster Build Times

One of the biggest improvements is how quickly we can make changes to the site. Our build times are now consistently fast, no matter how many pages we have. This is so much better than when builds would sometimes fail after running for 50 minutes.

Flexible Content Management

Sticking with Sanity as our headless CMS turned out to be a great choice. We were able to completely change how our website looks and works without having to move all our content around. This flexibility is super valuable for future updates and redesigns.

Wrapping Up

Rebuilding our website in 2024 wasn't just a tech project. It made us rethink our whole approach to having a web presence. We moved from the limitations of traditional Jamstack to a more flexible, faster, and cost-effective solution.

Here are the main things we learned:

1. Don't be afraid to question the popular way: Just because everyone's doing something doesn't mean it's the best for you.
2. Think about all the costs: Look beyond just the tech and consider things like hosting costs, how long builds take, and how productive your developers can be.
3. Be flexible: Choose solutions that let you adapt and change without having to start from scratch.
4. Performance isn't just one thing: Don't focus on just one speed measurement. Think about all aspects of performance, from how long builds take to how quickly users can interact with your site.
5. Keep your content separate from your design: A headless CMS can save you a ton of time and effort when you want to redesign or restructure your site.

Looking ahead, we're excited about what we can do with our new setup. We might try static generation again as tools like Remix get better, but for now, we're happy with our server-side rendering approach.

Remember, there's no one perfect solution in web development. What worked for us might not be the best for everyone. The key is to understand what you need, be willing to try new things, and always keep learning.

We hope sharing our story helps you with your own web projects. Here's to building better, faster, and more efficient websites in 2024 and beyond!