Key Takeaways
- Web Teams can utilize artificial intelligence (AI) to drastically reduce the time and cost of large-scale website redesigns.
- Purpose-built web crawlers coupled with AI models can help teams quickly inventory and understand large volumes of website content and structure.
- AI, with access to analytics data, can suggest actionable improvements to information architecture and UX design.
- Agentic workflows can transform legacy content into modern frameworks, like WordPress Gutenberg blocks, at scale.
Listen: How to streamline large-scale web design with AI assistance.
Websites are living documents, and redesigns are inevitable. User behaviors change, aesthetic trends evolve, and the technology behind the web continues to improve. What once felt modern and efficient eventually becomes dated, clunky, and increasingly difficult to maintain. When you’re working with a large website (i.e., hundreds, thousands, or even tens of thousands of pages), those challenges become magnified. Redesigns at this scale are not just difficult; they’re exponentially more complex and costly. Legacy code, bolted-on features, and outdated CMS platforms or plugins can turn even simple updates into risky, high-stakes maneuvers. For example, redesigning a small marketing site with dozens of pages might take a few weeks of focused work from a small team. In contrast, redesigning a sprawling site with 10,000 pages can take months, require coordination across departments, involve intricate content migration planning, and rack up six or seven-figure costs, especially if the process uncovers deeper issues with SEO, accessibility, or technical debt.
While large-scale redesigns will always demand effort and care, the time, cost, and complexity involved can be significantly reduced with the smart application of AI tooling and workflows. While there’s not an off-the-shelf AI product that can handle this kind of challenge by itself (at least at the time of this writing), forward-thinking teams have a number of strategies they can utilize to leverage the power of large language models and agentic workflows to accelerate their work and reduce the overall costs.
How did we get here? The best redesign is the one you never have to do
Most large-scale website redesigns don’t happen because someone woke up one day and decided the site looked a little dated. They happen because years of short-term decisions and accumulated workarounds have made meaningful change nearly impossible. Over time, as teams move fast to meet deadlines or react to shifting business needs, the site’s foundations begin to erode. Developers hard-code content to ship a landing page. Styles get duplicated and tweaked in isolation. Page templates diverge subtly but significantly. What began as a flexible system turns into a fragile, tangled mess of technical debt that resists updates and breaks under pressure.
The same slow breakdown happens on the content side, too. Writers and editors adjust tone, formatting, and structure to meet the needs of specific campaigns or stakeholders, often without a shared system or style guide to keep things aligned. As a result, brand voice becomes inconsistent, content structure varies, and the user experience starts to feel patchy and uneven. What once felt cohesive starts to read like it was assembled from different teams in different eras—because it was.
If you’ve arrived at the point where a full-scale redesign is necessary, it’s often because the cost of inaction (of continuing to patch and tweak) has finally exceeded the cost of starting over. But moving forward, the goal shouldn’t just be to launch a better version of what you had. It should be to build a system that evolves with you. That means investing in consistent design patterns, reusable components, and shared standards for both code and content. A centralized design system with clear rules for layout, spacing, typography, and interactivity helps ensure visual consistency and simplifies future updates. Components built with modern frameworks and templating systems reduce duplication and increase maintainability. And on the content side, a unified voice and clear formatting guidelines make it easier to scale messaging without diluting the brand.
When you build this kind of structure into your site from the ground up, you’re not just avoiding another massive redesign down the road—you’re creating a platform that can adapt continuously. You can iterate, test, and evolve without starting from scratch every few years.For large-scale redesigns, AI can streamline the process, cut costs and manual effort, uncover strategic insights, and enable your team to work faster and more confidently.
Using AI to assist with the redesign process
A successful website redesign typically unfolds in three key phases: Assess, Plan, and Execute. The Assess phase is all about understanding what you’re working with. That means taking a detailed inventory of your existing content and how it’s stored and presented. Are your pages managed in a CMS with structured HTML? Are they scattered across folders as static HTML files? Is your setup powered by a custom-built system? It’s also important to evaluate your information architecture in this phase, too. Is your content organized logically with a clear hierarchy, or is it inconsistently structured and hard to navigate?
In the Plan phase, you define where you’re headed, not just visually, but structurally. This involves creating new designs and clearly outlining any changes to how content is categorized and presented. Will some sections be removed, merged, or expanded? Will certain pages shift categories or be consolidated? This is where the future visual layouts and content structure take shape.
Finally, the Execute phase is where the transformation happens. This involves taking your existing content and structure, applying the new design, migrating everything into the new framework, and ensuring that it all works as intended.
Web teams can leverage AI at each of these phases to increase their productivity and reduce the burden of a redesign.
Assess: Using AI to inventory your content and understand information architecture
AI can significantly streamline the Assess phase by streamlining what would otherwise be an overwhelming manual process. For example, a purpose-built web crawler can scan every page of a website and convert each page’s content into vector embeddings. These embeddings allow the team to interact with the site’s content through natural language queries. Asking the AI questions like “What topics are covered in our help center?” or “Where do we talk about pricing or billing?” all without needing to sift through thousands of pages manually or relying on text searches alone. Developers can also enrich this process by adding metadata during the crawl, such as page type, language, author, and publication date, giving the AI more context to deliver accurate and structured answers. This setup provides a powerful research layer that helps content strategists, designers, and developers understand the current state of the website without auditing every page by hand.
Beyond content comprehension, AI can help teams visualize and evaluate a site’s information architecture. As the crawler processes each page, it can also map how pages are interlinked and reachable, effectively constructing a content graph. This graph reveals the implicit structure of the site, showing which pages are central hubs, which are orphaned, and how deeply certain sections are nested. With this graph in hand, the AI can analyze how users might traverse the site, identify areas of poor discoverability, and suggest optimizations such as consolidating thin content, flattening unnecessarily deep hierarchies, or reorganizing categories to better reflect user intent. Developers can also use the crawler in tandem with a large language model to detect common template constructs across the site (headers, footers, navigation, callouts, product details, etc.) and highlight their variations and inconsistencies. This surfaces structural and design fragmentation early in the process, allowing the team to plan for a more unified system.
It’s important to understand that this level of analysis isn’t something you can achieve by simply dropping a URL into ChatGPT or prompting a general-purpose model. These capabilities require custom development to integrate a site crawler with embedding generation, a vector database for querying, and possibly a front-end interface to explore the data. But compared to manually reviewing thousands of pages, building these tools offers an enormous return on investment for large redesigns. Not only does it save time and resources, but it also gives teams the clarity and structure they need to make informed, strategic decisions during the redesign.
OpenAI provides a tutorial, Web Q&A, which illustrates how a web crawler can be built to ingest an entire website’s content. Of course, the exact implementation is highly dependent on the specific redesign project and should be customized to surface the level of detail and information required to accomplish the goals of the project.
Plan: Using AI to improve information architecture, visual structure, and the user experience
In the Plan phase, the insights gathered during the Assess phase become the foundation for shaping the future structure and experience of the site. This is where AI can begin to act not just as an analyzer, but as a strategic collaborator. With access to the content embeddings and metadata generated earlier, large language models can surface high-level patterns and make informed suggestions about how to improve the site’s information architecture. For example, AI can identify pages with overlapping content that might be merged, redundant or overstuffed categories, or topics buried too deeply in the hierarchy that deserve more visibility. It can even highlight underutilized sections or “content deserts” that might benefit from expansion.
When this content analysis is paired with website analytics, AI’s recommendations become even more powerful. For instance, by correlating high-performing pages with low visibility in the current structure, AI can suggest elevating those pages within the navigation. Conversely, it might flag areas of the site that receive little traffic and offer suggestions for consolidation, rewriting, or removal. This hybrid approach of combining semantic understanding with behavioral data gives teams a more complete picture of how their content is organized and experienced, and how it could be reshaped to better serve users. Like in the previous phase, the AI can’t just conjure its own access to a website’s analytics data. Developers would need to provide a mechanism for the model to query the website’s analytics to enable this level of analysis using a technique called retrieval-augmented generation (RAG).
These AI-generated insights can then directly inform the visual and UX planning work. Designers can use AI’s structural suggestions to rethink navigation systems, streamline page layouts, or restructure content groupings to reduce cognitive load. If the AI identifies a pattern of users bouncing from long-form articles with no clear CTA, for example, that insight could inspire changes to page design that improve flow and engagement. Separate AI systems in common design tools, like Figma AI and Adobe AI, can also propose layout ideas or wireframe suggestions based on how certain types of content are best presented (blog posts, support documents, product detail pages, and so on). While the AI might not design the final experience, it can rapidly generate concepts and variations for the design team to iterate on.
It’s worth emphasizing that AI isn’t making these decisions on its own, and it shouldn’t. Planning the next version of a website involves trade-offs, business goals, and brand considerations that require human judgment. But by handling the tedious analysis, surfacing data-driven suggestions, and offering generative starting points, AI can drastically accelerate the planning phase. It helps teams spend less time on foundational analysis and more time on creative problem-solving, moving them from insight to action faster and with greater confidence.
Execute: Using agentic workflows and AI-pair programming to supercharge the transformation
The Execute phase is where plans become real. Where content gets migrated, components are built, and the new site begins to take shape. With the right configuration, agentic workflows (which are AI systems capable of reasoning through tasks and taking autonomous action) can begin transforming legacy content into modern formats and structures. An AI agent, for instance, could take semantically structured content extracted during the Assess phase and map it to reusable WordPress Gutenberg blocks. The AI might identify that a section of text corresponds to a “Hero” block, an image-and-text pairing to a “Media & Text” block, and a CTA to a predefined “Button” block. It can then generate the appropriate block markup or even interface with the WordPress REST API to programmatically build out full pages. There are several agentic coding tools now available, including Claude Code by Anthropic and Codex by OpenAI. These workflows don’t replace developers, but they can automate the repetitive, large-scale transformation of content, freeing up the team to focus on quality, experience, and performance.
Meanwhile, developers working on the more bespoke or complex parts of the redesign can benefit from AI as a kind of ever-present pair programmer. AI coding assistants like GitHub CoPilot and Cursor can quickly answer questions about unfamiliar libraries, help troubleshoot issues in real time, suggest code snippets, or even explain existing legacy code. This can significantly accelerate development, particularly in environments where teams are dealing with technical debt or undocumented systems. Rather than jumping between browser tabs and documentation, developers can stay in flow and maintain momentum with contextual, code-aware assistance.
For Content Teams, AI can also play a role in bringing consistency and polish to the final product. As content is migrated or rewritten for the new site, writers can use AI to align tone and voice across hundreds or thousands of pages. Whether that means rewriting overly formal documentation into something more conversational or unifying inconsistent product descriptions, AI can offer real-time suggestions and bulk editing capabilities that would be impossible to scale manually. It can also fill in gaps by suggesting headlines, summaries, or metadata that match the site’s voice and SEO strategy.
AI can’t run a launch or debug production bugs on its own, but it can drastically reduce the friction and fatigue that often slow large-scale redesigns in their final stages. By embedding AI into the transformation process through both automated agents and real-time collaboration, teams can move faster, produce more consistent results, and ship with greater confidence.
Human-led, AI-accelerated redesigns are the future
Redesigning a large website is a challenging task, but it doesn’t have to be paralyzing. With the right strategy and the smart application of AI, teams can assess legacy content more efficiently, make better-informed structural decisions, and execute the transformation faster and with greater consistency. While AI won’t replace the creativity, judgment, or leadership required for a successful redesign, it can eliminate much of the manual overhead that slows teams down. Whether you’re rebuilding from the ground up or preparing for your next evolution, a human-led, AI-accelerated approach offers a faster, smarter, and more scalable way forward.
About the authors
I am a career web developer who's been building sites since the dark ages of the 1990s, but eventually chose to major in theatre in college. What I did know then was that having some hard skills to fall back on would be really important, since I like doing things like… eating. I've worked on small sites, big sites; I was Director of Web Marketing for Pittsburg State University, and now I work with my team at Aquent building sites that help other people find jobs, and companies find employees. I place a hard emphasis on human-centered design, accessibility, and user experience. That's all a pretty cool, I think. I've written a book, spoken at events from the US to Europe, and generally like helping out other web folks be better web folks. Sometimes I succeed, but I try to have fun the whole time.
Josh is the cofounder and engineering force behind Cardsetter, a platform that helps small businesses get online. A U.S. Navy veteran and a graduate of the University of California, San Diego, he built his first website in 1997 over a dial-up modem. That early curiosity became a lifelong passion for web technology. Today, he creates smarter, more accessible web solutions and writes about AI and emerging technologies, making complex topics easy to understand and apply.
Latest.
Stop the price walk: keep more of your clients.
Consulting & Operations, Content & Creative, Experience & Product, Financial Services
How to reset culture through Creative Operations.
Insights from InsideOut, Leadership & Management, Consulting & Operations, Content & Creative
Why mental health benefits matter to freelancers.
Diversity, Equity & Inclusion, Leadership & Management, Retention Strategies, Talent Acquisition & Recruitment
