Technical SEO is the work of making a site’s pages discoverable, crawlable, renderable and indexable, so search engines and AI features can find and evaluate useful content. It covers HTTP responses, robots rules, rendering, canonical signals, internal links, structured data and page experience. It removes preventable barriers; it does not make a weak offer persuasive or guarantee rankings.
For a growth-stage SaaS website, one incorrectly inherited noindex directive or a malformed canonical can undermine an entire service cluster. Conversely, passing a crawler audit does not prove that the content addresses a buyer’s question. Both realities matter.
Key takeaways
- Diagnose discovery → crawl → render → canonical selection → index → search appearance in order; each requires different evidence.
- A robots.txt disallow is not the same as
noindex; blocking the crawler may stop it seeing the noindex directive. - Prioritise shared-template defects, broken internal links and non-rendered content before small metadata optimisations.
- Treat structured data as a faithful description of visible content—not a shortcut to rich results or AI citations.
- Evaluate Core Web Vitals with field context and actual page groups; a laboratory score is not the same as user experience.
What does the 2024 Web Almanac show about technical SEO?
The HTTP Archive Web Almanac 2024 SEO chapter, published in December 2024, measured technical basics across millions of pages. It is a useful reality check: many sites still miss fundamentals that are cheap to fix.
Common technical signals across the web
The 2024 Performance chapter adds the page-experience view: only 43% of mobile sites passed all three Core Web Vitals, an image was the LCP element on 73% of mobile pages, and 16% of pages lazy-loaded that LCP image, delaying the element that should load first. Our Core Web Vitals guide explains how to fix these.
Where in the pipeline is the page failing?
If a new landing page does not appear in Search, “Google cannot find it” is only one possible explanation. The URL may not be linked, may redirect incorrectly, may return an error to a bot, may be excluded by a directive, may have its meaningful content generated too late, or may be consolidated into a different canonical. A page can be indexed yet have little exposure because of relevance or competition. Keep these hypotheses distinct so the remedy fits the failure.
Google’s overview of how Search works distinguishes crawling, indexing and serving. Inspect the live HTTP response and rendered DOM for representative templates. Use Search Console URL Inspection where access exists to compare declared and engine-selected states. Do not claim a URL is indexed because a sitemap contains it.
Where a technical fault can interrupt discovery
A successful fetch is not proof that the final page is indexed or served.
How do you control crawling and indexing?
Robots.txt generally controls whether a compliant crawler requests a path. It is not a reliable URL-removal method; Google’s robots.txt guidance explicitly distinguishes crawl management from exclusion. If an HTML page must stay out of Search, use an appropriate index directive or authenticated access. The crawler must be able to fetch a noindex rule to see it. Review live responses rather than relying on WordPress admin settings alone.
| Symptom | Evidence to inspect | Likely class of issue | First corrective action |
|---|---|---|---|
| URL discovered, not fetched | Internal links, sitemap, robots, server logs | Discovery or crawl limitation | Add a meaningful route; check access |
| URL fetched, excluded | Robots meta/X-Robots-Tag, HTTP status | Index directive or quality/duplication | Verify intended policy and page value |
| Wrong URL appears | Declared/selected canonical, redirects | Canonicalisation | Align links, redirects and canonical hints |
| Main text missing to crawler | Initial HTML and rendered DOM | Rendering or resource failure | Provide reliable HTML text or repair scripts |
| Page indexed, no useful leads | Queries, offer, UX and tracking | Intent or commercial handoff | Investigate content and conversion separately |
Never put private client records or genuinely confidential documents on a public URL and assume noindex makes them secure. Access control belongs at the application layer.
How do canonical signals and architecture interact?
Duplicate parameters, print views, staging variants, category filters and overlapping article versions can confuse both users and reporting. Choose a preferred indexable URL when appropriate, link to it consistently, and keep sitemap and redirect signals aligned. Google’s canonicalisation guide notes that Google may choose a different canonical from your preference. Inspect its choice before declaring the implementation successful.
For B2B sites, link the parent service, distinct child capabilities and relevant editorial Guides using contextual anchors. A breadcrumb is useful to readers, but it cannot substitute for functional navigation. A “related” carousel whose links target draft Pages creates a false content architecture; link only to genuinely available destinations.
How do rendering and performance affect SEO?
A JavaScript app that ships an almost empty HTML shell may rely on a later rendering stage. Google can execute JavaScript, but its JavaScript SEO documentation explains the crawl–render–index distinction and notes that server-rendered content is straightforward for more crawlers. Make essential copy, links and metadata reliable in the served HTML wherever feasible, then verify actual rendered output.
Performance needs real-user context. Google’s Core Web Vitals documentation defines good-experience targets of LCP within 2.5 seconds, INP below 200 milliseconds, and CLS below 0.1. These are user-experience thresholds, not guaranteed SEO outcomes. Investigate the affected templates, device groups and 75th-percentile field conditions before optimising an arbitrary Lighthouse number.
Core Web Vitals — Google good-experience targets
Each metric has a different unit; thresholds are not percentages or ranking guarantees.
How should structured data be used?
Use JSON-LD only for legitimate page entities and supported properties. The markup must agree with visible content, dates, prices and real authors. Google’s structured-data introduction explicitly warns against describing hidden or fictitious information. Validate with the Rich Results Test for applicable Google-supported appearances, and use the Schema Markup Validator for generic vocabulary. Passing either test does not guarantee a rich result.
| Markup or feature | Appropriate use | Anti-pattern |
|---|---|---|
| Article | Actual published editorial with truthful metadata | Fabricated author or update date |
| Breadcrumb | Reflects the real page hierarchy | Links to absent or draft pages |
| Organisation | Verified organisation details | Unverified awards or invented location |
| FAQ content | Useful answers visible to readers | Expecting FAQ rich results for every website |
| Canonical link | Preferred equivalent URL | Pointing distinct useful pages to one parent |
For AI features in Google Search, there is no special AI schema requirement. If a page is not indexed and eligible for a snippet, technical AI-overview inclusion is not assured by adding JSON-LD.
What does a technical investigation look like?
Suppose a SaaS firm launches six service Pages and only two appear for branded queries. One parent Page carries an incorrect canonical to the homepage, two share an unintended noindex header, and one renders critical copy only after a blocked API request. The other two are technically eligible but address generic questions with little differentiation. Fix the three defect classes separately; the remaining content problem belongs to strategy and editorial work. The scenario is illustrative, not a measured case study.
An audit should produce a prioritised issue register with affected templates, URL examples, user impact, reproduction steps, fix owner and retest evidence—not simply an export of hundreds of warnings.
Diagnostic clinic: identify the failing layer before prescribing a fix
An SEO audit often generates a long list of warnings, but a warning is not the same as the reason a valuable page is absent. Use a representative URL from each WordPress template and trace it through discovery → HTTP response → directives → rendered main content → canonical selection → observed index status → search demand. Collect timestamps and the actual tested URLs. A page may pass a crawler’s HTML inspection while returning a different X-Robots-Tag header from the edge cache. A canonical in an SEO plugin’s settings is not proof of the final canonical emitted in the response.
| Observed symptom | Verify with | Owner and first fix | What a retest must show |
|---|---|---|---|
| New service URL remains excluded | Live HTTP headers, robots meta, URL Inspection, parent links | Developer: remove unintended exclusion; editor: confirm real page value | Intended index policy in the live and rendered response |
| Product description missing to crawler | Raw HTML, rendered DOM, script/network errors | Developer: make essential content reliably available | Meaningful text survives a representative render and failure test |
| Two versions compete | Redirects, canonicals, links, XML sitemap, parameter rules | SEO + developer: document canonical intent consistently | Selected canonical may update after recrawl; watch rather than promise |
| Mobile template feels slow | Field-data cohort and lab trace, asset waterfall | Performance owner: image sizing, interaction or layout defect | Improvement at representative devices, with no accessibility regression |
| Indexed page receives wrong queries | Query cohort, content intent, actual buyer tasks | Editorial owner: revise question and positioning | Better query fit, independent of index eligibility |
Prioritisation rule: resolve shared-template defects before one-off page polish. If 25 commercial pages inherit an accidental noindex header, correcting that configuration affects more real pages than adding ten schema properties to one Guide. But do not blindly remove noindex from legal drafts or staging: owner-approved visibility policy still applies.
For Core Web Vitals, Google’s guidance defines good-experience targets of LCP at or under 2.5 seconds, INP below 200 milliseconds and CLS below 0.1. These are user-experience thresholds, not guaranteed ranking or revenue outcomes. Compare mobile and desktop cohorts where data is sufficient; use lab tools to investigate specific failures rather than generalising from one run.
A realistic verification loop
Record the before-state URL, status, HTML excerpt, canonical and screenshot. Make one scoped fix, then confirm both the targeted defect and collateral functionality: mobile navigation, form submission, accessibility and caching. After release, Google may take time to recrawl and reassess. State “fix validated in rendered output; search reprocessing pending” until actual external evidence changes. A successful lint or local WordPress preview alone cannot prove indexing.
What should a 90-day remediation programme include?
Days 1–30: sample key templates and collect actual response, directive, rendered-content and canonical evidence; check mobile navigation and field performance. Classify systemic blockers.
Days 31–60: repair shared template issues, fix route hierarchy and validate representative URLs in a WordPress-equivalent environment; make redirect and canonical decisions with evidence.
Days 61–90: rerun the same tests, confirm accessible URLs and monitor Search Console changes with realistic crawl delays. Hand nontechnical relevance issues to the correct content or commercial owner.
What are the most common technical SEO mistakes?
A 200 response is not an indexation certificate. Robots disallow is not data protection. Indexation is not ranking. A good CWV score does not compensate for misleading content. A schema test pass does not promise AI citations. The useful question is always: which technical failure is preventing which user or crawler task, and what evidence shows it was corrected?
What should WordPress sites check after every change?
WordPress makes publishing easy, which also makes it easy to break technical signals without noticing. After any theme switch, plugin update or major content change, check these on two or three representative templates:
| Check | Why it breaks | How to verify |
|---|---|---|
| “Discourage search engines” is off | Left on after staging or a restore | Settings → Reading; live page source |
| Custom post types still resolve | Registered by the old theme, not a plugin | Visit a guide or tool URL; check for 404 |
| Canonical and meta robots unchanged | SEO plugin settings reset or duplicated by the theme | View source; URL Inspection |
| Structured data still present | Theme output removed or duplicated | Rich Results Test |
| Caches cleared | Old HTML or headers still served | Response headers; private window |
These five checks take minutes and catch the problems that most often follow routine maintenance.
Which technical topics need their own deep dive?
This guide is the overview. Several areas need more detail than one page can give:
- Page experience: metrics, thresholds and a fix order in our Core Web Vitals guide.
- Structure and links: crawl depth, orphan pages and anchor text in site architecture and internal linking.
- Redesigns and replatforms: redirect maps and launch checks in website migration SEO.
- Machine understanding: Organization, Article and Breadcrumb markup in entity SEO and structured data.
- A full health check: 64 checks across eight areas in the SEO audit checklist.
Technical work should always serve the wider plan set out in your SEO strategy: fix what blocks the pages that matter most to buyers first.
Frequently asked questions
Does a sitemap guarantee indexing?
No. It helps discovery; engines still decide whether and how to index content. Check accessibility, canonical signals, relevance and actual reporting.
Should we always server-render?
Not as an absolute requirement. But ensure essential content and links are reliably visible to the crawlers and users you need to support; verify real output instead of assuming all bots execute JavaScript similarly.
Will schema improve AI visibility?
Correct schema can improve understanding and eligibility for supported features. Google says no separate AI-specific schema is required, and no markup guarantees selection.
How often should a technical SEO review happen?
Run a full review once a year and before any migration, a lighter quarterly check of indexing, Core Web Vitals and new templates, and quick checks after every major release. Search Console’s Page indexing and Core Web Vitals reports are the best early warnings.
Do AI Overviews need special technical work?
Google’s AI features documentation says no. Pages need to be indexed and eligible to show a snippet; the same technical foundations apply. Google says Search Console’s generative AI performance report, available to all sites by 31 August 2026, shows AI Overviews and AI Mode impressions.
What is the most common technical SEO mistake on WordPress?
An indexing setting or noindex rule carried over from staging, or a theme or plugin change that removes custom post types, canonical tags or structured data. Check the live response of key templates after every theme or plugin change.
Sources and further reading
- Google — How Search works: crawl and index stages.
- Google — robots.txt: crawl versus index controls.
- Google — canonicalisation: duplicate URL signals.
- Google — Core Web Vitals: metric definitions and thresholds.
- Google — structured data: truthful markup.
- HTTP Archive Web Almanac 2024 — SEO chapter: technical adoption data.
- HTTP Archive Web Almanac 2024 — Performance chapter: Core Web Vitals across the web.
- Google — AI features and your website: eligibility for AI Overviews and AI Mode.
Want the technical layer checked properly? Bargaon’s technical SEO service traces key templates through every stage, from discovery to index, and delivers a prioritised issue register with retest evidence.