Technical SEO is the work of making the intended site and its content accessible, understandable and maintainable for search engines and real users. It covers HTTP responses, crawl access, rendering, canonical signals, internal linking, structured data and page experience. Its job is not to make a weak offer persuasive or to guarantee rankings: it removes preventable technical barriers so useful content has a chance to be discovered and evaluated.
For a growth-stage SaaS website, one incorrectly inherited noindex directive or a malformed canonical can undermine an entire service cluster. Conversely, passing a crawler audit does not prove that the content addresses a buyer’s question. Both realities matter.
Executive takeaways
- Diagnose discovery → crawl → render → canonical selection → index → search appearance in order; each requires different evidence.
- A robots.txt disallow is not the same as
noindex; blocking the crawler may stop it seeing the noindex directive. - Prioritise shared-template defects, broken internal links and non-rendered content before small metadata optimisations.
- Treat structured data as a faithful description of visible content—not a shortcut to rich results or AI citations.
- Evaluate Core Web Vitals with field context and actual page groups; a laboratory score is not the same as user experience.
1. Map the failure to a technical stage
If a new landing page does not appear in Search, “Google cannot find it” is only one possible explanation. The URL may not be linked, may redirect incorrectly, may return an error to a bot, may be excluded by a directive, may have its meaningful content generated too late, or may be consolidated into a different canonical. A page can be indexed yet have little exposure because of relevance or competition. Keep these hypotheses distinct so the remedy fits the failure.
Google’s overview of how Search works distinguishes crawling, indexing and serving. Inspect the live HTTP response and rendered DOM for representative templates. Use Search Console URL Inspection where access exists to compare declared and engine-selected states. Do not claim a URL is indexed because a sitemap contains it.
Where a technical fault can interrupt discovery
A successful fetch is not proof that the final page is indexed or served.
2. Control crawling and indexing precisely
Robots.txt generally controls whether a compliant crawler requests a path. It is not a reliable URL-removal method; Google’s robots.txt guidance explicitly distinguishes crawl management from exclusion. If an HTML page must stay out of Search, use an appropriate index directive or authenticated access. The crawler must be able to fetch a noindex rule to see it. Review live responses rather than relying on WordPress admin settings alone.
| Symptom | Evidence to inspect | Likely class of issue | First corrective action |
|---|---|---|---|
| URL discovered, not fetched | Internal links, sitemap, robots, server logs | Discovery or crawl limitation | Add a meaningful route; check access |
| URL fetched, excluded | Robots meta/X-Robots-Tag, HTTP status | Index directive or quality/duplication | Verify intended policy and page value |
| Wrong URL appears | Declared/selected canonical, redirects | Canonicalisation | Align links, redirects and canonical hints |
| Main text missing to crawler | Initial HTML and rendered DOM | Rendering or resource failure | Provide reliable HTML text or repair scripts |
| Page indexed, no useful leads | Queries, offer, UX and tracking | Intent or commercial handoff | Investigate content and conversion separately |
Never put private client records or genuinely confidential documents on a public URL and assume noindex makes them secure. Access control belongs at the application layer.
3. Canonical signals and site architecture
Duplicate parameters, print views, staging variants, category filters and overlapping article versions can confuse both users and reporting. Choose a preferred indexable URL when appropriate, link to it consistently, and keep sitemap and redirect signals aligned. Google’s canonicalisation guide notes that Google may choose a different canonical from your preference. Inspect its choice before declaring the implementation successful.
For B2B sites, link the parent service, distinct child capabilities and relevant editorial Guides using contextual anchors. A breadcrumb is useful to readers, but it cannot substitute for functional navigation. A “related” carousel whose links target draft Pages creates a false content architecture; link only to genuinely available destinations.
4. Rendering and website performance
A JavaScript app that ships an almost empty HTML shell may rely on a later rendering stage. Google can execute JavaScript, but its JavaScript SEO documentation explains the crawl–render–index distinction and notes that server-rendered content is straightforward for more crawlers. Make essential copy, links and metadata reliable in the served HTML wherever feasible, then verify actual rendered output.
Performance needs real-user context. Google’s Core Web Vitals documentation defines good-experience targets of LCP within 2.5 seconds, INP below 200 milliseconds, and CLS below 0.1. These are user-experience thresholds, not guaranteed SEO outcomes. Investigate the affected templates, device groups and 75th-percentile field conditions before optimising an arbitrary Lighthouse number.
Core Web Vitals — Google good-experience targets
Each metric has a different unit; thresholds are not percentages or ranking guarantees.
5. Structured data should match reality
Use JSON-LD only for legitimate page entities and supported properties. The markup must agree with visible content, dates, prices and real authors. Google’s structured-data introduction explicitly warns against describing hidden or fictitious information. Validate with the Rich Results Test for applicable Google-supported appearances, and use the Schema Markup Validator for generic vocabulary. Passing either test does not guarantee a rich result.
| Markup or feature | Appropriate use | Anti-pattern |
|---|---|---|
| Article | Actual published editorial with truthful metadata | Fabricated author or update date |
| Breadcrumb | Reflects the real page hierarchy | Links to absent or draft pages |
| Organisation | Verified organisation details | Unverified awards or invented location |
| FAQ content | Useful answers visible to readers | Expecting FAQ rich results for every website |
| Canonical link | Preferred equivalent URL | Pointing distinct useful pages to one parent |
For AI features in Google Search, there is no special AI schema requirement. If a page is not indexed and eligible for a snippet, technical AI-overview inclusion is not assured by adding JSON-LD.
6. An illustrative B2B website investigation
Suppose a SaaS firm launches six service Pages and only two appear for branded queries. One parent Page carries an incorrect canonical to the homepage, two share an unintended noindex header, and one renders critical copy only after a blocked API request. The other two are technically eligible but address generic questions with little differentiation. Fix the three defect classes separately; the remaining content problem belongs to strategy and editorial work. The scenario is illustrative, not a measured case study.
An audit should produce a prioritised issue register with affected templates, URL examples, user impact, reproduction steps, fix owner and retest evidence—not simply an export of hundreds of warnings.
Diagnostic clinic: identify the failing layer before prescribing a fix
An SEO audit often generates a long list of warnings, but a warning is not the same as the reason a valuable page is absent. Use a representative URL from each WordPress template and trace it through discovery → HTTP response → directives → rendered main content → canonical selection → observed index status → search demand. Collect timestamps and the actual tested URLs. A page may pass a crawler’s HTML inspection while returning a different X-Robots-Tag header from the edge cache. A canonical in an SEO plugin’s settings is not proof of the final canonical emitted in the response.
| Observed symptom | Verify with | Owner and first fix | What a retest must show |
|---|---|---|---|
| New service URL remains excluded | Live HTTP headers, robots meta, URL Inspection, parent links | Developer: remove unintended exclusion; editor: confirm real page value | Intended index policy in the live and rendered response |
| Product description missing to crawler | Raw HTML, rendered DOM, script/network errors | Developer: make essential content reliably available | Meaningful text survives a representative render and failure test |
| Two versions compete | Redirects, canonicals, links, XML sitemap, parameter rules | SEO + developer: document canonical intent consistently | Selected canonical may update after recrawl; watch rather than promise |
| Mobile template feels slow | Field-data cohort and lab trace, asset waterfall | Performance owner: image sizing, interaction or layout defect | Improvement at representative devices, with no accessibility regression |
| Indexed page receives wrong queries | Query cohort, content intent, actual buyer tasks | Editorial owner: revise question and positioning | Better query fit, independent of index eligibility |
Prioritisation rule: resolve shared-template defects before one-off page polish. If 25 commercial pages inherit an accidental noindex header, correcting that configuration affects more real pages than adding ten schema properties to one Guide. But do not blindly remove noindex from legal drafts or staging: owner-approved visibility policy still applies.
For Core Web Vitals, Google’s guidance defines good-experience targets of LCP at or under 2.5 seconds, INP below 200 milliseconds and CLS below 0.1. These are user-experience thresholds, not guaranteed ranking or revenue outcomes. Compare mobile and desktop cohorts where data is sufficient; use lab tools to investigate specific failures rather than generalising from one run.
A realistic verification loop
Record the before-state URL, status, HTML excerpt, canonical and screenshot. Make one scoped fix, then confirm both the targeted defect and collateral functionality: mobile navigation, form submission, accessibility and caching. After release, Google may take time to recrawl and reassess. State “fix validated in rendered output; search reprocessing pending” until actual external evidence changes. A successful lint or local WordPress preview alone cannot prove indexing.
7. A 90-day remediation programme
Days 1–30: sample key templates and collect actual response, directive, rendered-content and canonical evidence; check mobile navigation and field performance. Classify systemic blockers.
Days 31–60: repair shared template issues, fix route hierarchy and validate representative URLs in a WordPress-equivalent environment; make redirect and canonical decisions with evidence.
Days 61–90: rerun the same tests, confirm accessible URLs and monitor Search Console changes with realistic crawl delays. Hand nontechnical relevance issues to the correct content or commercial owner.
8. Common mistakes and decisions
A 200 response is not an indexation certificate. Robots disallow is not data protection. Indexation is not ranking. A good CWV score does not compensate for misleading content. A schema test pass does not promise AI citations. The useful question is always: which technical failure is preventing which user or crawler task, and what evidence shows it was corrected?
9. Frequently asked questions
Does a sitemap guarantee indexing?
No. It helps discovery; engines still decide whether and how to index content. Check accessibility, canonical signals, relevance and actual reporting.
Should we always server-render?
Not as an absolute requirement. But ensure essential content and links are reliably visible to the crawlers and users you need to support; verify real output instead of assuming all bots execute JavaScript similarly.
Will schema improve AI visibility?
Correct schema can improve understanding and eligibility for supported features. Google says no separate AI-specific schema is required, and no markup guarantees selection.
10. References and further learning
- Google — How Search works: crawl and index stages.
- Google — robots.txt: crawl versus index controls.
- Google — canonicalisation: duplicate URL signals.
- Google — Core Web Vitals: metric definitions and thresholds.
- Google — structured data: truthful markup.
Review an SEO engagement through the matching published service page when available. No page or integration is represented here as live solely because it appears in a planning registry.