There are 96 unindexed items and only 36 indexed items.This must be abnormal.

Understanding Indexing Discrepancies: Why Your Website Shows 96 Unindexed Items Despite Limited Content

Maintaining an effective SEO strategy requires close attention to how your website’s pages are crawled and indexed by search engines. If you notice that a significant number of your pages—say, 96 items—remain unindexed while only 36 pages are visible in search results, it’s understandable to feel concerned, especially when your site features limited content.

The Core Issue: Discrepancy Between Crawled and Indexed Pages

In some cases, website owners observe that:

• Only a subset of pages has been indexed (in this scenario, 36 pages).
• Many pages, including product and menu pages, are either crawled but not indexed or returning 404 errors.
• There are no recent page additions, yet the number of unindexed pages remains high.

This situation suggests underlying issues that may be impeding proper indexing and visibility.

Possible Causes and Considerations

1. Crawling and Indexing Differences

2. Search engines crawl all accessible pages, but they only index those deemed valuable or relevant based on various criteria such as content quality, uniqueness, and site structure.

3. If pages are low-quality, duplicate, or thin in content, search engines may choose not to index them, leading to discrepancies.

4. Presence of 404 Error Pages

5. A significant number of 404 error pages indicate broken links or deleted content.

6. Search engines encountering numerous 404 errors may reduce their crawling frequency on your site, leaving some recently deleted or non-existent pages unindexed.

7. Frequent or widespread 404 errors can negatively impact your site’s overall SEO health.

8. Site Structure and Internal Linking

9. Poor internal linking or an incomplete sitemap can hinder search engine bots from discovering all relevant pages.

10. Ensuring your site has a clear, logical structure helps search engines crawl and understand your content effectively.

11. Indexing Filters and Noindex Tags

12. Check if certain pages are inadvertently marked with noindex directives or meta tags that prevent indexing.

13. Review robots.txt files to ensure that essential pages are not blocked from crawling.

14. Recent Content Uploads and Crawling Frequency

15. Even if you upload new products regularly, if your site isn’t being crawled frequently or if there are technical barriers, new content may not appear in search results quickly.

16. Submitting an updated sitemap via Search Console can help prioritize crawl requests.

Recommendations to Address the Issue

• Audit Your Website for Broken Links: Use tools like Screaming Frog or Ahrefs to identify and fix or redirect broken URLs resulting in 404 errors.

• Review Your Robots.txt and Meta Tags: Ensure that critical pages are not unintentionally blocked from indexing.

• Optimize Your Site Structure: Enhance internal linking and create a comprehensive XML sitemap to guide search engines more effectively.

• Monitor and Submit Your Sitemap: Regularly update and submit your sitemap through Google Search Console to facilitate efficient crawling.

• Evaluate Content Quality: Improve thin or duplicate content to encourage indexing of relevant pages.

Conclusion

A mismatch between crawled and indexed pages is a common SEO challenge that can stem from technical issues, site structure, or content quality concerns. By systematically auditing your site and addressing potential barriers to crawling and indexing, you can improve your website’s visibility in search engine results, ensuring that your content reaches your target audience effectively.