+++
title = "Index Blocking"
description = "Define pages to prevent them from appearing in search."
categories = ["search"]
tags = ["links", "search", "robots", "metadata"]
features = ["code highlighter", "related content", "snippets"]
[[copyright]]
  owner = "Josh Habdas"
  date = "2019"
  license = "agpl-3.0-or-later"
+++

After Dark uses the `noindex` robots meta directive to prevent search engines from crawling and indexing certain parts of your site. It appears in the HTML document `head` and looks like this:

```html
<meta name="robots" content="noindex">
```

Unlike {{< external href="http://www.robotstxt.org" text="robots.txt" />}} meta directives are defined within page content itself and unambiguously indicate which, if any, pages should be blocked from indexing — even if some of those pages appear in your site's {{< external href="https://gohugo.io/templates/sitemap-template/" text="Sitemap" />}}.

To facilitate the discovery of index blocked pages [Fuzzy Search](../fuzzy-search) utilizes the very same meta directive exposed to search engines to prevent disclosure of pages in its own result listings. Therefore, if a page can be found in fuzzy search, that page may ultimately appear on a search engine result page.

Adjust index blocking per-page using `noindex` {{< external href="https://gohugo.io/content-management/front-matter/" text="Front Matter" />}}:

```toml
noindex = true # set false or remove to unblock
```

Block entire sections using an `_index.md` file with the above setting:

```
├── content
│   ├── legal
│   │   ├── _index.md
│   │   ├── terms.md
│   │   └── privacy.md
│   ├── post
```

By default the following page types are blocked automatically:

- Section listings automatically linked to from the [Section Menu](../section-menu);
- [Taxonomy Pages](../taxonomy-pages) such as `Category`, `Tag` and terms listings; and,
- If enabled, the [Fuzzy Search](../fuzzy-search) page or any deep-linked result within.

Adjust defaults using the `noindex_kinds` setting from {{< external href="https://gohugo.io/getting-started/configuration/" text="Site Configuration" />}}. For example, to enable crawling of section pages add the following to the config:

```toml
[params]
  noindex_kinds = [
    "taxonomy",
    "taxonomyTerm"
  ] # crawl "section" pages
```

Learn about {{< external href="https://moz.com/learn/seo/robots-meta-directives" text="Robots Meta Directives on Moz" />}} and see how Google uses `noindex` in {{< external href="https://support.google.com/webmasters/answer/93710" text="Block search indexing with 'noindex'" />}}.