Generating a Sitemap with Next.js Pages Router and Sanity

In this blog, I'll guide you through creating a sitemap within the Next.js pages router while using Sanity as a CMS and TypeScript for type safety. We will cover essentials like creating the sitemap type, adding a field to schemas include/exclude pages from appearing in the sitemap, setting up a query, and fetching data from Sanity.

Explaining sitemaps

When it comes to optimising your website for search engines, like Google and Bing, a well structured sitemap has a role to play. It helps out search engines in discovering your pages so they can be indexed and show up in search results. I can also be used a mode of navigation for your users, allowing them to more efficiently find content in your site.

Dependencies

Here is a list of the dependencies I used for this blog:

"next": "14.1.0"
"sanity": "3.26.1"
"next-sanity": "7.0.14"

Creating a sitemap TypeScript type

As I will be using TypeScript for this, I created the following type to ensure type safety. If you are using JavaScript, you can skip this.

types/index.ts
1import type { Slug } from 'sanity';
2
3export type TSiteMap = {
4  _type: string;
5  _updatedAt: Date;
6  slug?: Slug;
7};

Creating the query to fetch documents

Here is a basic query that can be used to fetch a set of documents and the required properties for to be displayed on the sitemap.

queries.ts
1import { groq } from 'next-sanity';
2
3export const sitemapQuery = groq`
4  *[_type == "page"] {
5    _updatedAt,
6    _type,
7    slug,
8  }
9`;

But there are a couple of improvements we can make to this query and our schemas to firstly make sure we are only fetching documents with a valid slug, turning the query into a function for reusability across multiple documents, and also give yourself or your editors the option to exclude the page from being indexed.

Fetching documents with a valid slug

This is a simple but effective change to add to your query as fetching a document without a valid slug and adding it to sitemap will not be great as it would cause your sitemap to be invalid and Google, or other search engines, might not be able to read it.

queries.ts
1export const sitemapQuery = groq`
2  *[_type == "page" && defined(slug)] {
3    _updatedAt,
4    _type,
5    slug,
6  }
7`;

Making a function for reusability

If you are like me and have multiple types of documents which eventually show up on the web, then coping and pasting the above query and changing the type each time is not only time wasting but you are repeating yourself which is not good practice.

I currently have a page, blog, service, and project schemas which all display a page on my frontend. The below is a massive time saver.

queries.ts
1export const sitemapQuery = (type: string) => {
2  return groq`
3    *[_type == "${type}" && defined(slug)] {
4      _updatedAt,
5      _type,
6      slug,
7    }
8  `;
9};

Functionality to exclude pages from being indexed

Another neat thing we can do with Sanity is to give your editors or yourself the option to exclude certain pages from being indexed. One reason might be because you only want traffic to come from an external source like social media and see how it performs.

Here is my page document schema with a publish status string field which allows me to decide whether I want the page to be index.

page.ts
1import { defineType } from 'sanity';
2
3export default defineType({
4  name: 'page',
5  type: 'document',
6  fields: [
7    defineField({
8      title: 'Publish status',
9      name: 'publishStatus',
10      type: 'string',
11      options: {
12        layout: 'radio',
13        list: [
14          {
15            title: "Hidden (won't show up in Google, but accessible through URL)",
16            value: 'hidden',
17          },
18          'public',
19        ],
20      },
21      initialValue: 'hidden',
22    }),
23    ...other fields,
24  ],
25});

Here's the updated query to only fetch those pages with a publish status value of public.

queries.ts
1export const sitemapQuery = (type: string) => {
2  return groq`
3    *[_type == "${type}" && defined(slug) && publishStatus == "public"] {
4      _updatedAt,
5      _type,
6      slug,
7    }
8  `;
9};

Generating the sitemap

Now for the main event, generating the sitemap. Create a new file called sitemap.xml.ts under your pages folder.

pages/sitemap.xml.ts
1import { ServerResponse } from 'http';
2
3function generateSiteMap() {
4  return `<?xml version="1.0" encoding="UTF-8"?>
5   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
6     <url>
7       <loc>https://stelko.xyz</loc>
8     </url>
9   </urlset>
10 `;
11}
12
13function SiteMap() {}
14
15export async function getServerSideProps({ res }: { res: ServerResponse }) {
16  const sitemap = "";
17
18  res.setHeader('Content-Type', 'text/xml');
19  res.write(sitemap);
20  res.end();
21
22  return {
23    props: {},
24  };
25}
26
27export default SiteMap;

Let's fetch our documents that we wish to include within our sitemap.


1import { ServerResponse } from 'http';
2
3import { getClient } from '@/sanity/lib/client';
4import { sitemapQuery } from '@/sanity/lib/queries';
5import { TSitemap } from '@/types';
6
7function generateSiteMap() {
8  return `<?xml version="1.0" encoding="UTF-8"?>
9   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
10     <url>
11       <loc>https://stelko.xyz</loc>
12     </url>
13   </urlset>
14 `;
15}
16
17function SiteMap() {}
18
19export async function getServerSideProps({ res }: { res: ServerResponse }) {
20  const [pages, services, blog, projects] = await Promise.all([
21    getClient().fetch<TSitemap[]>(sitemapQuery('page')),
22    getClient().fetch<TSitemap[]>(sitemapQuery('service')),
23    getClient().fetch<TSitemap[]>(sitemapQuery('blog')),
24    getClient().fetch<TSitemap[]>(sitemapQuery('project')),
25  ]);
26
27  const sitemap = ""
28
29  res.setHeader('Content-Type', 'text/xml');
30  res.write(sitemap);
31  res.end();
32
33  return {
34    props: {},
35  };
36}
37
38export default SiteMap;
39

Now to generate the sitemap and show it off in the browser.


1import { ServerResponse } from 'http';
2
3import { WEBSITE_HOST_URL } from '@/lib/constants';
4import { getClient } from '@/sanity/lib/client';
5import resolveHref from '@/sanity/lib/links';
6import { sitemapQuery } from '@/sanity/lib/queries';
7import { TSitemap } from '@/types';
8
9function generateSiteMap(pages: TSitemap[]) {
10  return `<?xml version="1.0" encoding="UTF-8"?>
11   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
12     <url>
13       <loc>https://stelko.xyz</loc>
14     </url>
15     ${pages
16       .map(({ _type, slug, _updatedAt }) => {
17         return `
18       <url>
19           <loc>${`${WEBSITE_HOST_URL}${resolveHref(_type, slug?.current)}`}</loc>
20           <lastmod>${_updatedAt}</lastmod>
21       </url>
22     `;
23       })
24       .join('')}
25   </urlset>
26 `;
27}
28
29function SiteMap() {}
30
31export async function getServerSideProps({ res }: { res: ServerResponse }) {
32  const [pages, services, blog, projects] = await Promise.all([
33    getClient().fetch(sitemapQuery('page')),
34    getClient().fetch(sitemapQuery('service')),
35    getClient().fetch(sitemapQuery('blog')),
36    getClient().fetch(sitemapQuery('project')),
37  ]);
38
39  const sitemap = generateSiteMap([
40    ...pages,
41    ...services,
42    ...projects,
43    ...blog,
44  ]);
45
46  res.setHeader('Content-Type', 'text/xml');
47  res.write(sitemap);
48  res.end();
49
50  return {
51    props: {},
52  };
53}
54
55export default SiteMap;
56

If you are wondering what WEBSITE_SITE_URL is, in Vercel where I have the app deployed I have the environment key to my domain https://stelko.xyz.

lib/constants.ts
1export const WEBSITE_HOST_URL =
2  process.env.WEBSITE_HOST_URL || 'http://localhost:3000';

And the resolveHref function is used all over my website, not only for the sitemap but also for links.

sanity/lib/links.ts
1export default function resolveHref(
2  documentType?: string,
3  slug?: string
4): string | undefined {
5  switch (documentType) {
6    case 'home':
7      return '/';
8    case 'page':
9      return slug ? `/${slug}` : undefined;
10    case 'blog':
11      return slug ? `/blog/${slug}` : undefined;
12    case 'project':
13      return slug ? `/project/${slug}` : undefined;
14    case 'service':
15      return slug ? `/service/${slug}` : undefined;
16    default:
17      console.warn('Invalid document type:', documentType);
18      return undefined;
19  }
20}
21

Now if you go to the route sitemap.xml on your site you should see your sitemap.

Validating the sitemap created

You can use an online tool such as Code Beautify to validate your sitemap. It let's you copy and paste your sitemap while you are developing to make sure your sitemap is valid before your deploy it to production.

XML sitemap being validated through an online validator, results came back valid

Pushing the sitemap to Google Search Console

If you have not already, go create a Google Search Console account. As it says on the site, "tools and reports that help you measure your site's Search traffic and performance, fix issues and make your site shine in Google Search results".

Once you have created your account, or logged in, under Indexing click on Sitemaps. Enter in your sitemap and click submit.

Below is a screenshot from my Google Search Console, showing that one sitemap URL failed to be fetched while the other was a success. Only difference being a trailing slash.

At first I read this post in the Search Console Help community, the author mentions that it is fine and that Google just has not gotten to your sitemap yet. I waited a month no change...

Then while redoing my portfolio I found this issue on GitHub of others experiencing the same issue. One user suggested added a slash at the end, and what do you know it worked.

Screenshot of Stelko's Google Search Console sitemaps, showing one success and one failure

Conclusion

Generating a sitemap in not only your Next.js app is great step to take in optimising your website for search engines and improving overall user experience. Hopefully, by following this guide you would of been able to successfully generate a sitemap and submit it to Google Search Console for indexing.

If you were not able to achieve that, do not hesitate to get in touch with me and ask questions.