Page 4

If you’ve been following along with my series on using Tumblr as a Next.js CMS, last time we looked at fetching data and re-creating all the pages you’d expect to find on your average blog. That tutorial skipped two items that Tumblr normally handles: the sitemap and RSS feed. They require a little extra attention so in this post we’ll build on the earlier code to generate those files.

Sitemap

The basic Tumblr sitemap contains links to your homepage and each individual post. You can also add any other URLs you want to include (I like to throw in a list of featured tags) but for now we’ll focus on posts.

In the tumblr.js file from the last entry, reuse the CLIENT and getPosts code to create a method to return all posts:

export async function findAll (limit = 50) {
  const client = tumblr.createClient(CLIENT);
  const initialResponse = await getPosts(client, limit, 0);
  const totalPages = Math.floor(initialResponse.total_posts / limit);

  const posts = await [...Array(totalPages).keys()].reduce(async (arr, i) => {
    const response = await getPosts(client, limit * (i + 1));
    return [ ...(await arr), ...response.posts ];
  }, []);

  return { ...initialResponse, posts: [ ...initialResponse.posts, ...posts ] };
}

The Tumblr API documentation isn’t super clear when it comes to the number of posts you can retrieve in one request. It implies that 20 is the maximum but I haven’t found that to be the case in practice so, to make updating easier just in case, the findAll method accepts limit as a parameter.

Next, in the pages directory, add a new file called sitemap.xml.js. The file will return an empty React component and most of the action takes place in getServerSideProps where we’ll get the posts, use them to generate an XML string, and then switch the Content-Type to XML with setHeader.

import { findAll } from '../utils/tumblr';

const Sitemap = () => {};

export async function getServerSideProps ({ res }) {
  const response = await findAll();

  const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 <a href="http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd</a>">
      <url>
        <loc>${process.env.NEXT_PUBLIC_BASE_URL}</loc>
        <changefreq>weekly</changefreq>
        <lastmod>${new Date().toISOString().substring(0, 10)}</lastmod>
      </url>

      ${response.posts.map((post) => `
        <url>
          <loc>${process.env.NEXT_PUBLIC_BASE_URL}${new URL(post.post_url).pathname}</loc>
          <lastmod>${new Date(post.date).toISOString().substring(0, 10)}</lastmod>
        </url>
      `).join('')}
    </urlset>
  `;

  res.setHeader('Content-Type', 'text/xml');
  res.write(sitemap);
  res.end();

  return {
    props: {}
  };
}

One thing to note, I store my blog’s base URL as an environmental variable so you’ll need to either replace NEXT_PUBLIC_BASE_URL above or add it to your .env files to get this working.

RSS

For the RSS feed, we’ll be using the Feed package to handle the data formatting. You can install it with:

yarn add feed

Create a directory called rss inside pages and then add an index.js file inside of it. By default, the Tumblr feed shows the ten most recent posts so we can use the same find method we used before for pages.

The file for the RSS feed will look pretty similar to the sitemap. The main difference is that instead of using string interpolation to generate XML we’ll follow the example from the Feed docs:

import { Feed } from 'feed';
import { find } from '../../utils/tumblr';

const Rss = () => {};

export async function getServerSideProps ({ res }) {
  const response = await find();

  const feed = new Feed({
    title: 'Your Tumblr Title',
    description: 'Your Tumblr description.',
    link: process.env.NEXT_PUBLIC_BASE_URL
  });

  response.posts.forEach((post) => {
    feed.addItem({
      title: post.title || post.summary,
      description: `${post.type === 'photo' ? post.photos.map((photo) => `<img src=${photo.original_size.url} /><br /><br />`) : ''}${post.type === 'video' ? `<iframe width="700" height="383" src="https://www.youtube.com/embed/${post.video.youtube.video_id}" frameborder="0" /><br /><br />` : ''}${post.type === 'link' ? `<a href=${post.url}>${post.title}</a>: ` : ''}${post.type === 'answer' ? `${post.question}: ` : ''}${post.trail[0].content_raw}`,
      link: `${process.env.NEXT_PUBLIC_BASE_URL}${new URL(post.post_url).pathname}`,
      pubDate: new Date(post.date).toISOString().substring(0, 10),
      category: post.tags.map((tag) => { return { name: tag }; })
    });
  });

  res.setHeader('Content-Type', 'text/xml');
  res.write(feed.rss2());
  res.end();

  return {
    props: {}
  };
}

export default Rss;

To match the Tumblr provided feed, I added some post type specific intros before the raw content in the description text.

That should cover sitemaps and RSS. My next post will dig into some of the bells and whistles that inspired the move to a separate app in the first place like code syntax highlighting and custom social sharing images.

Welp, I thought I could start a series of posts, get married shortly after, and not wind up with a huge delay between them. That was overoptimistic. But at long last, here’s my second post on using Tumblr with Next.js. Check out the first one for background on why Tumblr and thoughts on organizing repos or keep reading for pointers on fetching data (with a healthy smattering of mistakes I made that you should avoid).

The Pages

Assuming you’ve already bootstrapped a Next.js app (if you haven’t, follow the instructions here), the first step is figuring out which pages you’ll need to re-create to match your existing Tumblr. For a basic blog, your pages directory should look something like this:

pages:
  ├── page:
  │   └── [page].js
  ├── post:
  │   └── [...id].js
  ├── tagged:
  │   ├── [tag]:
  │   │   └── page:
  │   │       └── [page].js
  │   └── [tag].js
  ├── 404.js
  └── index.js

If your Tumblr has custom pages, you’ll add those directly under /pages. One thing to keep in mind, the Tumblr API doesn’t support saving user questions or submissions so Next.js might not be the best solution if you need that functionality.

Connecting to Tumblr

We’ll be using the tumblr.js NPM package to connect to the Tumblr API. Install it with:

yarn add tumblr.js

For authentication, you’ll need to get an OAuth consumer key, consumer secret, token, and token secret. Head over to Tumblr and register a new application to get your consumer key and consumer secret. Don’t stress over that form too much, you can ignore any non-required fields.

Once you have those values, enter them in the Tumblr API console and click authenticate to get your tokens. Then copy all four and save them to an .env.local file for use in your Next.js app:

TUMBLR_CONSUMER_KEY=****
TUMBLR_CONSUMER_SECRET=****
TUMBLR_TOKEN=****
TUMBLR_TOKEN_SECRET=****

If you’re wondering why I recommend this method instead of using fetch with an API key, that was one of my early mistakes. Everything was fine until I finished the new site and updated my blog’s visibility (hiding it outside of the Tumblr dashboard). At that point, all my requests started failing because I no longer had the proper permissions. This approach should work even if you change your settings.

Fetching Data

Now to write some actual code!

Start by creating a file called tumblr.js. I keep mine in a utils folder but you can put it wherever you’d like. This file will export a find method that uses the package we installed and the secrets we saved in the last step to create a Tumblr client and request blog posts. Here’s a basic example (swap out “yourtumblr” for your Tumblr):

import tumblr from 'tumblr.js';

const CLIENT = {
  consumer_key: process.env.TUMBLR_CONSUMER_KEY,
  consumer_secret: process.env.TUMBLR_CONSUMER_SECRET,
  token: process.env.TUMBLR_TOKEN,
  token_secret: process.env.TUMBLR_TOKEN_SECRET,
  returnPromises: true
};

export async function find (limit = 10, page = 1, id, tag) {
  const client = tumblr.createClient(CLIENT);
  return await getPosts(client, limit, limit * (page - 1), id, tag);
}

function getPosts (client, limit, offset, id, tag) {
  return new Promise ((resolve) => {
    client.blogPosts('yourtumblr.tumblr.com', { limit, offset, id, tag })
      .then((response) => resolve(response))
      .catch(() => resolve({}));
  });
}

The find method accepts limit, page number, post ID, and tag params so it can be reused across all the Next.js pages listed above.

Static Props & Paths

Using the same find method means writing similar getStaticProps and getStaticPaths blocks for every page so I’ll just run through index.js and post/[...id].js here. If you want more examples, check out the pages directory in my blog repo (although I do a little additional post parsing that I haven’t mentioned yet).

The index.js page uses the default params and is fairly simple:

import { find } from '../utils/tumblr';

...

export async function getStaticProps () {
  const response = await find();

  return {
    props: response,
    revalidate: 3600
  };
}

Things get a little more complicated in post/[...id].js. We’ll be using incremental static regeneration to avoid hitting Tumblr’s 300 API calls per minute rate limit during the build process. (My second early mistake: when I first started, ISG was brand new so I tried other workarounds for limiting requests like adapting this method for caching data globally Although it technically worked, it caused a lot of extra deploys and stale data.)

In the code below, I’m statically generating the most recent 20 posts on my blog and revalidating after an hour (3600 seconds):

import { find } from '../../utils/tumblr'; 

...

export async function getStaticPaths () {
  const response = await find(20);

  return {
    paths: response.posts.map((post) => {
      const params = new URL(post.post_url).pathname.replace('/post/', '').split('/');
      return { params: { id: [ params[0] , params[1] || '' ] }};
    }),
    fallback: 'blocking'
  };
}

export async function getStaticProps ({ params }) {
  const response = await find(1, 1, params.id[0]);

  if (!(response.posts || [])[0]) {
    return { notFound: true };
  }

  return {
    props: {
      post: response.posts[0]
    },
    revalidate: 3600
  };
}

You can play around with the number of posts per page and the time between revalidations to see what works for you. If Tumblr isn’t able to find any posts with the current ID, the 404 page renders instead.

That’s it for now! Up next, sitemaps and RSS.

Thought I would do a series of posts on moving my blog from a Tumblr-hosted and themed site to a Tumblr powered Next.js app.

You might reasonably ask, “Why would anyone want to do that when there are so many actual CMS options available?”.

In my case, it’s mostly sentimental. Way back when I was deciding whether to switch careers and try coding full time, I took a lot of freelance jobs converting PSDs to Tumblr themes and I’ve been using it ever since.

But, I finally wanted a little more control over my blog and a good excuse to experiment more with Next.js so I decided to mix it up just a little. Let’s dig into some of the decision making, adventures in data fetching, and general idiosyncrasies that come with using Tumblr as a CMS.

How To Set It Up

Starting at the beginning, the very first question was how to set everything up. Where should the site and the code live? How do you share the parts that need to be shared? It seems simple but there are a lot of options.

Keeping with the original Tumblr setup, I decided to create a new app for my blog instead of trying to integrate it with the rest of my site (which was already using Next.js). I wanted to use Vercel for hosting which meant I would just need to update my subdomain records once everything was up and running.

Since Tumblr templates are basically HTML files that include Tumblr’s custom blocks and variables, I’d never really shared much code between my blog and the rest of my site even though they look pretty similar. I had a small script to generate stylesheets but that was it. Using Next.js for both opened up the possibility of sharing React components, variables, and utilities in addition to CSS.

To figure out how to do that, I followed one article (6 Ways to Share React Components in 2020) to another (4 Git Submodules Alternatives You Should Know) and finally landed at Git subtree: the alternative to Git submodule for step-by-step instructions.

Any solution that required publishing components, individually or as a monorepo, seemed like overkill for a smallish, one dev project. Git subtrees provided a way to nest one repo as a subdirectory in others which was exactly what I wanted. I split out shared components like the header, footer, form elements, etc. into a separate repo, ran the commands from the tutorial above in my blog and site repos to hook it all up, and voila. No copying and pasting duplicate code between sections and no publishing to NPM or other third-parties.

So that’s the foundation, stay tuned for the next installment for data fetching and some actual code!

Well look at that, it’s a font I made for a change.

Quintet is a narrow, loopy font that tries to capture an old-timey, swing jazzy feel. It also comes with the instrument SVG illustrations I created for the poster images.

It’s free for personal use so go download it to try now. And definitely let me know if you run into any installation issues.