Posts tagged code

Next.js and Tumblr as a CMS Part 3: Sitemap and RSS

If you’ve been following along with my series on using Tumblr as a Next.js CMS, last time we looked at fetching data and re-creating all the pages you’d expect to find on your average blog. That tutorial skipped two items that Tumblr normally handles: the sitemap and RSS feed. They require a little extra attention so in this post we’ll build on the earlier code to generate those files.

Sitemap

The basic Tumblr sitemap contains links to your homepage and each individual post. You can also add any other URLs you want to include (I like to throw in a list of featured tags) but for now we’ll focus on posts.

In the tumblr.js file from the last entry, reuse the CLIENT and getPosts code to create a method to return all posts:

export async function findAll (limit = 50) {
  const client = tumblr.createClient(CLIENT);
  const initialResponse = await getPosts(client, limit, 0);
  const totalPages = Math.floor(initialResponse.total_posts / limit);

  const posts = await [...Array(totalPages).keys()].reduce(async (arr, i) => {
    const response = await getPosts(client, limit * (i + 1));
    return [ ...(await arr), ...response.posts ];
  }, []);

  return { ...initialResponse, posts: [ ...initialResponse.posts, ...posts ] };
}

The Tumblr API documentation isn’t super clear when it comes to the number of posts you can retrieve in one request. It implies that 20 is the maximum but I haven’t found that to be the case in practice so, to make updating easier just in case, the findAll method accepts limit as a parameter.

Next, in the pages directory, add a new file called sitemap.xml.js. The file will return an empty React component and most of the action takes place in getServerSideProps where we’ll get the posts, use them to generate an XML string, and then switch the Content-Type to XML with setHeader.

import { findAll } from '../utils/tumblr';

const Sitemap = () => {};

export async function getServerSideProps ({ res }) {
  const response = await findAll();

  const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 <a href="http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd</a>">
      <url>
        <loc>${process.env.NEXT_PUBLIC_BASE_URL}</loc>
        <changefreq>weekly</changefreq>
        <lastmod>${new Date().toISOString().substring(0, 10)}</lastmod>
      </url>

      ${response.posts.map((post) => `
        <url>
          <loc>${process.env.NEXT_PUBLIC_BASE_URL}${new URL(post.post_url).pathname}</loc>
          <lastmod>${new Date(post.date).toISOString().substring(0, 10)}</lastmod>
        </url>
      `).join('')}
    </urlset>
  `;

  res.setHeader('Content-Type', 'text/xml');
  res.write(sitemap);
  res.end();

  return {
    props: {}
  };
}

One thing to note, I store my blog’s base URL as an environmental variable so you’ll need to either replace NEXT_PUBLIC_BASE_URL above or add it to your .env files to get this working.

RSS

For the RSS feed, we’ll be using the Feed package to handle the data formatting. You can install it with:

yarn add feed

Create a directory called rss inside pages and then add an index.js file inside of it. By default, the Tumblr feed shows the ten most recent posts so we can use the same find method we used before for pages.

The file for the RSS feed will look pretty similar to the sitemap. The main difference is that instead of using string interpolation to generate XML we’ll follow the example from the Feed docs:

import { Feed } from 'feed';
import { find } from '../../utils/tumblr';

const Rss = () => {};

export async function getServerSideProps ({ res }) {
  const response = await find();

  const feed = new Feed({
    title: 'Your Tumblr Title',
    description: 'Your Tumblr description.',
    link: process.env.NEXT_PUBLIC_BASE_URL
  });

  response.posts.forEach((post) => {
    feed.addItem({
      title: post.title || post.summary,
      description: `${post.type === 'photo' ? post.photos.map((photo) => `<img src=${photo.original_size.url} /><br /><br />`) : ''}${post.type === 'video' ? `<iframe width="700" height="383" src="https://www.youtube.com/embed/${post.video.youtube.video_id}" frameborder="0" /><br /><br />` : ''}${post.type === 'link' ? `<a href=${post.url}>${post.title}</a>: ` : ''}${post.type === 'answer' ? `${post.question}: ` : ''}${post.trail[0].content_raw}`,
      link: `${process.env.NEXT_PUBLIC_BASE_URL}${new URL(post.post_url).pathname}`,
      pubDate: new Date(post.date).toISOString().substring(0, 10),
      category: post.tags.map((tag) => { return { name: tag }; })
    });
  });

  res.setHeader('Content-Type', 'text/xml');
  res.write(feed.rss2());
  res.end();

  return {
    props: {}
  };
}

export default Rss;

To match the Tumblr provided feed, I added some post type specific intros before the raw content in the description text.


That should cover sitemaps and RSS. My next post will dig into some of the bells and whistles that inspired the move to a separate app in the first place like code syntax highlighting and custom social sharing images.

Next.js and Tumblr as a CMS Part 2: Data Fetching

Welp, I thought I could start a series of posts, get married shortly after, and not wind up with a huge delay between them. That was overoptimistic. But at long last, here’s my second post on using Tumblr with Next.js. Check out the first one for background on why Tumblr and thoughts on organizing repos or keep reading for pointers on fetching data (with a healthy smattering of mistakes I made that you should avoid).

The Pages

Assuming you’ve already bootstrapped a Next.js app (if you haven’t, follow the instructions here), the first step is figuring out which pages you’ll need to re-create to match your existing Tumblr. For a basic blog, your pages directory should look something like this:

pages:
  ├── page:
  │   └── [page].js
  ├── post:
  │   └── [...id].js
  ├── tagged:
  │   ├── [tag]:
  │   │   └── page:
  │   │       └── [page].js
  │   └── [tag].js
  ├── 404.js
  └── index.js

If your Tumblr has custom pages, you’ll add those directly under /pages. One thing to keep in mind, the Tumblr API doesn’t support saving user questions or submissions so Next.js might not be the best solution if you need that functionality.

Connecting to Tumblr

We’ll be using the tumblr.js NPM package to connect to the Tumblr API. Install it with:

yarn add tumblr.js

For authentication, you’ll need to get an OAuth consumer key, consumer secret, token, and token secret. Head over to Tumblr and register a new application to get your consumer key and consumer secret. Don’t stress over that form too much, you can ignore any non-required fields.

Once you have those values, enter them in the Tumblr API console and click authenticate to get your tokens. Then copy all four and save them to an .env.local file for use in your Next.js app:

TUMBLR_CONSUMER_KEY=****
TUMBLR_CONSUMER_SECRET=****
TUMBLR_TOKEN=****
TUMBLR_TOKEN_SECRET=****

If you’re wondering why I recommend this method instead of using fetch with an API key, that was one of my early mistakes. Everything was fine until I finished the new site and updated my blog’s visibility (hiding it outside of the Tumblr dashboard). At that point, all my requests started failing because I no longer had the proper permissions. This approach should work even if you change your settings.

Fetching Data

Now to write some actual code!

Start by creating a file called tumblr.js. I keep mine in a utils folder but you can put it wherever you’d like. This file will export a find method that uses the package we installed and the secrets we saved in the last step to create a Tumblr client and request blog posts. Here’s a basic example (swap out “yourtumblr” for your Tumblr):

import tumblr from 'tumblr.js';

const CLIENT = {
  consumer_key: process.env.TUMBLR_CONSUMER_KEY,
  consumer_secret: process.env.TUMBLR_CONSUMER_SECRET,
  token: process.env.TUMBLR_TOKEN,
  token_secret: process.env.TUMBLR_TOKEN_SECRET,
  returnPromises: true
};

export async function find (limit = 10, page = 1, id, tag) {
  const client = tumblr.createClient(CLIENT);
  return await getPosts(client, limit, limit * (page - 1), id, tag);
}

function getPosts (client, limit, offset, id, tag) {
  return new Promise ((resolve) => {
    client.blogPosts('yourtumblr.tumblr.com', { limit, offset, id, tag })
      .then((response) => resolve(response))
      .catch(() => resolve({}));
  });
}

The find method accepts limit, page number, post ID, and tag params so it can be reused across all the Next.js pages listed above.

Static Props & Paths

Using the same find method means writing similar getStaticProps and getStaticPaths blocks for every page so I’ll just run through index.js and post/[...id].js here. If you want more examples, check out the pages directory in my blog repo (although I do a little additional post parsing that I haven’t mentioned yet).

The index.js page uses the default params and is fairly simple:

import { find } from '../utils/tumblr';

...

export async function getStaticProps () {
  const response = await find();

  return {
    props: response,
    revalidate: 3600
  };
}

Things get a little more complicated in post/[...id].js. We’ll be using incremental static regeneration to avoid hitting Tumblr’s 300 API calls per minute rate limit during the build process. (My second early mistake: when I first started, ISG was brand new so I tried other workarounds for limiting requests like adapting this method for caching data globally Although it technically worked, it caused a lot of extra deploys and stale data.)

In the code below, I’m statically generating the most recent 20 posts on my blog and revalidating after an hour (3600 seconds):

import { find } from '../../utils/tumblr'; 

...

export async function getStaticPaths () {
  const response = await find(20);

  return {
    paths: response.posts.map((post) => {
      const params = new URL(post.post_url).pathname.replace('/post/', '').split('/');
      return { params: { id: [ params[0] , params[1] || '' ] }};
    }),
    fallback: 'blocking'
  };
}

export async function getStaticProps ({ params }) {
  const response = await find(1, 1, params.id[0]);

  if (!(response.posts || [])[0]) {
    return { notFound: true };
  }

  return {
    props: {
      post: response.posts[0]
    },
    revalidate: 3600
  };
}

You can play around with the number of posts per page and the time between revalidations to see what works for you. If Tumblr isn’t able to find any posts with the current ID, the 404 page renders instead.

That’s it for now! Up next, sitemaps and RSS.

Next.js and Tumblr as a CMS

Thought I would do a series of posts on moving my blog from a Tumblr-hosted and themed site to a Tumblr powered Next.js app.

You might reasonably ask, “Why would anyone want to do that when there are so many actual CMS options available?”.

In my case, it’s mostly sentimental. Way back when I was deciding whether to switch careers and try coding full time, I took a lot of freelance jobs converting PSDs to Tumblr themes and I’ve been using it ever since.

But, I finally wanted a little more control over my blog and a good excuse to experiment more with Next.js so I decided to mix it up just a little. Let’s dig into some of the decision making, adventures in data fetching, and general idiosyncrasies that come with using Tumblr as a CMS.


How To Set It Up

Starting at the beginning, the very first question was how to set everything up. Where should the site and the code live? How do you share the parts that need to be shared? It seems simple but there are a lot of options.

Keeping with the original Tumblr setup, I decided to create a new app for my blog instead of trying to integrate it with the rest of my site (which was already using Next.js). I wanted to use Vercel for hosting which meant I would just need to update my subdomain records once everything was up and running.

Since Tumblr templates are basically HTML files that include Tumblr’s custom blocks and variables, I’d never really shared much code between my blog and the rest of my site even though they look pretty similar. I had a small script to generate stylesheets but that was it. Using Next.js for both opened up the possibility of sharing React components, variables, and utilities in addition to CSS.

To figure out how to do that, I followed one article (6 Ways to Share React Components in 2020) to another (4 Git Submodules Alternatives You Should Know) and finally landed at Git subtree: the alternative to Git submodule for step-by-step instructions.

Any solution that required publishing components, individually or as a monorepo, seemed like overkill for a smallish, one dev project. Git subtrees provided a way to nest one repo as a subdirectory in others which was exactly what I wanted. I split out shared components like the header, footer, form elements, etc. into a separate repo, ran the commands from the tutorial above in my blog and site repos to hook it all up, and voila. No copying and pasting duplicate code between sections and no publishing to NPM or other third-parties.

So that’s the foundation, stay tuned for the next installment for data fetching and some actual code!

React Inner Image Zoom Version 3.0.0

React Inner Image Zoom version 3.0.0 went out earlier this week with a handful of bug fixes, test and build improvements, and one major change.

What’s the big thing to look out for? By popular demand, the imgAttributes prop was added to pass down (almost) any valid React img attributes in a single object instead of as individual props. That means scrSet, sizes, alt, and title are gone but in exchange you get all the data attributes and event handlers you could want. I haven’t submitted updated type definitions to DefinitelyTyped yet but I’ll try to get that done in the next few days.

This release also included a handy new Changelog so I would be remiss not include the official record here:

Changed

  • Replaced srcSet, sizes, alt, and title props with imgAttributes to set the original image’s attributes.
  • Show close button when moveType is set to “drag” on all breakpoints.
  • Switched from setTimeout to onTransitionEnd to check that zoomed image has finished fading out.

Added

  • This handy CHANGELOG.

Fixed

  • Added stopPropagation on touchmove to prevent events below fullscreen modal.

If you run into any bugs, please let me know in the GitHub issues.

Vue Inner Image Zoom v2

As promised, now that migrating is easier I’ve updated my Vue Inner Image Zoom component to support Vue 3. If you’re still on Vue 2 and want to use the component, just make sure to install it as vue-inner-image-zoom@1.1.1.

I also updated the demos site to remove the lazy loading example since vue-lazyload isn’t compatible with Vue 3 (I’m open to any suggestions for replacements) and switched from vue-slick-carousel to Swiper both for compatibility and because it’s my preferred carousel library.

If I broke anything and you run into any new bugs, please report them on the GitHub issues page.

The Mysterious Case of Emotion and “exports is not defined”

Thought I’d share a bug I ran into a while back that sent me on a Poirot style investigation full of red herrings and unexpected culprits.

This is tangentially related to my recent page speed woes at work. We’d started using Emotion for CSS-in-JS in our component library and, combined with lazy and conditional component loading, it helped with some of the “Reduce unused CSS” warnings we were seeing in Lighthouse.

So adding Emotion as a styling option in our main codebase seemed like an obvious choice. We’d already installed @emotion/core (v10) when we started importing from our component library which meant it should be a quick, two-step process: 1) running yarn add @emotion/babel-preset-css-prop and 2) adding it to our babel.config.js presets after @babel/preset-react. I followed those steps, ran Webpack, and promptly got the error “ReferenceError: exports is not defined”.

Weird.

Moving a Node App from Google Compute Engine to DigitalOcean

This site can

There’s nothing like trying to deploy a minor site update and having your VM hang while SSHing 75% of the time, give up mid-build the other 25%, and then fall apart completely when you try to restart it. Since all I remember about setting up a Google Compute Engine Node instance in the first place is that it took me a while and wasn’t particularly intuitive, this seemed like a good opportunity to try a new host. I’m running a commercial Next.js site with a custom server so after browsing around DigitalOcean seemed like the best option.

And it was actually pretty painless!

Of course, it would be even quicker with a step-by-step guide so here’s a reference for my future self or anyone else who wants it.

My guide for migrating a Node app to DigitalOcean (this may vary a little depending on your site):

  1. Sign up for DigitalOcean and create a Droplet using NodeJS 1-Click App. Poking around the DigitalOcean Community questions, I ran into some talk about additional installations for production but my Droplet came with Nginx, PM2, and optional Let’s Encrypt for SSL out of the box so those discussions may be a little out of date.
  2. Add the new IPv4 address to MongoDB Atlas. DigitalOcean provides instructions for getting started with your Droplet but I wound up skipping around a bit. Since I already had a MongoDB database set up, I completely ignored their section on creating a new one. To use an existing database, just remember to log in to MongoDB Atlas and add your new IP address (using port 22) to the IP Address List under Network Access.
  3. Clone your repo. Using ssh root@0.0.0.0 (replacing the zeros with your IP address), ssh into your Droplet and clone your repo.
  4. Copy environmental variable files, assets, etc. From outside your Droplet, use scp -r /path/on/computer root@0.0.0.0:/path/on/droplet (replacing the paths and zeros) to copy any necessary files that weren’t included in the repo.
  5. Start your app. Back in the Droplet, start with a good old yarn install. Then, for a Next.js site, run yarn build followed by sudo NODE_ENV=production -u nodejs pm2 start server.js to start a custom server. This is also a good time to jump back to the DigitalOcean instructions for shutting down the starter app with sudo -u nodejs pm2 delete hello and saving your PM2 process list with sudo -u nodejs pm2 save.
  6. Create an Nginx configuration. I added “read up on everything Nginx can do” to my todo list but for now I have a pretty simple config with basic asset caching in my /etc/nginx/sites-enabled/ directory. You can view the file in this Gist but keep in mind it’s pretty tailored to my SSL setup. Whether you alter my example, the DigitalOcean default, or start fresh, make sure to 1) switch the root setting to something like /root/your-site/.next/pages; for Next.js and 2) update the proxy_pass to use your app’s port (8443 in my example). Restart with sudo systemctl restart nginx.
  7. Point your domain to the new IPv4 address. How you do this will depend on your DNS host but you’ll want to complete the migration by updating your A name to point to your new IP address.

There it is in seven steps. The first run through took a bit of trial and error but hopefully this will speed things up next time.