Express.js, MongoDB, Next.js

Express.js to Next.js API Routes Migration

This if a follow up on my previous post Explore Next.js 9 API Routes.

After much consideration, I decided to ditch Express.js and moved into API Routes.

Migrating to Next.js API Routes

In order to use the new API Routes, I need to update my Next.js module to v9 by running:

npm i next@latest react@latest react-dom@latest

This updates Next.js along with React.js to the latest versions.

Although this is a major update, I did not find any breaking changes that affect me in particular. However, if there was any for you, there is this upgrade guide to help you resolve any problems.

Rewrite codebase - more like, a lot of copypastes

Express.js to Next 9 API Routes

In my current express.js Server, to access an endpoint at /api/authenticate, my code in /server/components/account/accountController.js is:

// accountController.js
const express = require("express");


const User = require("../../api/models/userModel");


// In server.js, I called app.use('/api', AccountController);
const AccountController = express.Router();


AccountController.post("/authenticate", (req, res) => {
  const { email, password } = req.body;
  User.findByCredentials(email, password)
    .then((user) => user.generateSessionId())
    .then((sessionId) => {
      const { name } = user;
      res
        .cookie("sessionId", sessionId, { httpOnly: true, secure: true })
        .send(`welcome my homie, ${name}`);
    })
    .catch((e) => {
      // reject due to wrong email or password
      res.status(401).send("who are u, i dun know u, go away");
    });
});
module.exports = AccountController;

You can see how I made use of req and res. Let's look into the Next.js 9 API Routes' way:

export default function handle(req, res) {
  res.end("Hello World");
}

The handle function has the same syntax: it takes the same req and res. Better yet, Next.js 9's API Routes implements the similar Express.js's Middlewares, including parser req.body and helper function res.status and res.send. This means I do not have to make lots of changes.

// FIXME: No res.cookie in Next.js API Routes

It appears that there is no res.cookie helper function in Next.js 9 API Routes. I need to rewrite the function, falling back to http.ServerResponse's setHeader (Because NextApiResponse extends http.ServerResponse): res.cookie("sessionId", sessionId, { httpOnly: true, secure: true }) becomes res.setHeader('Set-Cookie', `sessionId=${sessionId}; HttpOnly; Secure`).

I have created a feature request on zeit/next.js for adding res.cookie. I hope they will add it. For now, I have to stick with res.setHeader.

// TODO: Creating API Routes version of /api/authenticate

I created pages/api/authenticate.js.

// authenticate.js
export default (req, res) => {
  const { email, password } = req.body;
  User.findByCredentials(email, password)
    .then((user) => user.generateSessionId())
    .then((sessionId) => {
      const { name } = user;
      res
        .setHeader("Set-Cookie", `sessionId=${sessionId}; HttpOnly; Secure`)
        .send(`welcome my homie, ${name}`);
    })
    .catch((e) => {
      // reject due to wrong email or password
      res.status(401).send("who are u, i dun know u, go away");
    });
};

Perfect, that was how I transformed my code from Express.js to Next.js API Routes: Just copypaste and make small touches. By doing so, I just ditched Express Router, making the code so much cleaner. I went and did the same thing for every single API endpoint.

No database connection

Back at the Express.js version, my npm start runs this server.js script:

const express = require("express");
const mongoose = require("mongoose");
const AccountController = require("./components/account/accountController");
const app = express();
mongoose.connect(process.env.MONGODB_URI, {
  useNewUrlParser: true,
  useFindAndModify: false,
  useCreateIndex: true,
});
app.use("/api", AccountController);
app.listen(process.env.PORT);

Notice that mongoose.connect() is how I connected to the database. Routing to /api/authenticate was then handled by app.use("/api", AccountController);.

Let's take a look at this diagram I draw:

No MongoDB connection There is no MongoDB connection in the Next.js version

As you can see, in the Express.js version, the server stays running and maintains the connection. However, in the Next.js version, the server does not have a starting point where it initializes the connection.

How about adding mongoose.connect() on every single endpoints (every single .js under /pages/api. Well, that's not quite the case.

Imagine each time an API Route is hit, it calls mongoose.connect(). Therefore, multiple mongoose.connect()will be called. However, you can only call mongoose.connect() once. Else, you will get this error:

MongooseError: You can not 'mongoose.connect()' multiple times while connected

// TODO: Maintain only one Mongoose connection

There must be a way to check if there is a mongoose connection. We only attempt to connect if there isn't one.

This is my approach:

// db.js
import mongoose from "mongoose";


export default async () => {
  if (mongoose.connections[0].readyState) return;
  // Using new database connection
  await mongoose.connect(process.env.MONGODB_URI, {
    useNewUrlParser: true,
    useFindAndModify: false,
    useCreateIndex: true,
  });
};

After successfully connecting to MongoDB, mongoose.connections[0].readyState will be 1 (true). Next time when the function is called, it will simply return.

What is left to do is to import the function from db.js in every API endpoint.

// authenticate.js
import connectToDb from "../../../api/db";


export default async (req, res) => {
  await connectToDb();


  const { email, password } = req.body;
  User.findByCredentials(email, password)
    .then((user) => user.generateSessionId())
    .then((sessionId) => {
      const { name } = user;
      res
        .setHeader("Set-Cookie", `sessionId=${sessionId}; HttpOnly; Secure`)
        .send(`welcome my homie, ${name}`);
    })
    .catch((e) => {
      // reject due to wrong email or password
      res.status(401).send("who are u, i dun know u, go away");
    });
};

I made the handler an async function so that I can use the keyword await on connectToDb(). By having the await keyword, we are making sure that connectToDB() is completed before anything else.

That's it!

The alternative way: Using middleware

A "middleware" can be achieved by wrapping the handler function.

Create a dbMiddleware.js:

import mongoose from "mongoose";


const connectDb = (handler) => async (req, res) => {
  if (mongoose.connections[0].readyState) return handler(req, res);
  // Using new database connection
  await mongoose.connect(process.env.MONGODB_URI, {
    useNewUrlParser: true,
    useFindAndModify: false,
    useCreateIndex: true,
  });
  return handler(req, res);
};


export default connectDb;

After that in my API functions, I wrap the handler function.

import connectDb from "../../../api/middlewares/dbMiddleware.js";
const handler = (req, res) => {
  const { email, password } = req.body;
  User.findByCredentials(email, password)
    .then((user) => user.generateSessionId())
    .then((sessionId) => {
      const { name } = user;
      res
        .setHeader("Set-Cookie", `sessionId=${sessionId}; HttpOnly; Secure`)
        .send(`welcome my homie, ${name}`);
    })
    .catch((e) => {
      // reject due to wrong email or password
      res.status(401).send("who are u, i dun know u, go away");
    });
};
export default connectDb(handler);

Learn more about it in Middleware in Next.js.

TODO: Be consistent when import and export

When using Express.js, I did not implement Babel and could not use ES6 Import / Export.

When I started to using API Routes (which includes Babel), I changed part of the codebase to use ES6 Import or ES6 Export. However, several functions still used module.exports. This caused an issue that I mention below. (see FIXME: ... is not a function).

Therefore, be consistent. I recommend using ES6 import/export for the whole codebase.

Miscellaneous issues

// FIXME: Blank page with no error

Note: This particular issue I had below was not originated from Next.js. You can skip it!

One of the problems I got was that when I ran next dev, the terminal shows build page: /, compiling... then compiled successfully. However, when I visited http://localhost/, I saw an empty page with the tab's status bar showing loading indication.

Blank loading page

When I looked at the Network tab, I saw GET localhost:3000/ kept on running with no response. (no Status, Response header, payload).

What so annoying about this issue was that there was no 500 internal server error or any red error texts in Console.

I looked through my code and checked all the syntax. Everything looked fine. I mean I just copied pasted a working version of my code to the new format. If there was an error in my code, it should had happened before I made the migration.

Luckily, when I attempted to run next build, I saw the error:

node-sass error I found out only when doing my next build

What was node-sass doing? It was totally irrelevant. Then, I thought of that stupid IT joke "Have You Tried Turning It Off And On Again?". Well, no, I did not literally restart my computer. What I did was to run npm rebuild. This allowed me to "reset/restart" the node modules (which of course include node-sass). It just magically worked. Deleting my node_modules folder and running npm install would achieve the same thing.

When in doubt, run npm rebuild

Running next build now showed compiled successfully and running next dev worked: No more blank page... Well, but now we had some 500 Internal Server error

// FIXME: ... is not a function

If you run the production version, you may encounter UnhandledPromiseRejectionWarning: TypeError: ... is not a function.

After some trials and errors, I noticed it that if I used ES6 import instead of require, the error went away.

I guessed that for some reason, webpack did not parse require correctly. I noticed in my code that I used two different variants: I imported the function by require but exported it by export default. It might be the cause of the problem.

Therefore, go ahead and change from require / modules.export to import / export. If you do not specific export *default*, you will have to explicitly mention the name of the function. For example:

import { nameOfFunction } from "path/to/theFunction";

// FIXME: Cannot overwrite model once compiled

I think it is not actually your error. You may think that it is because you import the model.js file multiple times. Back when I used Express.js, I had to do the same thing but did not encounter this problem. I suspect it was due to Hot Module Replacement (HMS). Because HMS compiles on-the-go, there is a chance that model.js gets compiled more than once, causing the problem.

I tested out my theory by trying to serve a production build using next build and next start. There was no error because Webpack did not do its compilation then.

Here is a workaround for the problem:

export default mongoose.models.User || mongoose.model("User", UserSchema);

As you can see, we first see if mongoose.models.User exists and only model it if it does not.

Hello Next.js API Routes, Goodbye Express.js

Uninstall redundant dependencies

Since we are no longer using Express.js, it is always a good idea to remove it.

With Express.js, I also need to uninstall along with two dependencies: nodemon and cookie-parser. I

I used to need nodemon to restart my server when I make a change to the code. This is no longer needed as I will use Webpack's Hot Module Replacement from now on.

I used to need cookie-parser to access req.cookies. This is no longer needed because Next.js 9 has already provided a way to do so.

I went ahead and uninstall them by running:

npm uninstall express nodemon cookie-parser

Make sure to remove any import / require of the mentioned dependencies from the code.

Change the scripts in package.json

In my Express.js version, my scripts were:

"scripts": {
    "dev": "nodemon server/server.js",
    "build": "next build",
    "start": "cross-env NODE_ENV=production node server/server.js",
 }

For npm run dev, I nodemon my custom server server.js. For npm run start, I node my server.js.

Moving to API Routes, I no longer need a custom server or hot reloading. All I have to do is run next dev and next start.

"scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
}

Conclusion

I managed to change my code base to using Next.js API Routes. It opens the possibility of serverless, which I will explore soon.

I still run into issues with this new Next.js API Routes now and then. When I do, I will make sure to include it in this article. Good luck deploying your Next.js API Routes.