Next.js and MongoDB full-fledged app Part 4: Posts and Comments

It is not a social app until you can post and comment. Let's learn how to do so with a pagination API, infinite loading UI pattern, and more.

This is a follow-up to Part 3. Make sure you read it before this post.

Again, Below are the Github repository and a demo for this project to follow along.

Github repo

Demo

About `nextjs-mongodb-app` project

> nextjs-mongodb-app is a Full-fledged serverless app made with Next.JS and MongoDB

Different from many other Next.js tutorials, this:

Does not use the enormously big Express.js, supports serverless
Minimal, no fancy stuff like Redux or GraphQL for simplicity in learning
Using Next.js latest features like API Routes or getServerSideProps

For more information, visit the Github repo.

What we are making

We are working on the social media features as seen on Facebook or Twitter:

Post Feature allows you to create a post
Comment Feature allows you to comment on such posts

Build the posts feature

Create Post API

Let's build an API to create a post at POST /api/posts. Create /pages/api/posts/index.js:

import { auths, database, validateBody } from "@/api-lib/middlewares";
import nc from "next-connect";

const handler = nc();

handler.use(database);

handler.post(
  ...auths,
  validateBody({
    type: "object",
    properties: {
      content: { type: "string", minLength: 1, maxLength: 280 },
    },
    required: ["content"],
    additionalProperties: false,
  }),
  async (req, res) => {
    if (!req.user) {
      return res.status(401).end();
    }

    const post = {
      content: req.body.content,
      creatorId: req.user._id,
      createdAt: new Date(),
    };

    const { insertedId } = await req.db.collection("posts").insertOne(post);
    post._id = insertedId;

    return res.json({ post });
  }
);

export default handler;

For this API, we need to use the database and auths middlewarefor database connection and authentication. However, notice that I only use the auths in .post() instead of putting it in .use(). The reason is that a later API in this file (GET posts) does not require authentication.

We first pass the request through our validateBody for validation. Currently, let's limit the length of the post to 280 characters (Twitter's).

We first check if the user is authenticated using req.user. Then, if he or she is, we created and inserted the post into the posts collection. The post _id is not known beforehand so we attach it (insertedId) later and return the post object.

Create Post UI

We can create a component that shows an input and a submit button that allow users to publish a post.

import { useCurrentUser } from "@/lib/user";
import Link from "next/link";
import { useCallback, useRef } from "react";

const Poster = () => {
  const { data, error } = useCurrentUser();
  const contentRef = useRef();

  const onSubmit = useCallback(
    async (e) => {
      e.preventDefault();
      try {
        await fetcher("/api/posts", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ content: contentRef.current.value }),
        });
        contentRef.current.value = "";
      } catch (e) {
        console.error(e.message);
      }
    },
    [mutate]
  );

  return (
    <div>
      <h3>Share your thoughts</h3>
      {data?.user ? (
        <form onSubmit={onSubmit}>
          <input
            ref={contentRef}
            placeholder={`What's on your mind, ${data.user.name}?`}
          />
          <button type="submit">Post</button>
        </form>
      ) : (
        <p>
          Please{" "}
          <Link href="/login">
            <a>sign in</a>
          </Link>{" "}
          to post
        </p>
      )}
    </div>
  );
};

Since the user must be authenticated, we use our useCurrentUser hook to get the current user and show a message if the hook return user = null.

On submission, we send the POST request to our just created API and reset the input content afterward.

Get Posts API with pagination

Let's build an API to get all posts at GET /api/posts. Create pages/api/posts/index.js:

import { database } from "@/api-lib/middlewares";
import nc from "next-connect";

const handler = nc();

handler.use(database);

handler.get(async (req, res) => {
  const posts = req.db
    .collection("posts")
    .find()
    .sort({ _id: -1 }) // sort by insertion order
    .toArray();

  res.json({ posts });
});

export default handler;

The above is sufficient to retrieve a list of all posts in the database. It would return the following:

[
  {
    "_id": "614dafac52fd31818950e464",
    "content": "Second.",
    "creatorId": "614dacd052fd31818950e463",
    "createdAt": "2021-09-24T10:59:56.194Z"
  },
  {
    "_id": "614dacda09d00ffae6abf2e4",
    "content": "First!",
    "creatorId": "614dacd052fd31818950e463",
    "createdAt": "2021-09-24T10:47:54.790Z"
  }
]

However, what if we want to get the creator info on each post? Introducing: MongoDB aggregation.

const posts = req.db
  .collection("posts")
  .aggregate([
    { $sort: { _id: -1 } },
    {
      $lookup: {
        from: "users",
        localField: "creatorId",
        foreignField: "_id",
        as: "creator",
      },
    },
    { $unwind: "$creator" },
    {
      $project: {
        "creator.password": 0,
        "creator.email": 0,
        "creator.emailVerified": 0,
        "creator.bio": 0,
      },
    },
  ])
  .toArray();

The aggregation pipeline runs through multiple stages to transform the documents. With the above, we:

Sort the posts using the $sort. Reverse sorting the _id allows us to list the posts in the reverse of the insertion order.
Use $lookup to find a document from users collection where the local field posts.creatorId is equal to the foreign field user._id, then set it to creator. This is similar to performing a "left outer join" in a tranditional SQL database.
However, creator is then an array (since we find all occurrences that match the above equality), so we use $unwind and turn it into a single object. (Note: $unwind actually outputs multiple documents for each element of the array, but since we know we always have only one, we "abuse" it to deconstruct the array into a single element.)
Use $project to hide the sensitive fields that come from the user document, similar to how we did in [Endpoint to get the current user](/blog/next-js-and-mongodb-app-1#endpoint-to-get-the-current user). We can also remove fields we don't need such as bio, to reduce bandwidth usage.

With that, the result is now:

[
  {
    "_id": "614dafac52fd31818950e464",
    "content": "Second.",
    "creatorId": "614dacd052fd31818950e463",
    "createdAt": "2021-09-24T10:59:56.194Z",
    "creator": {
      "_id": "614dacd052fd31818950e463",
      "profilePicture": "https://res.cloudinary.com/dbplcha6k/image/upload/v1632480534/gk9vbleo0nioprpx3mm3.jpg",
      "name": "Hoang Vo",
      "username": "hoangvvo"
    }
  },
  {
    "_id": "614dacda09d00ffae6abf2e4",
    "content": "First!",
    "creatorId": "614dacd052fd31818950e463",
    "createdAt": "2021-09-24T10:47:54.790Z",
    "creator": {
      "_id": "614dacd052fd31818950e463",
      "profilePicture": "https://res.cloudinary.com/dbplcha6k/image/upload/v1632480534/gk9vbleo0nioprpx3mm3.jpg",
      "name": "Hoang Vo",
      "username": "hoangvvo"
    }
  }
]

Ideally, however, we cannot load every single post in one request. Therefore, we need to implement pagination for the above Get Posts API.

The upcoming is a simple pagination implementation using "createdAt" as a "cursor" along with a limit argument. Since the posts fetched are always sorted in the newest-first order, we can find the next "pages" by querying documents whose createdAt is before that of the last fetched post.

// page 1: Fetch 10 posts no filter
[
 { "content": "First", "createdAt": 2021-09-24T06:00:00.000+00:00 },
 /* ... */
 { "content": "Tenth", "createdAt": 2021-09-24T01:00:00.000+00:00 }, // the cursor
]

// page 2: Fetch 10 posts, where `createdAt` < 2021-09-24T01:00:00.000+00:00
[
 { "content": "Eleventh", "createdAt": 2021-09-23T23:50:00.000+00:00 },
 /* ... */
]

Let's update our pipeline to handle it:

handler.get(async (req, res) => {
  const posts = req.db
    .collection("posts")
    .aggregate([
      {
        $match: {
          ...(req.query.before && {
            createdAt: { $lt: new Date(req.query.before) },
          }),
        },
      },
      { $sort: { _id: -1 } },
      { $limit: limit || 10 },
      {
        $lookup: {
          from: "users",
          localField: "creatorId",
          foreignField: "_id",
          as: "creator",
        },
      },
      { $unwind: "$creator" },
      {
        $project: {
          "creator.password": 0,
          "creator.email": 0,
          "creator.emailVerified": 0,
          "creator.bio": 0,
        },
      },
    ])
    .toArray();

  res.json({ posts });
});

We use a $match aggregation to select documents whose createdAt is less than the before query value if provided. For this to work req.query.before should be either a number or a string representation of the Date (which can come from Date.toJSON).

We also use $limit to limit the number of documents. Make sure $limit is placed after $sort because we need to sort the documents before taking the first numbers of them (otherwise, we may end up with incorrect sorting since it only sorts among the limited posts).

Get Posts UI

Post component

We can create a single Post component like below:

import { format } from "@lukeed/ms";
import { useMemo } from "react";

export const Post = ({ post }) => {
  const timestampTxt = useMemo(() => {
    // note: post.createdAt might be of type string sometimes
    // as shown in a later section
    const diff = Date.now() - new Date(post.createdAt).getTime();
    if (diff < 1 * 60 * 1000) return "Just now";
    return `${format(diff, true)} ago`;
  }, [post.createdAt]);

  return (
    <div>
      <Link href={`/user/${post.creator.username}`}>
        <div style={{ display: flex }}>
          <img src={post.creator.profilePicture} alt={post.creator.username} />
          <div>
            <p>{post.creator.name}</p>
            <p>{post.creator.username}</p>
          </div>
        </div>
      </Link>
      <p>{post.content}</p>
      <time dateTime={String(post.createdAt)} className={styles.timestamp}>
        {timestampTxt}
      </time>
    </div>
  );
};

I used the @lukeed/ms library to get the typical "9 hours ago" text. The Next.js Link component allows users to navigate the creator's profile upon clicking on their info.

To display the UI in the frontend, let's create a SWR useSWRInfinite hook:

export function usePostPages({ limit = 10 } = {}) {
  const { data, error, size, ...props } = useSWRInfinite(
    (index, previousPageData) => {
      // reached the end
      if (previousPageData && previousPageData.posts.length === 0) return null;

      const searchParams = new URLSearchParams();
      searchParams.set("limit", limit);

      if (index !== 0) {
        // using oldest posts createdAt date as cursor
        // We want to fetch posts which has a date that is
        // before (hence the .getTime()) the last post's createdAt
        const before = new Date(
          new Date(
            previousPageData.posts[previousPageData.posts.length - 1].createdAt
          ).getTime()
        );

        searchParams.set("before", before.toJSON());
      }

      return `/api/posts?${searchParams.toString()}`;
    },
    fetcher,
    {
      refreshInterval: 10000,
      revalidateAll: false,
    }
  );

  const isLoadingInitialData = !data && !error;
  const isLoadingMore =
    isLoadingInitialData ||
    (size > 0 && data && typeof data[size - 1] === "undefined");
  const isEmpty = data?.[0]?.length === 0;
  const isReachingEnd =
    isEmpty || (data && data[data.length - 1]?.posts?.length < limit);

  return {
    data,
    error,
    size,
    isLoadingMore,
    isReachingEnd,
    ...props,
  };
}

Several things going on in this hook, but the crucial part is that we try to construct our URL parameter based on our arguments:

limit parameter is set to limit.
before parameter is set to the string representation of the Date object. This is also what we expect on the server.

If this is not the first page (index !== 0), we will use the date of the oldest/last post (previousPageData.posts[previousPageData.posts.length - 1].createdAt) as our before parameter, since we want to fetch even older posts.

If the fetch returns nothing (previousPageData && previousPageData.posts.length === 0), we can guess that there are no more older posts. Be aware that we must first assert that previousPageData is not null since otherwise, the data might just not have arrived yet.

We also return some convenient variables like isLoadingInitialData, isLoadingMore, isEmpty, isReachingEnd. These are parts of the SWR example. You should try to understand their logic.

At this point, it is trivial to use the hook and the Post component to finish our Post list.

const PostList = () => {
  const { data, size, setSize, isLoadingMore, isReachingEnd } = usePostPages();
  const posts = data
    ? data.reduce((acc, val) => [...acc, ...val.posts], [])
    : [];
    return (<div>
  {
    posts.map((post) => (
      <Post key={post._id} className={styles.post} post={post} />
    ));
  }
  {
    isReachingEnd ? (
      <p>No more posts are found</p>
    ) : (
      <button disabled={isLoadingMore} onClick={() => setSize(size + 1)}>
        Load more
      </button>
    );
  }
  </div>)
};

Build the comment feature

Similarly, let's build our comment feature.

Create comment API

Let's build an API to create a post at POST /api/posts/[postId]/comments that create a comment for the post with ID postId. Create /pages/api/posts/[postId]/comments/index.js:

const handler = nc(ncOpts);

handler.use(database);

handler.post(
  ...auths,
  validateBody({
    type: "object",
    properties: {
      content: { type: "string", minLength: 1, maxLength: 280 },
    },
    required: ["content"],
    additionalProperties: false,
  }),
  async (req, res) => {
    if (!req.user) {
      return res.status(401).end();
    }

    const content = req.body.content;

    const post = await findPostById(req.db, req.query.postId);

    if (!post) {
      return res.status(404).json({ error: { message: "Post is not found." } });
    }

    const comment = {
      content,
      postId: new ObjectId(postId),
      creatorId,
      createdAt: new Date(),
    };
    const { insertedId } = await db.collection("comments").insertOne(comment);
    comment._id = insertedId;

    return res.json({ comment });
  }
);

We begin with authentication middleware and our validate body middleware to make sure the request is legit.

Before inserting the comment, we must check that the post exists by findPostById, which is simply a call of db.collection('posts').find(). If not, we return a 404.

Then, we simply insert the comment to the database, similar to the way we do with our Post API.

Create comment UI

Let's create a simple UI to submit the comment to the above API:

const Commenter = ({ post }) => {
  const contentRef = useRef();
  const [isLoading, setIsLoading] = useState(false);

  const { mutate } = useCommentPages({ postId: post._id });

  const onSubmit = useCallback(
    async (e) => {
      e.preventDefault();
      try {
        setIsLoading(true);
        await fetcher(`/api/posts/${post._id}/comments`, {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ content: contentRef.current.value }),
        });
        toast.success("You have added a comment");
        contentRef.current.value = "";
        // refresh post lists
        mutate();
      } catch (e) {
        toast.error(e.message);
      } finally {
        setIsLoading(false);
      }
    },
    [mutate, post._id]
  );

  return (
    <form onSubmit={onSubmit}>
      <input ref={contentRef} placeholder="Add your comment" />
      <button disable={isLoading}>Comment</button>
    </form>
  );
};

The UI above is simple enough, after inputting the comment, we send it to our API. This component should accept a prop so we know which post to add our comment to.

We will create the useCommentPages hook in the next part but the idea here is that we need to call mutate from it to refresh the comments after our comment submission.

Query comments API

Then, we create a paginated comments query API for a single post:

handler.get(async (req, res) => {
  const post = await findPostById(req.db, req.query.postId);

  if (!post) {
    return res.status(404).json({ error: { message: "Post is not found." } });
  }

  db.collection("comments")
    .aggregate([
      {
        $match: {
          postId: new ObjectId(req.query.postId),
          ...(req.query.before && {
            createdAt: { $lt: new Date(req.query.before) },
          }),
        },
      },
      { $sort: { _id: -1 } },
      { $limit: parseInt(req.query.limit, 10) },
      {
        $lookup: {
          from: "users",
          localField: "creatorId",
          foreignField: "_id",
          as: "creator",
        },
      },
      { $unwind: "$creator" },
      { $project: dbProjectionUsers("creator.") },
    ])
    .toArray();

  return res.json({ comments });
});

We similarly return 404 if the post in question is not found.

The aggregation code of comments is the same as that of posts. The only difference is that we also match against the postId field to select those from that post.

Comments List UI

We create the useCommentPages similar to what we do with usePostPages. The only additional argument is postId, since we only query comments for a specific post.

import { fetcher } from "@/lib/fetch";
import useSWRInfinite from "swr/infinite";

export function useCommentPages({ postId, limit = 10 } = {}) {
  const { data, error, size, ...props } = useSWRInfinite(
    (index, previousPageData) => {
      // reached the end
      if (previousPageData && previousPageData.comments.length === 0)
        return null;

      const searchParams = new URLSearchParams();
      searchParams.set("limit", limit);

      if (index !== 0) {
        const before = new Date(
          new Date(
            previousPageData.comments[
              previousPageData.comments.length - 1
            ].createdAt
          ).getTime()
        );

        searchParams.set("before", before.toJSON());
      }

      return `/api/posts/${postId}/comments?${searchParams.toString()}`;
    },
    fetcher,
    {
      refreshInterval: 10000,
      revalidateAll: false,
    }
  );

  const isLoadingInitialData = !data && !error;
  const isLoadingMore =
    isLoadingInitialData ||
    (size > 0 && data && typeof data[size - 1] === "undefined");
  const isEmpty = data?.[0]?.length === 0;
  const isReachingEnd =
    isEmpty || (data && data[data.length - 1]?.comments?.length < limit);

  return {
    data,
    error,
    size,
    isLoadingMore,
    isReachingEnd,
    ...props,
  };
}

Then, we create the comment list component. Again, this is similar to the post list component.

const CommentList = ({ post }) => {
  const { data, size, setSize, isLoadingMore, isReachingEnd } = useCommentPages(
    { postId: post._id }
  );

  const comments = data
    ? data.reduce((acc, val) => [...acc, ...val.comments], [])
    : [];

  return (
    <div>
      {comments.map((comment) => (
        <Comment key={comment._id} comment={comment} />
      ))}
      {isReachingEnd ? (
        <p>No more comments are found</p>
      ) : (
        <button disabled={isLoadingMore} onClick={() => setSize(size + 1)}>
          Load more
        </button>
      )}
    </div>
  );
};

Conclusion

It is the end of our implementation for the most important feature: Posts and Comments. Why don't we jump right into nextjs-mongodb.vercel.app/ and create some posts and comments.

I hope this has been helpful in working with your Next.js app. Again, check out the repository nextjs-mongodb-app. If you find this helpful, consider giving it a star to motivate me with further development and more content.

Good luck on your next Next.js + MongoDB project!

About nextjs-mongodb-app project