Tutorial - Concatenate and convert multiple markdown files to a single HTML

May 31, 2020

#Nodejs #Markdown #Tutorial

Today we are going to create a NodeJs app that reads multiple markdown files from a directory, then it will convert their content to HTML and put the result into a single HTML file.

// Our project structure

├── in
│   ├── test0.md
│   ├── test1.md
│   ├── test2.md
│   └── test3.md
├── out
│   └── index.html
├── index.js
├── package.json

The in folder will contain our input files. ( e.g. test0 test1… )
The out folder will contain our index.html that will be regenerated every time we run our application in order to add new contents.
All of our code will be in index.js ( we only need a few lines to get the job done 😉)

Prerequisites

Goal

Suppose you have a web page with a lot of contents and you want to regenerate this page every time someone adds some contents.
Maybe who is writing the contents is not comfortable with HTML, tags and so on. He/She wants to focus only on writing great contents.
The deal: The copywriter will use Markdown (it has very few rules and no programming knowledge is needed ) and the web master will only run this application to integrate the newly created articles/post/contents. Sounds great, isn’t it ?

An output example :

Hello from test 1

Can you read this ?

Hello from test 2

Some bold text here

Hello from test 3

You can learn how simple is markdown here

Hello from test 5

Non id nostrud commodo dolore consequat occaecat id tempor excepteur. Reprehenderit ad nulla aliqua anim labore do elit. Dolor ex ea incididunt ut nisi ipsum consectetur do adipisicing occaecat mollit labore aute. Et anim sint aute veniam dolore ad adipisicing officia reprehenderit ut. Ut ex magna tempor esse culpa pariatur nisi ea do. Est in ad Lorem labore. Occaecat ullamco exercitation ullamco culpa magna duis occaecat non sit cupidatat. Occaecat nisi ea magna enim mollit non anim Lorem reprehenderit velit sit sint ex ipsum. Eiusmod ex id tempor elit eu ut ut sunt exercitation quis cillum magna qui consectetur. Occaecat cillum labore laboris sunt est adipisicing voluptate elit. Quis nisi ex dolore commodo est ut et duis enim. Aliquip nisi anim consectetur pariatur ea. Incididunt cillum fugiat exercitation amet cupidatat aliquip. Veniam ea tempor esse fugiat cillum. Eu eu anim magna laboris sit laboris sit aliqua commodo dolore veniam eu labore eiusmod. Officia enim aute non consectetur sit veniam dolore culpa.

Let’s start !

Create a new directory and open it in Visual Studio Code. Open the terminal and type

npm init

Follow the installer and then a package.json file will be created. Now we can install a package that will help us to translate markdown to HTML:

npm install showdown

Showdown is a bidirectional converter, but in this tutorial we are going to translate only from Markdown to HTML. ( Showdown official website )

Create the in and the out folder, then add some markdown file inside the in folder.

Edit the package.json and add a start script.

"scripts": {
    "start": "node index.js"
}

Create a index.js file and import our dependencies:

const fs = require("fs").promises; //The fs module provides an API for interacting with the file system
const showdown = require("showdown");
const path = require("path"); // The path module provides utilities for working with file and directory paths

The fs module and the path module are standard NodeJs modules, so no installation is needed.
PLEASE NOTE: In this tutorial will be used fs.promises that gives you the possibility to handle promises instead of callbacks . As a Javascript developer i find this way more comfortable.

We can define the paths of the input and output folders :

const inDirectory = path.join(__dirname, "in");
const outDirectory = path.join(__dirname, "out");

Create a function to read all the markdown files inside the input folder:

async function readMarkdownFiles() {

  const files = await fs.readdir(inDirectory, "utf8"); // it returns the names of all the files

  let promises = [];

  // fs.readFile() returns a promise, we can collect all in an array
  files.forEach( (fileName) => {
    let filePath = path.join(inDirectory, fileName);
    promises.push(fs.readFile(filePath, "utf8"));
  });

  // return a promise that will be resolved when all the promises had done theirs job
  return Promise.all(promises);
}

Please feel free to add a try catch block here. I’ve omitted many error checks for brevity.

We can define two templates :

  • The start of the HTML file that we want to generate .
const htmlTemplateStart = `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>NodeJs and Markdown</title>
    <style>
    * {
        font-family: monospace;
    }
    body {
        margin: auto;
        max-width: 42rem;
        padding: 2rem;
    }
    h1 {
        margin-bottom: 0.8rem;
    }
    p {
        margin-bottom: 5rem;
        line-height: 1.6rem;
    }
</style>
</head>
<body>`
  • The end of the HTML file that we want to generate .
const htmlTemplateEnd = `</body>
</html>`;

Out translated markdown will be in the middle, inside the body .

In order to use this templates and the reading function we can create an anonymous function and immediately invoke it. ( Check my IIFE article)

( async () => {
    const files = await readMarkdownFiles(); // get the array of all files contents
    const content = files.join("\n"); // merge all files content in one string
    const converter = new showdown.Converter(); // instantiate the converter
    const convertedMarkdown = converter.makeHtml(content); // do the magic
    let outputFile = path.join(outDirectory, "index.html");

    fs.writeFile(
        outputFile,
        htmlTemplateStart + convertedMarkdown + htmlTemplateEnd );
})();

The final code here:

// index.js

const fs = require("fs").promises;
const showdown = require("showdown");
const path = require("path");

const inDirectory = path.join(__dirname, "in");
const outDirectory = path.join(__dirname, "out");

const htmlTemplateStart = `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>NodeJs and Markdown</title>
    <style>
    * {
        font-family: monospace;
    }

    body {
        margin: auto;
        max-width: 42rem;
        padding: 2rem;
    }

    h1 {
        margin-bottom: 0.8rem;
    }

    p {
        margin-bottom: 5rem;
        line-height: 1.6rem;
    }
</style>
</head>
<body>`;

const htmlTemplateEnd = `</body>
</html>`;

async function readMarkdownFiles() {
  const files = await fs.readdir(inDirectory, "utf8");
  let promises = [];

  files.forEach((fileName) => {
    let filePath = path.join(inDirectory, fileName);
    promises.push(fs.readFile(filePath, "utf8"));
  });

  return Promise.all(promises);
}

(async () => {
  const files = await readMarkdownFiles();
  const content = files.join("\n");
  const converter = new showdown.Converter();
  const convertedMarkdown = converter.makeHtml(content);
  let outputFile = path.join(outDirectory, "index.html");

  fs.writeFile(
    outputFile,
    htmlTemplateStart + convertedMarkdown + htmlTemplateEnd
  );
})();

Now we’re ready to try it ! Put some md files inside the in folder.

e.g.

// in/test0.md
# Hello from test 1

Can you read _this_ ?

( The underscore symbol will generate Italic Style. )

Run

npm start

Check the out folder. Open the index.html file inside you browser. Magic !
You can add a new markdown file, run the application and refresh the browser. The content is updated with the newly created file ! 🤩🤩🤩

Hope you enjoy this tutorial, Feel free to share the code and create your own NodeJs application

Pingu 🐧


Written by Pingu who lives and works in Genoa building useful modern applications.

Something to share ?