From: Ben Beltran <redacted>
Date: Tue, 2 Jun 2020 19:34:29 +0000 (+0200)
Subject: Merge branch 'feature/rbdr-rss-support' into develop
X-Git-Tag: 5.0.0~28
X-Git-Url: https://git.r.bdr.sh/rbdr/blog/commitdiff_plain/db7b464d701c7d48777ddd38a10f69093cd5442c?hp=863ccf104e7cae08268b9e46896b82a9918a22fd

Merge branch 'feature/rbdr-rss-support' into develop
---

diff --git a/.gitignore b/.gitignore
index 997ace3..790ecb1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,5 +64,4 @@ typings/
 .posts
 
 # Generated files
-static/assets
-static/index.html
+static
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a643cc0..1e3b1f8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## Unreleased
+### Added
+- Post Metadata support
+- RSS Support
+
+### Changed
+- Updated dependencies
+- Replace jsdoc with jsdoc-to-markdown
+- Replace showdown with marked
+- Replace mustache with dot
+
 ## [1.0.1] - 2017-07-03
 ### Added
 - Add styling for code
diff --git a/doc/specs/20200601-serving-different-versions.md b/doc/specs/20200601-serving-different-versions.md
new file mode 100644
index 0000000..9eb1bd8
--- /dev/null
+++ b/doc/specs/20200601-serving-different-versions.md
@@ -0,0 +1,241 @@
+# Problem
+
+Feed reader users should be able to subscribe to the blog
+
+# Background
+
+As of this writing, the blog is served as HTML which is appropriate for
+web browsers but maybe not for other mechanisms like feed readers.
+
+Feed readers have different formats that they support:
+  * h-feed is a microformat built into the page
+  * rss and atom are XML based publishing formats
+  * JSON feed is a JSON based publishing format
+  * rss 3.0 is a text based pblishing format :P
+
+Currently the blog contains a single generator function that copies
+assets and generates HTML out of markdown files. This is good enough for
+the current setup, but if it were to generate more it would get messy
+real quick.
+
+Given the constraints listed below, some formats are not recommended:
+  * RSS 3.0 is not a good candidate at the moment as it would require
+    us to parse the markdown to extract the title.
+  * Atom would work, however, given the requirement for an id, title, and
+    date this would require more effort than a more lenient format.
+  * RSS 2.0 fits the constraints as we wouldn't need to specify anything
+    for the item.
+  * JSON Feed would work, however given the requirement for an id, thtis
+    would require more effort than a more lenient format.
+
+It is unclear whether the current constraints are good enough for feed
+readers. If this causes issues, it's likely we will have to include date,
+id or title as required in the other formats.
+
+After reviewing the functionality of existing readers, it has been found
+that an id and publication date would be needed for readers to behave
+correctly. This means that ATOM and JSON Feed would be equally valid
+as solutions than RSS 2.0
+
+The current generator function depends on knowing a source for the post
+being generated, and a target on where the assets will be placed.
+
+# Hypothesis
+
+Given we serve the blog in a feed reader friendly format, users will be able to subscribe.
+
+# Test of Hypothesis
+
+Given I add the blog to a feed reader service like Reeder or Feedly, I will be able to see the entries.
+Given I add a new entry to the blog, the entries will be updated.
+
+# Assumptions
+
+* We can generate a valid feed with just the entries themselves and the existing
+  blog data.
+  * We can: Validated by generating an example file.
+* Including just a list of items with the whole content is good enough for
+  feed readers.
+  * We can't: It seems like we'll require at least a guid. The old reader
+    behaves correctly with just the guid. It's unclear whether feedly
+    does since it has caching. Will leave grok running.
+* It isn't required to link back, and we can include the whole text.
+  * This is correct, however it might make sense to just link to the
+    blog itself.
+
+# Constraints
+
+* We won't be parsing the markdown to generate feed items.
+* We won't be adding any sort of frontmatter to the entries.
+* The blog will remain ephemeral, and we won't introduce permalinks.
+* We won't have configurable templating or options to add/remove
+  output types.
+
+# Solution Proposal
+
+We will add a new step in the creation process to create metadata for the
+post that will allow each post to be uniquely identified, as well as
+having a publish date related to them.
+
+We will split the current generator function into generators, and create
+a new generator that will generate an RSS 2.0 file
+
+# Blackbox
+
+```
+                   ╔══════════════════════╗
+                   ║  When Adding a Post  ║
+                   ╚══════════════════════╝
+                      ┌───────────────┐          ┌───────────────┐
+                      │               │          │               │
+    ┌────────────────▶│ writeMetadata │─────────▶│ Metadata File │
+    │                 │               │          │               │
+    │                 └───────────────┘          └───────────────┘
+    │
+    │
+    │             ╔════════════════════════╗
+    │             ║ When Generating Output ║
+    │             ╚════════════════════════╝
+    │                 ┌─────────────────┐        ┌───────────────┐
+    │                 │                 │        │               │
+    │          ┌─────▶│ StaticGenerator │───────▶│ Static Assets │
+    │          │      │                 │        │               │
+    │          │      └─────────────────┘        └───────────────┘
+┌───────┐      │      ┌───────────────┐          ┌───────────┐
+│       │      │      │               │          │           │
+│ Blog  │──────┼─────▶│ HTMLGenerator │─────────▶│ HTML File │
+│       │      │      │               │          │           │
+└───────┘      │      └───────────────┘          └───────────┘
+               │      ┌──────────────┐           ┌──────────┐
+               │      │              │           │          │
+               └─────▶│ RSSGenerator │──────────▶│ RSS File │
+                      │              │           │          │
+                      └──────────────┘           └──────────┘
+```
+
+# Theory of Operation
+
+## When Adding a Post
+
+When the add function of the blog is triggered, it will shift the posts
+as it currently does and then will generate a new unique ID and take the
+current timestamp. This will be saved in a JSON file in the output
+directory called "metadata.json"
+
+## When Generating Output
+
+When the generate function of the blog is triggered, it will iterate
+over every post. For each of them it will parse the markdown content,
+and the metadata, creating an object of type `tPost` and pushing it
+to an array.
+
+Next, it will iterate from a list of generator functions and call them
+with the source and target directories, and an array containing the `tPost`
+objects. Each generator function will do its work, throwing an exception
+if they encounter an error.
+
+When the static generator is called, it will remove the current assets
+directory in the target directory, and recursively copy the assets from
+the source directory.
+
+When the HTML generator is called, it will parse an `html` template, using
+the posts as the context, and will place the resulting file in the target
+directory.
+
+When the RSS generator is called, it will parse an `rss` template, using
+the posts as the context, and will place the resulting file in the target
+directory.
+
+# Technical Specification
+
+## The Post Data Structure
+
+This spec introduces a data structure to help generate output.
+
+```
+tPost <Object>
+  +html <String> // The markup of the post
+  +publishedOn <Number> // The timestamp when this post was added
+  +id <String> // The Unique ID for this post
+```
+
+Given that posts won't come in at a high enough rate, and that the
+purpouse is only to help feed readers identify each unique piece of
+content, for this version the `id` will be the same number as the
+`publishedOn`.
+
+## The Generator Interface
+
+Every generator must implement this interface in order to work with
+Blog.
+
+* Generators MUST be a function
+* Generators SHOULD read the source, destination, and posts parameters to
+  write files.
+* Generators MUST NOT write anything into the source directory
+* Generators MUST return a promise
+* Generators SHOULD NOT resolve the promise with any information, as it will
+  be discarded
+* Generators MUST throw exceptions if they encounter an unrecoverable error
+
+```
+IGenerator(source<String>, destination<String>, posts<Array<String>>) => Promise
+```
+
+## New Generators
+
+### Static Generator
+
+This generator will have the logic to move static assets around. It will
+re-use the current asset logic in the `#_generate` method in Blog.
+
+```
+StaticGenerator <IGenerator>
+```
+
+### HTML Generator
+
+This generator will have the logic to generate an HTML file. It will
+re-use the current HTML logic in the `#_generate` method in Blog.
+
+```
+HTMLGenerator <IGenerator>
+```
+
+### RSS Generator
+
+This generator will have the logic to generate an RSS file. It will
+re-use the current HTML logic in the `#_generate` method in Blog,
+however, instead of using the `index.html` template it will use a
+`feed.xml` template that generates a valid RSS 2.0 feed document.
+
+```
+RSSGenerator <IGenerator>
+```
+
+## Modifications to existing parts of the code
+
+The `#_generate` function will be modified so it will now parse the
+post markdown, and then iterate over the generators, calling them 
+so they create the appropriatet files.
+
+## Important Metrics
+
+Given we're only processing 3 blog posts, and this is a compile time
+activity and not runtime, there are no recommended metrics in terms
+of file throughput performance or runtime performance.
+
+This should change if this would ever handle a higher volume, or would
+be expected to run this process runtime.
+
+## Furhter Improvements
+
+It's recommended to eventually put more effort in assigning a unique ID
+to each post so we can use more feed formats.
+
+For more compatibility and future proofing, the same solution for
+RSS could be used to generate other feed formats, just adding
+a new generator
+
+This same solution could be extended to serve the blog in different formats
+(eg. a .txt that is easy to read in terminals)
diff --git a/lib/blog.js b/lib/blog.js
index 9024a00..98efabd 100644
--- a/lib/blog.js
+++ b/lib/blog.js
@@ -1,28 +1,27 @@
 'use strict';
 
-const { exec } = require('child_process');
 const { access, mkdir, readdir, readFile, rmdir, writeFile } = require('fs/promises');
-const { template } = require('dot');
 const { ncp } = require('ncp');
 const { join } = require('path');
 const Marked = require('marked');
 const { debuglog, promisify } = require('util');
 
+const StaticGenerator = require('./generators/static');
+const HTMLGenerator = require('./generators/html');
+const RSSGenerator = require('./generators/rss');
+
 const internals = {
 
   // Promisified functions
   ncp: promisify(ncp),
 
-  exec: promisify(exec),
   debuglog: debuglog('blog'),
 
   // constants
 
-  kAssetsDirectoryName: 'assets',
-  kIndexName: 'index.html',
   kFileNotFoundError: 'ENOENT',
   kMarkdownRe: /\.md$/i,
-  kRemoveCommand: 'rm -rf',
+  kMetadataFilename: 'metadata.json',
 
   // Strings
 
@@ -36,7 +35,7 @@ const internals = {
  * updating posts, and handling the publishing.
  *
  * @class Blog
- * @param {Potluck.tConfiguration} config the initialization options to
+ * @param {Blog.tConfiguration} config the initialization options to
  * extend the instance
  */
 module.exports = class Blog {
@@ -59,6 +58,7 @@ module.exports = class Blog {
    */
   async add(postLocation) {
 
+    await this._ensurePostsDirectoryExists();
     await this._shift();
     await this.update(postLocation);
   }
@@ -75,7 +75,11 @@ module.exports = class Blog {
    */
   async update(postLocation) {
 
+    const metadata = await this._getMetadata();
+    await this._ensurePostsDirectoryExists();
     await this._copyPost(postLocation);
+    await this._writeMetadata(metadata);
+
     await this._generate();
   }
 
@@ -97,33 +101,40 @@ module.exports = class Blog {
 
   async _generate() {
 
-    const assetsTarget = join(this.staticDirectory, internals.kAssetsDirectoryName);
-    const indexTarget = join(this.staticDirectory, internals.kIndexName);
-    const indexLocation = join(this.templatesDirectory, internals.kIndexName);
-    const posts = [];
+    internals.debuglog('Generating output');
+
+    const posts = await this._readPosts(this.postsDirectory);
+
+    await StaticGenerator(this.postsDirectory, this.staticDirectory, posts);
+    await HTMLGenerator(this.templatesDirectory, this.staticDirectory, posts);
+    await RSSGenerator(this.templatesDirectory, this.staticDirectory, posts);
+  }
 
-    internals.debuglog(`Removing ${assetsTarget}`);
-    await rmdir(assetsTarget, { recursive: true });
+  // Reads the posts into an array
+
+  async _readPosts(source) {
+
+    internals.debuglog('Reading posts');
+    const posts = [];
 
     for (let i = 0; i < this.maxPosts; ++i) {
-      const sourcePath = join(this.postsDirectory, `${i}`);
+      const postSourcePath = join(source, `${i}`);
 
-      try {
-        await access(this.postsDirectory);
+      internals.debuglog(`Reading ${postSourcePath} into posts array`);
 
-        const assetsSource = join(sourcePath, internals.kAssetsDirectoryName);
-        const postContentPath = await this._findBlogContent(sourcePath);
+      try {
+        await access(postSourcePath);
 
-        internals.debuglog(`Copying ${assetsSource} to ${assetsTarget}`);
-        await internals.ncp(assetsSource, assetsTarget);
+        const metadata = await this._getMetadata(i);
 
+        const postContentPath = await this._findBlogContent(postSourcePath);
         internals.debuglog(`Reading ${postContentPath}`);
         const postContent = await readFile(postContentPath, { encoding: 'utf8' });
 
         internals.debuglog('Parsing markdown');
         posts.push({
-          html: Marked(postContent),
-          id: i + 1
+          ...metadata,
+          html: Marked(postContent)
         });
       }
       catch (error) {
@@ -136,30 +147,24 @@ module.exports = class Blog {
       }
     }
 
-    internals.debuglog(`Reading ${indexLocation}`);
-    const indexTemplate = await readFile(indexLocation, { encoding: 'utf8' });
-
-    internals.debuglog('Generating HTML');
-    const indexHtml = template(indexTemplate)({ posts });
-    await writeFile(indexTarget, indexHtml);
+    return posts;
   }
 
   // Shift the posts, delete any remainder.
 
   async _shift() {
 
-    await this._ensurePostsDirectoryExists();
 
-    for (let i = this.maxPosts - 1; i > 0; --i) {
+    for (let i = this.maxPosts - 1; i >= 0; --i) {
       const targetPath = join(this.postsDirectory, `${i}`);
       const sourcePath = join(this.postsDirectory, `${i - 1}`);
 
       try {
-        await access(sourcePath);
-
         internals.debuglog(`Removing ${targetPath}`);
         await rmdir(targetPath, { recursive: true });
 
+        await access(sourcePath); // check the source path
+
         internals.debuglog(`Shifting blog post ${sourcePath} to ${targetPath}`);
         await internals.ncp(sourcePath, targetPath);
       }
@@ -174,12 +179,42 @@ module.exports = class Blog {
     }
   }
 
+  // Attempts to read existing metadata. Otherwise generates new set.
+
+  async _getMetadata(index = 0) {
+
+    const metadataTarget = join(this.postsDirectory, String(index), internals.kMetadataFilename);
+
+    try {
+      internals.debuglog(`Looking for metadata at ${metadataTarget}`);
+      return JSON.parse(await readFile(metadataTarget, { encoding: 'utf8' }));
+    }
+    catch (e) {
+      internals.debuglog(`Metadata not found or unreadable. Generating new set.`);
+      const createdOn = Date.now();
+      const metadata = {
+        id: String(createdOn),
+        createdOn
+      };
+
+      return metadata;
+    }
+  }
+
+  // Writes metadata. Assumes post 0 since it only gets written
+  // on create
+
+  async _writeMetadata(metadata) {
+
+    const metadataTarget = join(this.postsDirectory, '0', internals.kMetadataFilename);
+    internals.debuglog(`Writing ${metadataTarget}`);
+    await writeFile(metadataTarget, JSON.stringify(metadata, null, 2));
+  }
+
   // Copies a post directory to the latest slot.
 
   async _copyPost(postLocation) {
 
-    await this._ensurePostsDirectoryExists();
-
     const targetPath = join(this.postsDirectory, '0');
 
     internals.debuglog(`Removing ${targetPath}`);
diff --git a/lib/generators/html.js b/lib/generators/html.js
new file mode 100644
index 0000000..ba2676c
--- /dev/null
+++ b/lib/generators/html.js
@@ -0,0 +1,35 @@
+'use strict';
+
+const { template } = require('dot');
+const { readFile, writeFile } = require('fs/promises');
+const { join } = require('path');
+const { debuglog } = require('util');
+
+const internals = {
+  debuglog: debuglog('blog'),
+
+  kIndexName: 'index.html'
+};
+
+/**
+ * Generates the blog index page
+ *
+ * @name HTMLGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function HTMLGenerator(source, target, posts) {
+
+  internals.debuglog('Generating HTML');
+  const indexTarget = join(target, internals.kIndexName);
+  const indexLocation = join(source, internals.kIndexName);
+
+  internals.debuglog(`Reading ${indexLocation}`);
+  const indexTemplate = await readFile(indexLocation, { encoding: 'utf8' });
+
+  internals.debuglog('Writing HTML');
+  const indexHtml = template(indexTemplate)({ posts });
+  await writeFile(indexTarget, indexHtml);
+};
+
diff --git a/lib/generators/rss.js b/lib/generators/rss.js
new file mode 100644
index 0000000..a1fcb08
--- /dev/null
+++ b/lib/generators/rss.js
@@ -0,0 +1,42 @@
+'use strict';
+
+const { template } = require('dot');
+const { encodeXML } = require('entities');
+const { readFile, writeFile } = require('fs/promises');
+const { join } = require('path');
+const { debuglog } = require('util');
+
+const internals = {
+  debuglog: debuglog('blog'),
+
+  kFeedName: 'feed.xml'
+};
+
+/**
+ * Generates an RSS feed XML file
+ *
+ * @name RSSGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function RSSGenerator(source, target, posts) {
+
+  internals.debuglog('Generating RSS');
+  const feedTarget = join(target, internals.kFeedName);
+  const feedLocation = join(source, internals.kFeedName);
+
+  internals.debuglog(`Reading ${feedLocation}`);
+  const feedTemplate = await readFile(feedLocation, { encoding: 'utf8' });
+
+  internals.debuglog('Writing RSS');
+  posts = posts.map((post) => ({
+    ...post,
+    createdOn: (new Date(post.createdOn)).toUTCString(),
+    html: encodeXML(post.html)
+  }));
+  const feedXml = template(feedTemplate)({ posts });
+  await writeFile(feedTarget, feedXml);
+};
+
+
diff --git a/lib/generators/static.js b/lib/generators/static.js
new file mode 100644
index 0000000..eb3c631
--- /dev/null
+++ b/lib/generators/static.js
@@ -0,0 +1,50 @@
+'use strict';
+
+const { access, rmdir } = require('fs/promises');
+const { ncp } = require('ncp');
+const { join } = require('path');
+const { debuglog, promisify } = require('util');
+
+const internals = {
+  ncp: promisify(ncp),
+  debuglog: debuglog('blog'),
+
+  kAssetsDirectoryName: 'assets'
+};
+
+/**
+ * Generates the static assets required for the blog
+ *
+ * @name StaticGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function StaticGenerator(source, target, posts) {
+
+  const assetsTarget = join(target, internals.kAssetsDirectoryName);
+
+  internals.debuglog(`Removing ${assetsTarget}`);
+  await rmdir(assetsTarget, { recursive: true });
+
+  for (let i = 0; i < posts.length; ++i) {
+    const postSourcePath = join(source, `${i}`);
+
+    try {
+      await access(postSourcePath);
+
+      const assetsSource = join(postSourcePath, internals.kAssetsDirectoryName);
+
+      internals.debuglog(`Copying ${assetsSource} to ${assetsTarget}`);
+      await internals.ncp(assetsSource, assetsTarget);
+    }
+    catch (error) {
+      if (error.code === internals.kFileNotFoundError) {
+        internals.debuglog(`Skipping ${i}`);
+        continue;
+      }
+
+      throw error;
+    }
+  }
+};
diff --git a/package-lock.json b/package-lock.json
index 939bce7..eca5389 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -536,8 +536,7 @@
     "entities": {
       "version": "2.0.2",
       "resolved": "https://registry.npmjs.org/entities/-/entities-2.0.2.tgz",
-      "integrity": "sha512-dmD3AvJQBUjKpcNkoqr+x+IF0SdRtPz9Vk0uTy4yWqga9ibB6s4v++QFWNohjiUGoMlF552ZvNyXDxz5iW0qmw==",
-      "dev": true
+      "integrity": "sha512-dmD3AvJQBUjKpcNkoqr+x+IF0SdRtPz9Vk0uTy4yWqga9ibB6s4v++QFWNohjiUGoMlF552ZvNyXDxz5iW0qmw=="
     },
     "escape-string-regexp": {
       "version": "1.0.5",
diff --git a/package.json b/package.json
index d127963..f2ed098 100644
--- a/package.json
+++ b/package.json
@@ -22,11 +22,12 @@
   },
   "homepage": "https://gitlab.com/rbdr/blog#readme",
   "dependencies": {
+    "dot": "^1.1.3",
+    "entities": "^2.0.2",
     "getenv": "^1.0.0",
+    "marked": "^1.0.0",
     "minimist": "^1.2.5",
-    "dot": "^1.1.3",
-    "ncp": "^2.0.0",
-    "marked": "^1.0.0"
+    "ncp": "^2.0.0"
   },
   "devDependencies": {
     "eslint": "^7.1.0",
diff --git a/templates/feed.xml b/templates/feed.xml
new file mode 100644
index 0000000..d22d656
--- /dev/null
+++ b/templates/feed.xml
@@ -0,0 +1,18 @@
+<?xml version="1.0"?>
+<rss version="2.0" xmlns:blogChannel="http://backend.userland.com/blogChannelModule">
+  <channel>
+    <title>Blog at Unlimited Pizza 🍕</title>
+    <link>https://blog.unlimited.pizza</link>
+    <description>This is the blog at unlimited.pizza</description>
+    <language>en</language>
+    {{~ it.posts: post}}
+      <item>
+        <guid isPermaLink="false">unlimited-pizza:{{= post.id}}</guid>
+        <pubDate>{{= post.createdOn}}</pubDate>
+        <description>
+          {{= post.html}}
+        </description>
+      </item>
+    {{~}}
+  </channel>
+</rss>
diff --git a/templates/index.html b/templates/index.html
index 0c45ce2..3e37304 100644
--- a/templates/index.html
+++ b/templates/index.html
@@ -1,28 +1,33 @@
 <!doctype html>
-<html>
+<html lang="en" class="h-feed">
   <head>
     <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <meta class="p-author" name="author" content="Rubén Beltrán del Río">
     <meta name="description" content="This is the blog at unlimited.pizza">
 
-    <title>blog 🍕</title>
+    <meta name="theme-color" content="#ffffff">
+
+    <title class="p-name">Blog at Unlimited Pizza 🍕</title>
 
     <script src="/js/blog.js"></script>
 
-    <link href="css/style.css" rel="stylesheet">
+    <link href="/css/style.css" rel="stylesheet">
+    <link href="/feed.xml" rel="alternate" hreflang="en" title="RSS feed">
 
   </head>
   <body>
-    <header class="main-header">
+    <header aria-label="Logo" class="main-header">
       <a href="/">Blog</a>
     </header>
     <main>
     {{~ it.posts: post}}
-      <article id="{{= post.id}}">
+      <article class="h-entry" id="{{= post.id}}">
         {{= post.html}}
       </article>
       <hr>
     {{~}}
-    {{? it.posts}}
+    {{? it.posts.length === 0}}
       <h1>This is a fresh blog!</h1>
       <p>There are no posts yet.</p>
     {{?}}