.posts
# Generated files
-static/assets
-static/index.html
+static
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
+## Unreleased
+### Added
+- Post Metadata support
+- RSS Support
+
+### Changed
+- Updated dependencies
+- Replace jsdoc with jsdoc-to-markdown
+- Replace showdown with marked
+- Replace mustache with dot
+
## [1.0.1] - 2017-07-03
### Added
- Add styling for code
--- /dev/null
+# Problem
+
+Feed reader users should be able to subscribe to the blog
+
+# Background
+
+As of this writing, the blog is served as HTML which is appropriate for
+web browsers but maybe not for other mechanisms like feed readers.
+
+Feed readers have different formats that they support:
+ * h-feed is a microformat built into the page
+ * rss and atom are XML based publishing formats
+ * JSON feed is a JSON based publishing format
+ * rss 3.0 is a text based pblishing format :P
+
+Currently the blog contains a single generator function that copies
+assets and generates HTML out of markdown files. This is good enough for
+the current setup, but if it were to generate more it would get messy
+real quick.
+
+Given the constraints listed below, some formats are not recommended:
+ * RSS 3.0 is not a good candidate at the moment as it would require
+ us to parse the markdown to extract the title.
+ * Atom would work, however, given the requirement for an id, title, and
+ date this would require more effort than a more lenient format.
+ * RSS 2.0 fits the constraints as we wouldn't need to specify anything
+ for the item.
+ * JSON Feed would work, however given the requirement for an id, thtis
+ would require more effort than a more lenient format.
+
+It is unclear whether the current constraints are good enough for feed
+readers. If this causes issues, it's likely we will have to include date,
+id or title as required in the other formats.
+
+After reviewing the functionality of existing readers, it has been found
+that an id and publication date would be needed for readers to behave
+correctly. This means that ATOM and JSON Feed would be equally valid
+as solutions than RSS 2.0
+
+The current generator function depends on knowing a source for the post
+being generated, and a target on where the assets will be placed.
+
+# Hypothesis
+
+Given we serve the blog in a feed reader friendly format, users will be able to subscribe.
+
+# Test of Hypothesis
+
+Given I add the blog to a feed reader service like Reeder or Feedly, I will be able to see the entries.
+Given I add a new entry to the blog, the entries will be updated.
+
+# Assumptions
+
+* We can generate a valid feed with just the entries themselves and the existing
+ blog data.
+ * We can: Validated by generating an example file.
+* Including just a list of items with the whole content is good enough for
+ feed readers.
+ * We can't: It seems like we'll require at least a guid. The old reader
+ behaves correctly with just the guid. It's unclear whether feedly
+ does since it has caching. Will leave grok running.
+* It isn't required to link back, and we can include the whole text.
+ * This is correct, however it might make sense to just link to the
+ blog itself.
+
+# Constraints
+
+* We won't be parsing the markdown to generate feed items.
+* We won't be adding any sort of frontmatter to the entries.
+* The blog will remain ephemeral, and we won't introduce permalinks.
+* We won't have configurable templating or options to add/remove
+ output types.
+
+# Solution Proposal
+
+We will add a new step in the creation process to create metadata for the
+post that will allow each post to be uniquely identified, as well as
+having a publish date related to them.
+
+We will split the current generator function into generators, and create
+a new generator that will generate an RSS 2.0 file
+
+# Blackbox
+
+```
+ ╔══════════════════════╗
+ ║ When Adding a Post ║
+ ╚══════════════════════╝
+ ┌───────────────┐ ┌───────────────┐
+ │ │ │ │
+ ┌────────────────▶│ writeMetadata │─────────▶│ Metadata File │
+ │ │ │ │ │
+ │ └───────────────┘ └───────────────┘
+ │
+ │
+ │ ╔════════════════════════╗
+ │ ║ When Generating Output ║
+ │ ╚════════════════════════╝
+ │ ┌─────────────────┐ ┌───────────────┐
+ │ │ │ │ │
+ │ ┌─────▶│ StaticGenerator │───────▶│ Static Assets │
+ │ │ │ │ │ │
+ │ │ └─────────────────┘ └───────────────┘
+┌───────┐ │ ┌───────────────┐ ┌───────────┐
+│ │ │ │ │ │ │
+│ Blog │──────┼─────▶│ HTMLGenerator │─────────▶│ HTML File │
+│ │ │ │ │ │ │
+└───────┘ │ └───────────────┘ └───────────┘
+ │ ┌──────────────┐ ┌──────────┐
+ │ │ │ │ │
+ └─────▶│ RSSGenerator │──────────▶│ RSS File │
+ │ │ │ │
+ └──────────────┘ └──────────┘
+```
+
+# Theory of Operation
+
+## When Adding a Post
+
+When the add function of the blog is triggered, it will shift the posts
+as it currently does and then will generate a new unique ID and take the
+current timestamp. This will be saved in a JSON file in the output
+directory called "metadata.json"
+
+## When Generating Output
+
+When the generate function of the blog is triggered, it will iterate
+over every post. For each of them it will parse the markdown content,
+and the metadata, creating an object of type `tPost` and pushing it
+to an array.
+
+Next, it will iterate from a list of generator functions and call them
+with the source and target directories, and an array containing the `tPost`
+objects. Each generator function will do its work, throwing an exception
+if they encounter an error.
+
+When the static generator is called, it will remove the current assets
+directory in the target directory, and recursively copy the assets from
+the source directory.
+
+When the HTML generator is called, it will parse an `html` template, using
+the posts as the context, and will place the resulting file in the target
+directory.
+
+When the RSS generator is called, it will parse an `rss` template, using
+the posts as the context, and will place the resulting file in the target
+directory.
+
+# Technical Specification
+
+## The Post Data Structure
+
+This spec introduces a data structure to help generate output.
+
+```
+tPost <Object>
+ +html <String> // The markup of the post
+ +publishedOn <Number> // The timestamp when this post was added
+ +id <String> // The Unique ID for this post
+```
+
+Given that posts won't come in at a high enough rate, and that the
+purpouse is only to help feed readers identify each unique piece of
+content, for this version the `id` will be the same number as the
+`publishedOn`.
+
+## The Generator Interface
+
+Every generator must implement this interface in order to work with
+Blog.
+
+* Generators MUST be a function
+* Generators SHOULD read the source, destination, and posts parameters to
+ write files.
+* Generators MUST NOT write anything into the source directory
+* Generators MUST return a promise
+* Generators SHOULD NOT resolve the promise with any information, as it will
+ be discarded
+* Generators MUST throw exceptions if they encounter an unrecoverable error
+
+```
+IGenerator(source<String>, destination<String>, posts<Array<String>>) => Promise
+```
+
+## New Generators
+
+### Static Generator
+
+This generator will have the logic to move static assets around. It will
+re-use the current asset logic in the `#_generate` method in Blog.
+
+```
+StaticGenerator <IGenerator>
+```
+
+### HTML Generator
+
+This generator will have the logic to generate an HTML file. It will
+re-use the current HTML logic in the `#_generate` method in Blog.
+
+```
+HTMLGenerator <IGenerator>
+```
+
+### RSS Generator
+
+This generator will have the logic to generate an RSS file. It will
+re-use the current HTML logic in the `#_generate` method in Blog,
+however, instead of using the `index.html` template it will use a
+`feed.xml` template that generates a valid RSS 2.0 feed document.
+
+```
+RSSGenerator <IGenerator>
+```
+
+## Modifications to existing parts of the code
+
+The `#_generate` function will be modified so it will now parse the
+post markdown, and then iterate over the generators, calling them
+so they create the appropriatet files.
+
+## Important Metrics
+
+Given we're only processing 3 blog posts, and this is a compile time
+activity and not runtime, there are no recommended metrics in terms
+of file throughput performance or runtime performance.
+
+This should change if this would ever handle a higher volume, or would
+be expected to run this process runtime.
+
+## Furhter Improvements
+
+It's recommended to eventually put more effort in assigning a unique ID
+to each post so we can use more feed formats.
+
+For more compatibility and future proofing, the same solution for
+RSS could be used to generate other feed formats, just adding
+a new generator
+
+This same solution could be extended to serve the blog in different formats
+(eg. a .txt that is easy to read in terminals)
'use strict';
-const { exec } = require('child_process');
const { access, mkdir, readdir, readFile, rmdir, writeFile } = require('fs/promises');
-const { template } = require('dot');
const { ncp } = require('ncp');
const { join } = require('path');
const Marked = require('marked');
const { debuglog, promisify } = require('util');
+const StaticGenerator = require('./generators/static');
+const HTMLGenerator = require('./generators/html');
+const RSSGenerator = require('./generators/rss');
+
const internals = {
// Promisified functions
ncp: promisify(ncp),
- exec: promisify(exec),
debuglog: debuglog('blog'),
// constants
- kAssetsDirectoryName: 'assets',
- kIndexName: 'index.html',
kFileNotFoundError: 'ENOENT',
kMarkdownRe: /\.md$/i,
- kRemoveCommand: 'rm -rf',
+ kMetadataFilename: 'metadata.json',
// Strings
* updating posts, and handling the publishing.
*
* @class Blog
- * @param {Potluck.tConfiguration} config the initialization options to
+ * @param {Blog.tConfiguration} config the initialization options to
* extend the instance
*/
module.exports = class Blog {
*/
async add(postLocation) {
+ await this._ensurePostsDirectoryExists();
await this._shift();
await this.update(postLocation);
}
*/
async update(postLocation) {
+ const metadata = await this._getMetadata();
+ await this._ensurePostsDirectoryExists();
await this._copyPost(postLocation);
+ await this._writeMetadata(metadata);
+
await this._generate();
}
async _generate() {
- const assetsTarget = join(this.staticDirectory, internals.kAssetsDirectoryName);
- const indexTarget = join(this.staticDirectory, internals.kIndexName);
- const indexLocation = join(this.templatesDirectory, internals.kIndexName);
- const posts = [];
+ internals.debuglog('Generating output');
+
+ const posts = await this._readPosts(this.postsDirectory);
+
+ await StaticGenerator(this.postsDirectory, this.staticDirectory, posts);
+ await HTMLGenerator(this.templatesDirectory, this.staticDirectory, posts);
+ await RSSGenerator(this.templatesDirectory, this.staticDirectory, posts);
+ }
- internals.debuglog(`Removing ${assetsTarget}`);
- await rmdir(assetsTarget, { recursive: true });
+ // Reads the posts into an array
+
+ async _readPosts(source) {
+
+ internals.debuglog('Reading posts');
+ const posts = [];
for (let i = 0; i < this.maxPosts; ++i) {
- const sourcePath = join(this.postsDirectory, `${i}`);
+ const postSourcePath = join(source, `${i}`);
- try {
- await access(this.postsDirectory);
+ internals.debuglog(`Reading ${postSourcePath} into posts array`);
- const assetsSource = join(sourcePath, internals.kAssetsDirectoryName);
- const postContentPath = await this._findBlogContent(sourcePath);
+ try {
+ await access(postSourcePath);
- internals.debuglog(`Copying ${assetsSource} to ${assetsTarget}`);
- await internals.ncp(assetsSource, assetsTarget);
+ const metadata = await this._getMetadata(i);
+ const postContentPath = await this._findBlogContent(postSourcePath);
internals.debuglog(`Reading ${postContentPath}`);
const postContent = await readFile(postContentPath, { encoding: 'utf8' });
internals.debuglog('Parsing markdown');
posts.push({
- html: Marked(postContent),
- id: i + 1
+ ...metadata,
+ html: Marked(postContent)
});
}
catch (error) {
}
}
- internals.debuglog(`Reading ${indexLocation}`);
- const indexTemplate = await readFile(indexLocation, { encoding: 'utf8' });
-
- internals.debuglog('Generating HTML');
- const indexHtml = template(indexTemplate)({ posts });
- await writeFile(indexTarget, indexHtml);
+ return posts;
}
// Shift the posts, delete any remainder.
async _shift() {
- await this._ensurePostsDirectoryExists();
- for (let i = this.maxPosts - 1; i > 0; --i) {
+ for (let i = this.maxPosts - 1; i >= 0; --i) {
const targetPath = join(this.postsDirectory, `${i}`);
const sourcePath = join(this.postsDirectory, `${i - 1}`);
try {
- await access(sourcePath);
-
internals.debuglog(`Removing ${targetPath}`);
await rmdir(targetPath, { recursive: true });
+ await access(sourcePath); // check the source path
+
internals.debuglog(`Shifting blog post ${sourcePath} to ${targetPath}`);
await internals.ncp(sourcePath, targetPath);
}
}
}
+ // Attempts to read existing metadata. Otherwise generates new set.
+
+ async _getMetadata(index = 0) {
+
+ const metadataTarget = join(this.postsDirectory, String(index), internals.kMetadataFilename);
+
+ try {
+ internals.debuglog(`Looking for metadata at ${metadataTarget}`);
+ return JSON.parse(await readFile(metadataTarget, { encoding: 'utf8' }));
+ }
+ catch (e) {
+ internals.debuglog(`Metadata not found or unreadable. Generating new set.`);
+ const createdOn = Date.now();
+ const metadata = {
+ id: String(createdOn),
+ createdOn
+ };
+
+ return metadata;
+ }
+ }
+
+ // Writes metadata. Assumes post 0 since it only gets written
+ // on create
+
+ async _writeMetadata(metadata) {
+
+ const metadataTarget = join(this.postsDirectory, '0', internals.kMetadataFilename);
+ internals.debuglog(`Writing ${metadataTarget}`);
+ await writeFile(metadataTarget, JSON.stringify(metadata, null, 2));
+ }
+
// Copies a post directory to the latest slot.
async _copyPost(postLocation) {
- await this._ensurePostsDirectoryExists();
-
const targetPath = join(this.postsDirectory, '0');
internals.debuglog(`Removing ${targetPath}`);
--- /dev/null
+'use strict';
+
+const { template } = require('dot');
+const { readFile, writeFile } = require('fs/promises');
+const { join } = require('path');
+const { debuglog } = require('util');
+
+const internals = {
+ debuglog: debuglog('blog'),
+
+ kIndexName: 'index.html'
+};
+
+/**
+ * Generates the blog index page
+ *
+ * @name HTMLGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function HTMLGenerator(source, target, posts) {
+
+ internals.debuglog('Generating HTML');
+ const indexTarget = join(target, internals.kIndexName);
+ const indexLocation = join(source, internals.kIndexName);
+
+ internals.debuglog(`Reading ${indexLocation}`);
+ const indexTemplate = await readFile(indexLocation, { encoding: 'utf8' });
+
+ internals.debuglog('Writing HTML');
+ const indexHtml = template(indexTemplate)({ posts });
+ await writeFile(indexTarget, indexHtml);
+};
+
--- /dev/null
+'use strict';
+
+const { template } = require('dot');
+const { encodeXML } = require('entities');
+const { readFile, writeFile } = require('fs/promises');
+const { join } = require('path');
+const { debuglog } = require('util');
+
+const internals = {
+ debuglog: debuglog('blog'),
+
+ kFeedName: 'feed.xml'
+};
+
+/**
+ * Generates an RSS feed XML file
+ *
+ * @name RSSGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function RSSGenerator(source, target, posts) {
+
+ internals.debuglog('Generating RSS');
+ const feedTarget = join(target, internals.kFeedName);
+ const feedLocation = join(source, internals.kFeedName);
+
+ internals.debuglog(`Reading ${feedLocation}`);
+ const feedTemplate = await readFile(feedLocation, { encoding: 'utf8' });
+
+ internals.debuglog('Writing RSS');
+ posts = posts.map((post) => ({
+ ...post,
+ createdOn: (new Date(post.createdOn)).toUTCString(),
+ html: encodeXML(post.html)
+ }));
+ const feedXml = template(feedTemplate)({ posts });
+ await writeFile(feedTarget, feedXml);
+};
+
+
--- /dev/null
+'use strict';
+
+const { access, rmdir } = require('fs/promises');
+const { ncp } = require('ncp');
+const { join } = require('path');
+const { debuglog, promisify } = require('util');
+
+const internals = {
+ ncp: promisify(ncp),
+ debuglog: debuglog('blog'),
+
+ kAssetsDirectoryName: 'assets'
+};
+
+/**
+ * Generates the static assets required for the blog
+ *
+ * @name StaticGenerator
+ * @param {string} source the source directory
+ * @param {string} target the target directory
+ * @param {Array.<Blog.tPost>} posts the list of posts
+ */
+module.exports = async function StaticGenerator(source, target, posts) {
+
+ const assetsTarget = join(target, internals.kAssetsDirectoryName);
+
+ internals.debuglog(`Removing ${assetsTarget}`);
+ await rmdir(assetsTarget, { recursive: true });
+
+ for (let i = 0; i < posts.length; ++i) {
+ const postSourcePath = join(source, `${i}`);
+
+ try {
+ await access(postSourcePath);
+
+ const assetsSource = join(postSourcePath, internals.kAssetsDirectoryName);
+
+ internals.debuglog(`Copying ${assetsSource} to ${assetsTarget}`);
+ await internals.ncp(assetsSource, assetsTarget);
+ }
+ catch (error) {
+ if (error.code === internals.kFileNotFoundError) {
+ internals.debuglog(`Skipping ${i}`);
+ continue;
+ }
+
+ throw error;
+ }
+ }
+};
"entities": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/entities/-/entities-2.0.2.tgz",
- "integrity": "sha512-dmD3AvJQBUjKpcNkoqr+x+IF0SdRtPz9Vk0uTy4yWqga9ibB6s4v++QFWNohjiUGoMlF552ZvNyXDxz5iW0qmw==",
- "dev": true
+ "integrity": "sha512-dmD3AvJQBUjKpcNkoqr+x+IF0SdRtPz9Vk0uTy4yWqga9ibB6s4v++QFWNohjiUGoMlF552ZvNyXDxz5iW0qmw=="
},
"escape-string-regexp": {
"version": "1.0.5",
},
"homepage": "https://gitlab.com/rbdr/blog#readme",
"dependencies": {
+ "dot": "^1.1.3",
+ "entities": "^2.0.2",
"getenv": "^1.0.0",
+ "marked": "^1.0.0",
"minimist": "^1.2.5",
- "dot": "^1.1.3",
- "ncp": "^2.0.0",
- "marked": "^1.0.0"
+ "ncp": "^2.0.0"
},
"devDependencies": {
"eslint": "^7.1.0",
--- /dev/null
+<?xml version="1.0"?>
+<rss version="2.0" xmlns:blogChannel="http://backend.userland.com/blogChannelModule">
+ <channel>
+ <title>Blog at Unlimited Pizza 🍕</title>
+ <link>https://blog.unlimited.pizza</link>
+ <description>This is the blog at unlimited.pizza</description>
+ <language>en</language>
+ {{~ it.posts: post}}
+ <item>
+ <guid isPermaLink="false">unlimited-pizza:{{= post.id}}</guid>
+ <pubDate>{{= post.createdOn}}</pubDate>
+ <description>
+ {{= post.html}}
+ </description>
+ </item>
+ {{~}}
+ </channel>
+</rss>
<!doctype html>
-<html>
+<html lang="en" class="h-feed">
<head>
<meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta class="p-author" name="author" content="Rubén Beltrán del Río">
<meta name="description" content="This is the blog at unlimited.pizza">
- <title>blog 🍕</title>
+ <meta name="theme-color" content="#ffffff">
+
+ <title class="p-name">Blog at Unlimited Pizza 🍕</title>
<script src="/js/blog.js"></script>
- <link href="css/style.css" rel="stylesheet">
+ <link href="/css/style.css" rel="stylesheet">
+ <link href="/feed.xml" rel="alternate" hreflang="en" title="RSS feed">
</head>
<body>
- <header class="main-header">
+ <header aria-label="Logo" class="main-header">
<a href="/">Blog</a>
</header>
<main>
{{~ it.posts: post}}
- <article id="{{= post.id}}">
+ <article class="h-entry" id="{{= post.id}}">
{{= post.html}}
</article>
<hr>
{{~}}
- {{? it.posts}}
+ {{? it.posts.length === 0}}
<h1>This is a fresh blog!</h1>
<p>There are no posts yet.</p>
{{?}}
git.r.bdr.sh / rbdr / blog / commitdiff