From cce07b52712ea6f3b8a9e8e5f4b072c4278bd023 Mon Sep 17 00:00:00 2001 From: Ben Beltran Date: Mon, 1 Jun 2020 13:28:28 +0200 Subject: Add the spec for this feature --- doc/specs/20200601-serving-different-versions.md | 198 +++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 doc/specs/20200601-serving-different-versions.md (limited to 'doc') diff --git a/doc/specs/20200601-serving-different-versions.md b/doc/specs/20200601-serving-different-versions.md new file mode 100644 index 0000000..351cc8c --- /dev/null +++ b/doc/specs/20200601-serving-different-versions.md @@ -0,0 +1,198 @@ +# Problem + +Feed reader users should be able to subscribe to the blog + +# Background + +As of this writing, the blog is served as HTML which is appropriate for +web browsers but maybe not for other mechanisms like feed readers. + +Feed readers have different formats that they support: + * h-feed is a microformat built into the page + * rss and atom are XML based publishing formats + * JSON feed is a JSON based publishing format + * rss 3.0 is a text based pblishing format :P + +Currently the blog contains a single generator function that copies +assets and generates HTML out of markdown files. This is good enough for +the current setup, but if it were to generate more it would get messy +real quick. + +Given the constraints listed below, some formats are not recommended: + * RSS 3.0 is not a good candidate at the moment as it would require + us to parse the markdown to extract the title. + * Atom would work, however, given the requirement for an id, title, and + date this would require more effort than a more lenient format. + * RSS 2.0 fits the constraints as we wouldn't need to specify anything + for the item. + * JSON Feed would work, however given the requirement for an id, thtis + would require more effort than a more lenient format. + +It is unclear whether the current constraints are good enough for feed +readers. If this causes issues, it's likely we will have to include date, +id or title as required in the other formats. + +After reviewing the functionality of existing readers, it has been found +that an id and publication date would be needed for readers to behave +correctly. This means that ATOM and JSON Feed would be equally valid +as solutions than RSS 2.0 + +The current generator function depends on knowing a source for the post +being generated, and a target on where the assets will be placed. + +# Hypothesis + +Given we serve the blog in a feed reader friendly format, users will be able to subscribe. + +# Test of Hypothesis + +Given I add the blog to a feed reader service like Reeder or Feedly, I will be able to see the entries. +Given I add a new entry to the blog, the entries will be updated. + +# Assumptions + +* We can generate a valid feed with just the entries themselves and the existing + blog data. + * We can: Validated by generating an example file. +* Including just a list of items with the whole content is good enough for + feed readers. + * We can't: It seems like we'll require at least a guid. The old reader + behaves correctly with just the guid. It's unclear whether feedly + does since it has caching. Will leave grok running. +* It isn't required to link back, and we can include the whole text. + * This is correct, however it might make sense to just link to the + blog itself. + +# Constraints + +* We won't be parsing the markdown to generate feed items. +* We won't be adding any sort of frontmatter to the entries. +* The blog will remain ephemeral, and we won't introduce permalinks. +* We won't have configurable templating or options to add/remove + output types. + +# Solution Proposal + +(Note: This solution is incomplete. We need to add guid and pubDate +generation, as this is required for the RSS to be usable) + +We will split the current generator function into generators, and create +a new generator that will generate an RSS 2.0 file + +# Blackbox + +``` + ┌─────────────────┐ ┌───────────────┐ + │ │ │ │ + ┌─────▶│ StaticGenerator │──────────▶│ Static Assets │ + │ │ │ │ │ + │ └─────────────────┘ └───────────────┘ +┌───────┐ │ ┌───────────────┐ ┌───────────┐ +│ │ │ │ │ │ │ +│ Blog │──────┼─────▶│ HTMLGenerator │────────────▶│ HTML File │ +│ │ │ │ │ │ │ +└───────┘ │ └───────────────┘ └───────────┘ + │ ┌──────────────┐ ┌──────────┐ + │ │ │ │ │ + └─────▶│ RSSGenerator │─────────────▶│ RSS File │ + │ │ │ │ + └──────────────┘ └──────────┘ +``` + +# Theory of Operation + +When the generate function of the blog is triggered, it will iterate over +a list of generator functions and call them with the source and target +directories, and an array containing the parsed markdown from the post. +Each generator function will do its work, throwing an exception if they +encounter an error. + +When the static generator is called, it will remove the current assets +directory in the target directory, and recursively copy the assets from +the source directory. + +When the HTML generator is called, it will parse an `html` template, using +the posts as the context, and will place the resulting file in the target +directory. + +When the RSS generator is called, it will parse an `rss` template, using +the posts as the context, and will place the resulting file in the target +directory. + +# Technical Specification + +## The Generator Interface + +Every generator must implement this interface in order to work with +Blog. + +* Generators MUST be a function +* Generators SHOULD read the source, destination, and posts parameters to + write files. +* Generators MUST NOT write anything into the source directory +* Generators MUST return a promise +* Generators SHOULD NOT resolve the promise with any information, as it will + be discarded +* Generators MUST throw exceptions if they encounter an unrecoverable error + +``` +IGenerator(source, destination, posts>) => Promise +``` + +## New Generators + +### Static Generator + +This generator will have the logic to move static assets around. It will +re-use the current asset logic in the `#_generate` method in Blog. + +``` +StaticGenerator +``` + +### HTML Generator + +This generator will have the logic to generate an HTML file. It will +re-use the current HTML logic in the `#_generate` method in Blog. + +``` +HTMLGenerator +``` + +### RSS Generator + +This generator will have the logic to generate an RSS file. It will +re-use the current HTML logic in the `#_generate` method in Blog, +however, instead of using the `index.html` template it will use a +`feed.xml` template that generates a valid RSS 2.0 feed document. + +``` +RSSGenerator +``` + +## Modifications to existing parts of the code + +The `#_generate` function will be modified so it will now parse the +post markdown, and then iterate over the generators, calling them +so they create the appropriatet files. + +## Important Metrics + +Given we're only processing 3 blog posts, and this is a compile time +activity and not runtime, there are no recommended metrics in terms +of file throughput performance or runtime performance. + +This should change if this would ever handle a higher volume, or would +be expected to run this process runtime. + +## Furhter Improvements + +It's recommended to eventually put more effort in assigning a unique ID +to each post so we can use more feed formats. + +For more compatibility and future proofing, the same solution for +RSS could be used to generate other feed formats, just adding +a new generator + +This same solution could be extended to serve the blog in different formats +(eg. a .txt that is easy to read in terminals) -- cgit From ae442b8f192ab0f9d4de483462d5d18c43d9ec23 Mon Sep 17 00:00:00 2001 From: Ben Beltran Date: Mon, 1 Jun 2020 18:53:59 +0200 Subject: Update spec witth the metadata step --- doc/specs/20200601-serving-different-versions.md | 82 +++++++++++++++++------- 1 file changed, 60 insertions(+), 22 deletions(-) (limited to 'doc') diff --git a/doc/specs/20200601-serving-different-versions.md b/doc/specs/20200601-serving-different-versions.md index 351cc8c..545febb 100644 --- a/doc/specs/20200601-serving-different-versions.md +++ b/doc/specs/20200601-serving-different-versions.md @@ -73,8 +73,9 @@ Given I add a new entry to the blog, the entries will be updated. # Solution Proposal -(Note: This solution is incomplete. We need to add guid and pubDate -generation, as this is required for the RSS to be usable) +We will add a new step in the creation process to create metadata for the +post that will allow each post to be uniquely identified, as well as +having a publish date related to them. We will split the current generator function into generators, and create a new generator that will generate an RSS 2.0 file @@ -82,30 +83,56 @@ a new generator that will generate an RSS 2.0 file # Blackbox ``` - ┌─────────────────┐ ┌───────────────┐ - │ │ │ │ - ┌─────▶│ StaticGenerator │──────────▶│ Static Assets │ - │ │ │ │ │ - │ └─────────────────┘ └───────────────┘ -┌───────┐ │ ┌───────────────┐ ┌───────────┐ -│ │ │ │ │ │ │ -│ Blog │──────┼─────▶│ HTMLGenerator │────────────▶│ HTML File │ -│ │ │ │ │ │ │ -└───────┘ │ └───────────────┘ └───────────┘ - │ ┌──────────────┐ ┌──────────┐ - │ │ │ │ │ - └─────▶│ RSSGenerator │─────────────▶│ RSS File │ - │ │ │ │ - └──────────────┘ └──────────┘ + ╔══════════════════════╗ + ║ When Adding a Post ║ + ╚══════════════════════╝ + ┌───────────────┐ ┌───────────────┐ + │ │ │ │ + ┌────────────────▶│ writeMetadata │─────────▶│ Metadata File │ + │ │ │ │ │ + │ └───────────────┘ └───────────────┘ + │ + │ + │ ╔════════════════════════╗ + │ ║ When Generating Output ║ + │ ╚════════════════════════╝ + │ ┌─────────────────┐ ┌───────────────┐ + │ │ │ │ │ + │ ┌─────▶│ StaticGenerator │───────▶│ Static Assets │ + │ │ │ │ │ │ + │ │ └─────────────────┘ └───────────────┘ +┌───────┐ │ ┌───────────────┐ ┌───────────┐ +│ │ │ │ │ │ │ +│ Blog │──────┼─────▶│ HTMLGenerator │─────────▶│ HTML File │ +│ │ │ │ │ │ │ +└───────┘ │ └───────────────┘ └───────────┘ + │ ┌──────────────┐ ┌──────────┐ + │ │ │ │ │ + └─────▶│ RSSGenerator │──────────▶│ RSS File │ + │ │ │ │ + └──────────────┘ └──────────┘ ``` # Theory of Operation -When the generate function of the blog is triggered, it will iterate over -a list of generator functions and call them with the source and target -directories, and an array containing the parsed markdown from the post. -Each generator function will do its work, throwing an exception if they -encounter an error. +## When Adding a Post + +When the add function of the blog is triggered, it will shift the posts +as it currently does and then will generate a new UUID and take the +current timestamp. This will be saved in a JSON file in the output +directory called "metadata.json" + +## When Generating Output + +When the generate function of the blog is triggered, it will iterate +over every post. For each of them it will parse the markdown content, +and the metadata, creating an object of type `tPost` and pushing it +to an array. + +Next, it will iterate from a list of generator functions and call them +with the source and target directories, and an array containing the `tPost` +objects. Each generator function will do its work, throwing an exception +if they encounter an error. When the static generator is called, it will remove the current assets directory in the target directory, and recursively copy the assets from @@ -121,6 +148,17 @@ directory. # Technical Specification +## The Post Data Structure + +This spec introduces a data structure to help generate output. + +``` +tPost + +html // The markup of the post + +publishedOn // The timestamp when this post was added + +uuid // The UUID for this post +``` + ## The Generator Interface Every generator must implement this interface in order to work with -- cgit From d2dd1134bc8065f001ebb3feff54a309098f861a Mon Sep 17 00:00:00 2001 From: Ben Beltran Date: Mon, 1 Jun 2020 22:43:53 +0200 Subject: Clarify type of id used --- doc/specs/20200601-serving-different-versions.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/specs/20200601-serving-different-versions.md b/doc/specs/20200601-serving-different-versions.md index 545febb..9eb1bd8 100644 --- a/doc/specs/20200601-serving-different-versions.md +++ b/doc/specs/20200601-serving-different-versions.md @@ -118,7 +118,7 @@ a new generator that will generate an RSS 2.0 file ## When Adding a Post When the add function of the blog is triggered, it will shift the posts -as it currently does and then will generate a new UUID and take the +as it currently does and then will generate a new unique ID and take the current timestamp. This will be saved in a JSON file in the output directory called "metadata.json" @@ -156,9 +156,14 @@ This spec introduces a data structure to help generate output. tPost +html // The markup of the post +publishedOn // The timestamp when this post was added - +uuid // The UUID for this post + +id // The Unique ID for this post ``` +Given that posts won't come in at a high enough rate, and that the +purpouse is only to help feed readers identify each unique piece of +content, for this version the `id` will be the same number as the +`publishedOn`. + ## The Generator Interface Every generator must implement this interface in order to work with -- cgit