Winter is... err... almost over

2014-02-10 02:02:13 -0500

While "ThokTober" sounded good when I started, we're well into February and Spring is only 40 days off. Not that that makes the project unsucessful per se - I understand the problem much more deeply which was certainly a goal, and there was never an external deadline, just a self-inflicted one.

The deepest bit of understanding was that attempting to shoehorn the "weight of history" (that is, the legacy thok.org content) into an existing system was misguided effort - the legacy content was some 2300 files, the new content (that could easily be adapted to whatever system I chose) was only 45 files - "And we decided that one big pile is better than two little piles, and rather than bring that one up we decided to throw ours down." - and that the optimal deployment tool for the legacy content was simply rsync. This reduced the complexity of the rest of the problem a great deal, as now it was simply a matter of recognizing files that needed markdown processing, and recognizing files of more interesting type ("blog post", "blog", "photoessay"...) and doing something with them.

Once I got past the problem of stuffing the large pile into a third party blogging system, what about the small pile? All of the static-blog tools I looked at where a little too opinionated (which is great when starting from scratch) and the static-site-generator tools were not opinionated enough. I finally concluded that in order to figure out what features I actually wanted, I'd need to sit down and implement them from scratch, and discover what aspects of "blog" features were artifacts of "what is easy to write" and what things actually mattered. (This also, admittedly, let me side-step the conclusion that ikiwiki, not nikola, was closest in behaviour to what I was looking for, even though it was a pile of perl with character set issues...)

Having concluded that the big opinion was that "text should be in markdown" pulling little things together was a lot simpler, and I could make visible progress a feature at a time. Other useful conclusions include:

• source goes in git
  • therefore a post date can easily be the git-author-date
  • if the destination is in git, that's an unrelated deployment issue; mkdir and rsync are entirely sufficient for 0.1
• python-markdown has an "arbitrary metadata" extension; for 0.1 I need to reparse the input files to extract it, but with 45 files that just isn't important, since I can clearly put in time later to be as clever as I need to be. For the first round, having Tags metadata that gets pulled out to a single tag-index.html demonstrates the concept and is a good enough implementation of article keywords; adding other metadata like "twitter summary" can follow in time, but I don't need to start with it.

As a nod to codes-well-with-others, I've built most of the recent pieces atop two new third party libraries:

• for things like "blogs" (rather than individual "posts") that are entirely metadata (title, description, content pointer), Python configobj turns out to support a variety of nicely python-esque syntax options, reasonably structured data, and isn't YAML.
• lxml.html is quite nice for mucking about with generated html; since HTML is horribly fragile but no tools report legitimate diagnostics for it, sticking to carefully constructed operations on the element tree seems like the only sane way to perform operations like "add stylesheet" or "promote the first H1 to a head title element" in arbitrary contexts.

All that said, it really does look like I could do a first deployment this week, though perhaps the start of Spring is ultimately more realistic...