Codemonth is over, and while I didn't get as much done as I'd hoped, I did make some real progress... and while stuffing more software into the EEEpc and configuring it has been a major distraction, it just now gave something back - turns out that there is a useful outline editor in debian after all, called treeline (found during a mis-aimed apt-cache search for "python doc".) It's an outline editor that can make a fair guess at arbitrary XML files; it loaded meterstone's one sample file immediately - and was able to "export" it back (as "Generic" XML) to a file that meterstone could read (treeline's save format is somewhat different, though.)
A related search also turned up tcm, the "Toolkit for Conceptual Modeling", which looks like it's trying to cover what would be the overall task breakdown part problem in more detail, without actually looking at estimation at all. It's also a very retro-looking lesstif app, which apparently hasn't been touched since 2003, so I'm not sure it will actually help, but it seemed worth noting.
Turns out that pos_name is updated in list's draw method, but that gets run before any input is processed; adding a gratuitous run(-1) after the run(0) call works around the bug, and lets me move forward, and lets me declare the title-bar feature done. (Of course, it hadn't actually made it on to the example schedule yet, so I can't cross it off either :-)
More hackathon time: figured out how to track named objects too, and implemented the title-bar "where in the tree am I" feature. Also learned that xml.etree doesn't have any tools for figuring out where an element is within a tree, but it's simple to track that "on the way down" as it were. However, it turns out that updates seem to happen "one too late" - not sure yet if it's the set call that isn't rendering or the cursor not updating in time...
Figured out how to track a cursor within a list (key to being able to insert and expand and such...) Baby steps, and more useful than redoing the syntax as python objects :-)
Found that hitting M-f or other high-bit chars confused some state machine, possibly within ncurses, hanging the app... but the STFL mailing list archives pointed out that hitting BACKSPACE (really DEL) gets STFL to report "TIMEOUT" (and then "BACKSPACE"), so I joined the list and asked about the problem (since the lists seemed to imply that it had been fixed.) It was nice to see that there's both recent activity, and the Debian maintainer, on the list.
Stumbled across python-stfl, a python/swig binding for STFL, a "Structured Terminal Forms Language/Library". Syntax, and features, are pretty trivial, but unlike urwid the model for changing the interface on the fly is pretty usable, possibly because it doesn't expose so much of the power of the underlying curses, and also that text is mostly just text. I suspect it's more likely that I'll find STFL too simple but when the interface grows to that extent it'll be time to move it anyway...
It helps that STFL's "microlanguage" for form definition is indented text, so it "smells right" among python code (though it isn't really that close a match.)
It took about 20 minutes to get a viewer up that would parse and display the basic task titles from XML format in tree form. Manipulation comes next...
Did some more writing over dinner, will transcribe later; in the meantime, I've got a sample task file for meterstone's own schedule starting to grow. I started playing with using OPML directly, but it was somewhat contorting because most of the data model would be in extension elements and attributes, and I don't have any linux-based OPML editors handy in Ubuntu (only a few things that import OPML "subscription lists" into RSS readers) so it wouldn't actually save me any effort, I'd have to write at least part of an outliner to start with anyway. This suggests adding OPML and XOXO output stages later, with a secondary goal of doing round-trips between those formats... but round-trips with emacs are more important, and will have a much greater impact on productivity from the beginning.
I threw together an XML schema, which is stunningly verbose when used raw (it's currently larger than the sample schedule itself :-) and clearly should have been generated automatically, though I don't see much in terms of tools for that either (maybe they're over in javaspace - which has never played well with linux, or unix really, so they're less likely to be packaged - and less likely to do me any good.)
Turns out that I should have used RELAX NG instead, that's an actual "direct" syntax... still pretty verbose, but at least it's shorter than the worked example. English is still reasonable at this point, though:
Values of leafnode range elements get summed up the chain to the top; that's why model is a list, because swags and times can't actually be combined; further experimentation may conclude that fuzzy to precise is continuous which makes combining even messier (see prior reference to Evidence Based Scheduling), but we'll start simple.
Values of task.spent just get combined up the chain; implicitly they can only change in leafnode tasks, and only increase; recording which range of that node's model the spent time comes from remains an exercise.
At this point there's enough of a concrete representation that the next steps are to implement some basic operations on it:
These were hand-added to the sample schedule, but should be enough that further estimation gets done with the tool directly :-)
It's already the second week in November, and I've yet to pick a CodeMonth project.
My experience using HiveMinder over the last few months has both given me insights into listmaking tools (and how to work off of them - in particular, the importance of GTD-style "review" for eliminating clutter and opportunities for dithering) and brought into clearer relief the additional things beyond listmaking that are needed for planning projects. Throw in some recent experience with an over-designed but under-estimated project, and being hit on the head with Steve McConnell's latest estimation book, and I have some ideas I want to explore... in code.
(If I get anywhere with this, I'll probably get all evangelistic and try to get people to put this stuff into other things, rather than maintaining the prototype tools myself. You Are Forewarned.)
First concept: every estimate breaks down into the part you actually know how to do, and the part you don't know how to do. Software is "special" in that we're always "inventing" stuff and spend a lot of time doing things in novel ways, so a lot of our estimates have large "haven't done that before" parts. (After all, if you had done that before, you'd reuse that code, and it wouldn't contribute that much to the overall time taken...) Therefore a big part of refining an estimate is to move more things in to the "known" part, by doing experiments or actually completing high-risk parts.
(This aspect led to an attempt to use the "turtles all the way down" theme for a project name, but too many things already used "turtle feet" as a product name, and even "nanoturtle" gets 120 hits. That brought to mind the "milestone/inchstone" construction, but inchstone has connotations of picayunity; thus, meterstone, as more "modern" and sensibly larger, and has had very little use. I will also note that neither planckstone or angstromstone appear on google at this time.)
This attack-by-parts approach sometimes has secondary benefits - when the "shape" of the unknown part is constrained by the known parts, drilling down on the known parts can make the unknown remainder more tractable or at least less daunting. (In extreme cases, it can vanish entirely.) Listmaking tools are sufficient for taking those parts you've identified and getting you to apply forward momentum to them... but they don't directly help turning higher level projects into achievable pieces, especially if you only identify the pieces as-you-go.
Second concept: there are existing file formats and tools for manipulating structures like this - outliners, or tree editors, for example, all support the basic "split item" and "expand/contract view" pieces, and although none of them handle the associated calculations, they may still serve as useful initial shortcuts. Experience with other tools in this space shows that
are all key features of the file store for any such format.
Third concept: it's possible to bootstrap this by having it represent itself ("it's dogfood all the way down") though it will likely help to have a more constrained tracer-bullet project "on the side". Given that execution is a key element of refinement, meterstone has to actually record progress, not just record predictions; this also helps build up evidence, or at least examples, for concrete reference in future projects (instead of the traditional "personal memory" approach.) Since gathering that information is a precursor to recording it, it starts to look like at least some "lab notebook" features should creep in ("what am I doing now, how long have I been doing it, wow this bit really did take rather a while...") but that is potentially "too big" for this round.
Final concept: this needs to be "painted" on to a calendar somehow, to relate "work time" to "real time". This leads to other "outside event" connections - things where you're not spending "work time", just waiting for "real time" to pass (batch test cycle running... shipment of prototypes arriving in the mail...) but still have an impact on what you can do when, and when you'll be finished.
I've now got a name, 700 words of design-overview, a rough bit of intial test data, and half a dozen footnotes. At this stage I have no idea at all of how long this will take - I might be able to say "I'm going to cut my losses at the end of a month", but that is not anything to do with an estimate - it's a constraint that changes what is being estimated, at most.
More generally - now I have a direction in which to apply forward momentum :-)