A loose collection of photographic workflow tools

This started out as a directory to unpack some EXIF processing code into, and grew into several generations of album-building tools; the current version is centered around kphotoalbum for management and flickr for publication. Some code is documented here.

exif: Tue Dec 2 22:22:00 2014

Tue Dec 2 22:22:00 2014

Just noticed an actual "stuttering" repeat upload caused by the flickr/flickd processing mishandling duplicated file (one md5sum, 3 "file" names and it would re-upload the last one but tag the first one as flickd...) After fixing the immediate problem by hand (deleting the duplicates on Flickr itself, then moving the flickd tags around with emacs) I tried to measure the scope of the problem:

import kimdaba_album
album = kimdaba_album.parse(kimdaba_album.kimdaba_default_album())
km = {}
for img in album.findall("images/image"):
  mm = img.get("md5sum")
  ff = img.get("file")
  km[mm] = km.get(mm, []) + [ff]
print len([k for k,v in km.items() if len(v) > 1])

Turns out there are 289 distinct images that each have multiple paths - one of which is the less-surprising 18 zero-length files, primarily from old cellphones, one is the 3-way that caused the problem, and the rest are duplicates of various kinds. Not much of a mess given that there are 131446 entries total, but it still caused a visible problem and needs fixing.

exif: Wed Mar 10 22:06:00 2010

Wed Mar 10 22:06:00 2010

I wrote about this a bit elsewhere, while it was in progress, but since I've used it for half a dozen pictures now - auto_cropr.py (which rather needs a new name) is working, for me at least. All it does is look on flickr at flickr.photos.recentlyUpdated photos within the last day, and tries to find a specific cropr_me tag and specific note-text (same.) Then it takes the rectangle of the note, makes a crop of the Original image that matches it, and uploads that as a new flickr image. It then links the note to the new image, cleans up the tags, updates the metadata on the new image...

There are a few holes right now - it doesn't copy permissions or safety (all of my stuff is public, so the defaults are right), it doesn't put new EXIF data in the cropped image (and it looks like libexiv2 supports working with in-memory images, but python-pyexiv2 doesn't yet - and I get the Original image directly from flickr, and never put it on disk...) nor does it handle rotation.

More relevant to having other people use it is that I've never worked my way through the parts of the flickr api that would let other people use the app. I'm using them properly for my own account, at least... if I do go that direction I should seriously consider using a flickr library that someone else maintains, too :-)

exif: Tue Oct 13 02:26:00 2009

Tue Oct 13 02:26:00 2009

kphotoalbum added a geolocation interface, with geoLat, geoLon, geoAlt (altitude) and geoPrec (precision, currently -1 anyway.) It was trivial to start picking these up and feeding them to flickr.photos.geo.setLocation calls after the uploads (just as with rotation, in async mode you have to wait for the photo ids to come back and then apply the changes.) Yay! Now I have end-to-end geotagging...

On top of that, I finally discovered what was wrong with flickr.photos.getWithGeoData - I was attempting to use page and per_page to work through smaller batches, and failing (the fields in the response were updated but the content wasn't.) Turns out that it simply doesn't work with XMLRPC - but passing the exact same arguments to the REST interface works just fine! Based on looking at the raw payloads I'm reasonably well convinced that it isn't my code or the python xmlrpclib itself... but flickr's API "community support" is a mix of unattended flickr groups and unusable yahoo lists, so now that I have a workaround (and a weak explanation/theory, namely, "nobody uses XMLRPC when there's REST available"), there isn't much more I can do. (It's too bad, as I've also caught a few spelling errors in the API docs that I'd be happy to see fixed...) So the next step is to use that interface and stuff older locations back into kphotoalbum; apparently flickr truncates to 6 digits, so kphotoalbum will remain the preferred source, but it will at least give me enough of a starting data set to start thinking about my own tools.

exif: Sun Jun 22 03:25:00 2008

Sun Jun 22 03:25:00 2008

flickr added video, with no changes to the upload API - and yet, I kept getting upload failures, in particular "Upload failed: Filesize was zero". Turns out that this was probably due to flickr misdetecting my video upload as an image upload (and a "reasonable" 60M video exceeds the "giant panorama" 20M JPEG threshold.)

Of course, the reason they misdetected it was some old hardcoding in my mimeencode function - flickr used to fail outright if the filename wasn't included as an extra tag in the Content-Disposition, so I stuffed in a default of something.jpg (which has worked for years... but of course is finally no longer true.)

Haven't determined yet if it is only the filename= that matters, or the additional Content-type qualifier change as well, but both are now basically correct...

The upstream uploadr client is now a xulrunner app, though, so I should given that another look (at least as an interesting example, my workflow is still "add a flickr tag in kphotoalbum and it'll get uploaded" which is far less work than any GUI could ever be :-)

exif: Mon Jan 21 00:32:00 2008

Mon Jan 21 00:32:00 2008

Even though the python-xscreensaver stuff started as an effort to produce a kimdaba/kphotoalbum screensaver, that itself is on the backburner since my kphotoalbum working set is on a memory stick in my pocket, not on either laptop, and I don't have a desktop, and I've moved all of the code to a new project directory to reduce the clutter here. Since CVS doesn't have renames at all, and I never got around to implementing redirects in webrcs, I'll just leave the old ones behind but copy the history to the new place.


exif: Mon Jan 7 01:11:00 2008

Mon Jan 7 01:11:00 2008

Another tweak: a parse_args method in xss_base.py which handles the contortions needed to handle xscreensaver's "tres retro" use of -window-id as a long argument, using optparse. (If it needs to be ugly - hide the ugly :-)

There's not much left to even do with xss_chbg.py - I don't want to bother with fades (and it's attention-getting enough as it is); I probably want to add

but as-is, I'm actually using it, which is more than I expected.

Hmmm. I just realized something.

There has been a screensaver module on my back burner for a long time...

Someone else even did a bunch of the work, though I still haven't seen the code. And I have a secondary source for realistic data. And while it needs graphics, it doesn't need fancy graphics...

exif: Fri Jan 4 00:55:00 2008

Fri Jan 4 00:55:00 2008

More progress on xss_chbg.py - enough that I'll now start using it as a replacement for chbg in general :-) Features include

This has actually been a fun little exercise (it's still under a hundred lines of code.)

exif: Thu Jan 3 00:58:00 2008

Thu Jan 3 00:58:00 2008

I discovered that drawable.py already had a put_pil_image very much like the one in Circus, so I changed xss_colorsquares.py to call that one directly; this cut the import list in half, and simplified the rest of the code as well.

Also started on xss_base.py which is a generic xscreensaver base class, and xss_chbg.py which is an instantiation that is a cheap clone of the chbg desktop cycler I use - none of the transitions or fades, just chewing through a list of files. They're both still in flux, though.

exif: Wed Jan 2 00:07:00 2008

Wed Jan 2 00:07:00 2008

Another few baby steps towards the kimdaba-screensaver (which I'm still not sure I actually have a use for) - changing the foreground of the randomly generated rectangle, and actually putting a JPG image up some of the time.

xss_colorsquares.py has both changes. (Looking for other examples of apps that use both PIL and python-xlib, google code failed me, but a straight google search found a http://www.koders.com search which had a number of useful hints, in the code to the Circus project. The important bits are

Obviously this isn't fully general, but it handles the "straight-through" case and I can add any needed details later, if any of the asserts fail.


exif: Mon Dec 31 22:25:00 2007

Mon Dec 31 22:25:00 2007

I've had "make a screensaver that uses my kimdaba archive directly" on my todo list for rather a long time. Thanks to hiveminder, it finally percolated to the top... and turned out to be easier than I thought. (Not done, but easier.)

First step: find out how xscreensaver modules are written in the first place. Found a simple, though somewhat out of date, tutorial on writing them in C. Turns out that all that really matters is "draw something on someone else's window". (The tutorial has it as the root window; the modern implementation uses $XSCREENSAVER_WINDOW passed in the environment in hex.)

As sort of a tribute to the tutorial (for getting me off the ground), I duplicated the first exercise from scratch in python. xss_simplesquares.py does the same thing as the first tutorial, with two modifications:

Most of the use of python-xlib is painfully straightforward. The one trick is generating a window object from 0x80059E. After turning it into a number with int(windowid, 16), all you have to do is display.create_resource_object('window', windowid) to create a wrapper object that works just like any window you'd create directly.

You can just add this code to your .xscreensaver file, using the gui, or just add it to the programs: entry with all of the others.

Next step: actually tie this in with the other kimdaba/kphotoalbum XML file parsing code around here, and add an image renderer; that should be enough to implement "any picture with the tag(s) given on the commandline gets displayed". The hard part will be finding the motivation; these days, my photos-in-progress working set is on a memory stick, and not actually plugged in to the laptop.

(Also, don't let the lack of recent changes to python-xlib deter you; the protocol hasn't changed in many years, after all - even the code from my late-90's TPJ article on "Xlib in Pure Perl" still works. I am of course much more interested in the python version these days :-)


exif: Thu Aug 10 08:22:00 2006

Thu Aug 10 08:22:00 2006

Found and fixed a unicode encoding bug when syncing things tagged Skögar (u'Sk\xf3gar').

exif: Thu May 25 03:13:00 2006

Thu May 25 03:13:00 2006



moinmoin one is larger, so probably newer - yeah, has Orientation values and comments logging more patches

exif: Wed Oct 20 21:26:00 2004

Wed Oct 20 21:26:00 2004

diff -x '*~' -x '*.toenail' -x '*.avi' -x '*.jpg' -wurp ~/PIX/SL300RT/downcase/100cxbox/ /afs/thok.org/user/eichin/pictures/SL300RT/100cxbox/

tests capt-to-bins changes

exif: Sun Oct 10 20:24:00 2004

Sun Oct 10 20:24:00 2004

After seeing the jotspot wiki, jesse's active-wiki idea is more inspiring...

exif: Mon Oct 4 13:42:00 2004

Mon Oct 4 13:42:00 2004



interesting image tagging usage - filter on a tag, and get a list of the tags available in that set for additional filtering.

exif: Mon Sep 13 03:34:00 2004

Mon Sep 13 03:34:00 2004

After reading http://www.teledyn.com/mt/archives/002126.html and hearing comments from jesse about Flickr "doing rss", I'm starting to see the idea - suppose each album [dynamic or not] has an rss feed of the full thing. Then an rss album-reader becomes a generic component, and can handle division into pages and such...

exif: Tue Jul 20 03:39:00 2004

Tue Jul 20 03:39:00 2004

try new fonts

implement fill_default_titles

exif: Sat Jun 5 05:14:00 2004

Sat Jun 5 05:14:00 2004

viewimage has progressed to the point of be able to

next steps: