Wednesday, June 3, 2009

[BLUG] Console-based podcatching and script-based dialogs

Relatively recently I migrated to a console-based podcatcher.

This is due, in part, to the current state of Amarok 2. Amarok has seen
a lot of changes in the migration to KDE 4.2. It's a complete rewrite,
and it promises to be really nice. However, right now it has zero
podcatching capabilities. (Truthfully, Banshee had better podcatching
capabilities than the Amarok 1.x line. However, that's GNOME.)

I wanted something light-weight. I wanted something with a near
transparent list of the URLs, as I expect to migrate back to Amarok when
it is ready.

There are two main contenders for this feature available in my distro's
package repository: hpodder and podget.

These are both similar to the 'bashpodder' podcatcher, except that's not
in my distro's package repository. I know it's a simple small script,
but by using tools that ship with my OS, I never need to deal with
updates. I consider it a security issue, as well as a laziness issue.

'hpodder' does all the work with a set of command-line options. The list
of podcasts are stored internally in a database. While it may work well,
it wasn't transparent enough for me. Some commands required running
first other commands to get the right values. It just seemed far too
over-complicated.

'podget' uses a plain-text 'serverlist' file to list the podcasts.

'podget' has a comment character in this file, so it is possible to
activate and deactivate podcasts. This allows me to find a podiobook
(audiobook sent by podcast), mark it as something I want to listen to,
and only activate it when I have the time to listen to it, and the space
on my laptop for it.

In Amarok 1.x I used a similar feature. There were podcasts for which I
never downloaded media, and which I didn't check for updates. They were
basically notes waiting for space and time.

This meant that 'podget' met my requirements better.

It's a nice text-based podcatcher. While I could put it in a crontab,
I'm usually tight enough on space that I prefer to run it manually.

Then I was left with no integration with Firefox, leaving me to copy
the URL by hand. That wouldn't do, so I whipped up a little script.

This leads to the second part. I wanted what I thought was a fairly
standard set to script-based dialog features for the script. It wasn't
supposed to be a fire-and-forget script. I needed things confirmed.
I needed dialogs.

I explored the following script-based dialog tools:

* X-based dialog tools:
* kdialog
* Xdialog
* zenity
* gtkdialog

* Console-based dialog tools:
* dialog
* whiptail
* tcdialog
* bash

Additionally, the script does have support for a blind hands-free mode.
This happens when there is no DISPLAY and the input is not a TTY. I call
this "auto" mode, and it can be explicitly requested. Auto mode takes
the default values and plugs it in to the serverlist with no feedback.
It doesn't activate the podcast, so the settings can be tweaked by hand
later.

I thought what I wanted out of a dialog tool was fairly
straight-forward.

kdialog is the dialog tool bundled with KDE. I used the version
included in my kdebase-bin package (which is KDE 4.2). As KDE is my
desktop of choice, it was what I started with. This impacted some of my
expectations with the other dialog tools.

My script asks questions, and expects the user to respond with yes/no
responses. It turns out, 'zenity' doesn't support this. 'zenity' only
has ok/cancel, and you can't change the button text to turn it in to
yes/no.

I could have rewritten my questions to fit with ok/cancel, except I
actually use yes/no/cancel when I ask about activating the podcast.
'Yes' activates it, 'No' adds it in an inactive state, and 'Cancel' does
not add an entry for it at all.

'zenity' just grew worse from there. I suppose if you explicitly wrote
a script planning to support zenity you could have your expectations
reasonably met with other dialog tools. If your expectations are that of
any other dialog tool, though, you'll find 'zenity' doesn't meet them.

Yes/no/cancel is problematic for a number of dialog tools. Most that
don't have an explicit button do support the feature by closing the
window explicitly.

I used kdialog's --error and --sorry message dialogs to generate message
boxes with an appropriate icon. This was frequently not available in
other options, but could be worked around by prefixing "Error:" or
"Sorry:" to the dialog message.

Most script-based dialog tools rely upon the command line to set things
up. Even when there is a config file feature, it is rarely the primary
mechanism used to configure the tool. The exception to this is 'gtkdialog',
which exclusively uses XML for configuration. (It does, however, support
reading the XML from an environment variable.)

'gtkdialog' clearly can not be a transparent drop-in relacement for any
other dialog-like tool. However, when you start looking at them, except
for very simple use cases, none of them can be transparently swapped
in/out. I use a support function to map my internal expectations to the
appropriate dialog commands.

'gtkdialog' has the capacity to produce some really complex, nice
looking dialogs. It has the capacity to use "stock images" for the
error and sorry icons, so even that could be handled. Unfortunately, it
expects you to source its output to get the variables in your script,
and it does not properly escape them. All it does is wrap double-quotes
around the string values, allowing for arbitrary command execution by
entering a string value like '" ; evil-command ; true "' (That is, you
type what is within the single-quotes.)

Needless to say, I take the 'gtkdialog' output and stuff it in a
variable which either gets grepped for explicit strings (in cases of
simple message boxes), or passed through sed (in cases of choices and
string input). The worst thing that can happen to me if I get bogus
input is my variable will be empty. -- If they enter double-quotes, I'll
get them. If they manage to enter a NL character (the only character
I treat as special, and it shouldn't be possible to enter), I get
whatever preceeded it.

This reminds me, those of you unfamiliar with BASH's <<< "$VAR" feature
should check it out. It is relatively new, but allows you to do things
like 'sed ... <<< "$VAR"' where otherwise you would need to do 'echo
"$VAR" | sed ...'.

For those unfamiliar with shell scripts and TTY-based dialog tools, some
dialog tools allow you to send the displayed screen to stderr, and the
dialog output to stdout. This allows constructs like VAR=`somedialog`
to work. This isn't supported on all UNIX-like operating systems, so it
should be avoided. (In addition to not being available for all dialog
tools.) These constructs work well for X-based tools, which have no
other use for stdout, but can bite you in the portability-ass if you
expect them to work everywhere for console-based tools.

This means you need to use a temporary file. I hate temporary files, but
it is the canonical method. You also need to be making your temporary
file in a safe manner. /tmp/$APPNAME.$$ is unsafe. I use tempfile, then
mktemp, then fallback to $APPNAME.$$.`hostname` if neither of those
tools are available. (The last is known unsafe, but also known to exist
everywhere.)

I use wget and sed to do an initial download of the feed to scoop the
title out of it, presenting it to the user.

Podget has a concept of grouping the podcasts by group or genre. I've
relatively recently started implementing support for presenting a menu
of the currently used genres, with the option of specifying a new one.
(They're pulled from the directories, so genres which have completed
titles sitting in them get found, but new genres which have never been
downloaded in to do not.)

The original 'dialog' command has a super sweet 'progressbox'
feature. (This is different than the 'gauge' feature which shows a
progress gauge. 'zenity' conflates these, calling its 'gauge' feature
'progress'.) Basically, it is a stdin tailbox that quits cleanly
when the pipe breaks. (Example: Using a command like `$PROG | dialog
--progressbox ...` 'dialog' displays the output of $PROG within a nice
looking 'dialog'-styled window and quits cleanly when $PROG finishes.)
Unfortunately, few other dialog tools seem to support this feature. It
allows me to offer to run 'podget' (if they activated the feed) and show
the output within a 'dialog'-style window. (I can just show the output
within a terminal, sure, but it isn't as pretty.)

Oh, 'tcdialog' appears to be a stripped down version of 'dialog' which
is bundled with TeX Live. (On my system it is in the 'texlive-base-bin'
package.) It is basically worthless, having fewer working features than
it actually documents in the --help.

In the end, I decided I could not support 'tcdialog' or 'zenity'.

My particular use case found Xdialog, while having all the core
features, generally was uglier than kdialog or gtkdialog. (In
particular, it lacks support for icons for sorry and error message. It
also has an unnatural fondness for fixed-width text.)

I really wanted to support 'whiptail'. I looked at it at some point,
and I distinctly remember it handling unknown screen sizes better than
dialog. That's no longer the case, though. I do support it, but 'dialog'
results in a better looking display. 'whiptail' favors text wrapping at
the max column size possible, while 'dialog' balances width and height
to generate a more visually appealing display.

Now, I know you saw me mention 'bash' as a supported option, and you may
be wondering what features are missing from it. In fact, it is easier
to support 'bash' than it is to support 'zenity'. I only ask questions
one at a time, so 'read' functions well there. And, for the selection of
group/genre, 'select' works well (it also includes the ability for the
user to type a new group/genre not present in the list). What it doesn't
do is any line-wrapping. This is pretty much a non-issue in the BASH
use-case, though. It means it supports all sizes of terminals equally,
and only relies on minimal termcap/terminfo support. Best of all, it
means it is serious when it says it only relies upon BASH.

Once I finish polishing it up, I intend to contribute it to the podget
folks.

... and here I thought I hadn't done much in Linux recently outside of
work...

--
Steven Black <blacks@indiana.edu> / KeyID: 8596FA8E
Fingerprint: 108C 089C EFA4 832C BF07 78C2 DE71 5433 8596 FA8E

_______________________________________________
BLUG mailing list
BLUG@linuxfan.com
http://mailman.cs.indiana.edu/mailman/listinfo/blug

No comments: