[r-t] Leary's 23-spliced

[Don Morrison]

This is a long message. If you don't feel like reading the whole
thing, I do suggest you at least skip to the end. The last four
paragraphs contain a plea for help, which one or more of you might
feel like offering.

On Tue, Aug 19, 2008 at 1:44 PM, Philip Earis <Earisp at rsc.org> wrote:
> Thanks - I was unaware of this. So if this is a CC collection why
> is it hosted on Roger Bailey's website? Why is it not on the CC
> compositions committee website
> (http://www.cccbr.org.uk/compositions/) or on www.ringing.org?

Be careful what you ask for. You just might get an long reply from me.

The short answer is "because no one has done the work of massaging it
into a suitable format for pouring into the ringing.org database; any
volunteers?". Long answer follows.

On Tue, Aug 19, 2008 at 1:53 PM, James Hustler <mcwomble at gmail.com> wrote:
> Wasn't Roger on the peals committee at one time? (Pre Don's tenure as Chairman?)

Yes, Roger was my immediate predecessor as PCC chairman, and remains
on the Committee today, I believe (I'm no longer on the Council, so
may be mistaken about that).

And during Roger's tenure is when the tale begins.

I presume Roger still reads this list, and will correct me if my
memory plays any tricks on me in any of this. Please forgive any
errors I may make.

For probably over a decade ago now all publication of compositions by
the PCC, whether in the RW or in a print collection, have,
unsurprisingly, been via desktop publishing software of one sort or
another, and thus are available in some machine readable format.
There is wide diversity on what that format is. Some are (or were, I'm
not current in how the RW handles its publication process any longer)
in Quark XPress files, some in Microsoft Word, some as text files, and
at least one is only available as a Postscript document. There may be
other formats.

It seemed a great idea to gather all this information together and
make it available on the web. That idea languished at the back of
several peoples' minds for years, as there is some work required to
convert things appropriately, harmonize them, and so on. Not rocket
science, but neither is it "no work at all just push a button".

About a decade ago Roger and Roddy Horton took a first step towards
making this a reality. Roddy had done a huge piece of work gathering
all the peals of spliced surprise that had ever been published in the
RW into a single volume. He didn't just copy them, he reproved them
all by computer as well. Together they also put this collection onto
the web as a big text file, subsequently broken down into a few
smaller text files. It was that which Peter King pointed out John
Leary's composition in.

At the time this was done I suspect, though don't remember for sure,
that the Council had no web site at all. In any case, I'm sure it was
completely natural that Roger, who was then chairman of the committee,
put it onto his site, where it has remained ever since. In recent
years Richard Allton has looked after updating it with compositions of
spliced surprise that have been published in the RW since the
collection's original version. I think such updates happen
irregularly, probably only once every few years.

Sometime subsequent to that I started ringing.org. Its history and so
on can be seen in a very long message sent to the change-ringers
mailing list a few months ago: <http://tinyurl.com/625rom> (I'm not
sure how easy that will be to open, since access to the archives is
password protected--works ok for me, but your mileage may vary). So
long a message, in fact, it blew out the size limit for messages to
that list, and required John to bump it up a few KB. That long message
also explains something about the technology used (J2EE), and that
such technology is not currently available on the Council's web site.

It is a long range plan to eventually either move ringing.org onto a
host under the ICT Committee's control, or at least to leave the keys
to ringing.org in their hands to take over in case I get hit by a bus.
Note the adjective "long", however; it never seems to be the most
pressing thing.

When ringing.org first went live it incorporated an older quarter peal
collection and new collection of peal compositions, into which it was
hoped to pour as much of the existing stuff as possible. It's been
through a couple of iterations, but is currently a big database of
peal compositions, a description of how the pages of compositions are
to be structured, and a bunch of code that on request sucks
compositions out of the database and builds the requested web page.

I poured several print collections, for which electronic versions were
available, into the database: Stedman Triples, Grandsire, the latest
Popular Major, the Peter Border collection Rod Pipe assembled, and
undoubtedly some more I don't now recall. Roddy Horton kindly reproved
all the compositions from the previous version of Popular Major and
those were poured in, too.

The goal was also to pour in all the things published in the RW. I was
able to keep up with that for a little while, but I'm afraid my
erratic personal work habits do not accord well with something that
requires quotidian, or even weekly or monthly, efforts. Fortunately
Richard Allton is far better than I am about things requiring this
sort of diligence, and since he's taken over formatting things
intended for the RW into a form that can be poured into the database
it's gone much better.

Also poured in are things (such as compositions from John Ketteringham
and Peter Sheppard) which were submitted just for the web site, not
for the RW.

My own compositions show up as well, as I use just a single database
for all the compositions, peals and quarter peals.

A further artifact is that any extents or MEBs suitable for use in
peals that are in the quarter peal also show up automatically in the
peal collection.

There are several more print collections for which we have electronic
versions. They are languishing on my list of things to get to one day,
but never seem to bubble up to the top. One of these collections,
actually started, though not far along, is converting the spliced
collection into a form suitable for pouring into the database.

While still on the PCC, and once now since leaving it, I have made a
plea for its members, in looking for tasks to tackle, to please
consider putting some effort into the formatting one or another of
these collections into a suitable form for pouring into the database.
A few former members started to head such calls, and I supplied them
with the details of how to format things, but they appear to have lost
interest, and are no longer involved. I'm guessing the grunt work of
just formatting and tinkering with some existing files does not seem
as pressing or rewarding to folks as various alternatives.

In fact, it's worth noting that one or two other folks have been
interested in pouring things in, and I've given them details of how to
format things, but somehow, with one exception, nothing ever seems to
come of it. The one exception is Richard Allton who is doing a
wonderful job of pouring everything the committee receives newly now
in. From my perspective, at least, that is working very well. And I
think that's actually a harder job than formatting a single, extent
collection.

Anyway, if you think about how the large (over 10,000 compositions)
collection of peal compositions on ringing.org must be organized to
support building up meaningful pages, support searching, and present a
reasonably consistent look, it will be clear there needs to be some
meta-data kept consistently about the various compositions.

To do this, there's a simple XML format into which compositions need
to be cast. When I've poured collections in in the past I've generally
been able to automate much its creation with assorted Python scripts
and Emacs keyboard macros, all of which depend upon the consistency
that can be expected in the textual format of a single collection.
Generally such automation does not carry over to the the next
collection, and a different batch of automation thingies needs to be
written each time. And, of course, the automation only gets you 90% of
the way there, and there is invariably a little hand tweaking required
in the end. An important goal, though, is that the hand tweaking
applies only to the meta-data -- the text of the compositions
themselves should not be touched, lest errors be inadvertently
introduced.

Anyway, it's now time for the plea for help. There is no reason why
this massaging things into an XML format can't be done by one of you.
If you're willing to put in the effort to convert one of these
collections, your help would be much appreciated. Depending upon your
own skills and tastes you might want to do as much of it as possible
by writing scripts and so on, or you might do it the straightforward
way, formatting things by hand. While the former is probably less
drudgery and a bit less likely to introduce individual errors, either
seems perfectly practical. The major requirement of someone tackling
this is a willingness to be careful and pay attention to detail.

If you don't feel up to massaging a collection into a suitable form
for pouring into the database, there is another job that needs doing,
albeit it's seemed like a lower priority: proof reading. There are
plenty of instances where the formatting of something isn't quite what
it should be, or duplicates of the same composition have crept in from
multiple sources. While these problems get fixed when noted, it would
be wonderful to have someone look over the whole thing with a careful
eye and note all the problems.

As you'll have deduced from my own quietness on this list the last few
days, right now is a busy time for me in other matters, so I may not
be able to put much effort into helping someone get started on either
of these tasks for a few weeks yet, but we can certainly start the
conversation.

Any volunteers for either of them?



-- 
Don Morrison <dfm at ringing.org>
"We must not excite ourselves too much about political corruption or
democratic mismanagement: politics is not life, but only a graft upon
life; under its vulgar melodrama the traditional order of society
quietly persists."  -- Will Durant, "The Ten Peaks of Human Progress"