2025-01-20

A Saga of WAE and DCL

At some point in the last few years, the application procedure for the Worked All Europe award  has been changed, profoundly. In the past, the process was more or less the ordinary one used by many awards: make a list of the cards needed to qualify; have it checked by a couple of local amateurs; submit it to the relevant DARC award manager, along with the fee. This took a bit of time, but was easy and essentially stress-free once the needed cards were in hand.

Recently, I decided that, as I now had the requisite cards, I would apply for the WAE-TOP plaque as described on the WAE page. (I note in passing that, from my location in Colorado, WAE-TOP is considerably more difficult than is 5BDXCC, as if the latter weren't hard enough.) The plaque, I thought, would look good on my shack wall, and would celebrate a feat that is rather difficult from here. So I started out the application process with some enthusiasm; you can imagine as you read what follows how that keenness waned with each roadblock that I encountered.

On the WAE page I discovered that one is now expected to submit one's application "in the Darc (sic) Community Logbook (DCL)". There is a statement that "A traditional application form may be requested in exceptional cases from DF8AA", but, even though it doesn't suggest what might qualify as "exceptional", there was nothing obviously different about my application that would suggest that it would meet whatever those criteria might be. So, "OK", I thought, "if that's what I have to do, that's what I'll do".

Obviously, the first thing to do is to register for DCL. Hmmm... the page is in German (hardly surprising), which I don't read. But there is a Union flag, so I clicked that. Well, a bit of the page is now in English, but most of it remains in German -- although in an oddly inconsistent way.

There is a "Register new account" link, but before doing that, I thought it a good idea to see roughly how the process of submitting a WAE application would work. There's a "Help" tab, and that brings up two sub-items: "Help" (again) and "Logbook Quick Start Guide". The latter sounds promising, so I click on it. Ah! That brings up a complicated sheet written in German -- even though I had clicked the Union flag. OK, click the "Help" instead... and this page appears. Even though the URL (https://confluence.darc.de/display/DX/DCL+-+DARC+Community+Logbook) is in English, the actual page that is displayed is in German. So DCL seems to have forgotten that I had clicked the Union flag... and on the German Help page, there is no flag to click.

Could I have bumbled my way forward? Yes, of course I could. But if such simple functions weren't working in a sensible manner, this did not bode for the more complex interactions that would doubtless be necessary later. So I thought I would communicate with the DCL administrator (Martina, DO8MKR) and see what she had to say.

(Throughout what follows, there were several interactions with DO8MKR. She was unfailingly polite and, clearly, doing her best to help in what are perhaps invidious circumstances. Unfortunately, it quite soon transpired that she was in the position of being forced to defend some very sloppy programming that, in my opinion, should have undergone considerably more testing before being made available, at least to non-German-speaking users. In what follows I omit some details of our interactions because they would not be useful.)

Responding to my suggestion that, even though the Union flag had been clicked, much of what was presented to a user was still in German, she said, "I could not discover any fundamental problems with the British flag", which I interpreted to mean that my expectation that information would be presented in English after clicking the flag did not match the intent of those who created the DCL website.

I then registered on the site, on 25 September. Martina then sent me an e-mail to say that "Your registration was done on 2024-08-17". This made (and still makes) no sense, so I just but put it down to a bug and ignored it, not bothering to raise it as an issue.

I created an ADIF file containing 341 records, corresponding to the QSL cards I had available in support of my application for the WAE-TOP plaque. (I note in passing that, apart from contests, I log on paper; so it took a while for me to create this file.)

In the DCL, one uploads an ADIF file by clicking "Logbook" and then clicking "ADIF-Import"; one then follows the prompts. I did this, but was confused by the result. The lack of useful feedback (in any language), as I later discovered, is endemic in DCL. I had received no indication of an error, and yet when I tried to examine the logbook by clicking on "Search/Manage Log", the logbook seemed to have no QSOs. Unlabelled columns only added to my feeling of hopeless confusion. 

So I wrote to Martina:

I'm unsure what to do next. The attached screenshot shows the "List of Import Jobs". It has a red "X" in the rightmost column. I don't know if that matters. The column has no header, so I can't tell what is the purpose of that column. If I go to the "Logbook" page, I don't see any of the uploaded QSOs, so I'm not sure whether the upload worked, or, if it failed, what the reason was for the failure.

The response (but also see below):

When testing with an adi-Checker, I receive the following error message: Consistency error on line 1299: Frequency '18' is out of range for band '17M' Therefore no QSOs are uploaded to DCL

 The referenced QSO was (with line numbers added):

  1299    <BAND:3>17m
  1300    <CALL:5>G4AMT
  1301    <FREQ:2>18
  1302    <MODE:2>CW
  1303    <QSL_RCVD:1>Y
  1304    <QSO_DATE:8>20110114
  1305    <RST_RCVD:3>559
  1306    <RST_SENT:3>559
  1307    <STATION_CALLSIGN:4>N7DR
  1308    <TIME_ON:6>163400
  1309    <EOR>

  • The "consistency error" (to the extent that there is one) is not on line 1299. That is the first line of the QSO, so there is nothing against which to compare it for consistency.
  • The "error" seems to be that line 1301, which defines the frequency to be 18MHz, is deemed incompatible with the band, 17m.

OK, so, technically, 18MHz is not inside the 17m band in any part of the world. But, really... an error? I log on paper, and always log the WARC bands as simply a number of MHz (completely eliding here the issue that the 12m band is a lot closer to 25MHz than 24 MHz). Other systems that accept ADIF files (for example, Clublog and QDURE) , happily accept the QSO, and almost all QSLs I receive for QSOs on WARC bands similarly record the frequency simply as an integral number of MHz -- that is, technically, outside the band. Ultimately, there is no requirement in the specification (that I can see) one way or the other about the consistency of the BAND and FREQ fields, so it's certainly permitted that DCL can impose a constraint that the frequency must be inside the pertinent band. But it certainly doesn't seem very sensible that they do so.

  • Even if one holds that the QSO should be rejected, that seems no reason to reject all the other QSOs in the file: "[t]herefore no QSOs are uploaded" (emphasis added).

The "fix", regardless of whether it should be necessary, was easy: move all the frequencies so that they are inside the band. So I shifted 10MHz to 10.1MHz, 18 MHz to 18.1MHz and 24MHz to 24.9MHz. Roughly, and ignoring error processing, one trivial way of doing this would be:

 adif3_field& field { _elements["FREQ"s] };

 if (field.value() == "10"s)
    field.value("10.1"s);

 if (field.value() == "18"s)
    field.value("18.1"s);

 if (field.value() == "24"s)
   field.value("24.9"s);

Uploading the revised ADIF file... still failed. And still without providing any kind of reason.

This time. Martina responded:

After checking again, I have discovered that it was not necessarily due 
to the frequency. I only suspected this... 

It is unfortunate when user-facing support simply guesses as to the reason for a failure; especially when the said support does not hint that the response is only "suspected".

She continued:

However, with your new adi I was still able to add the QSO for you after updating the top line. You nead a header line in the adi-file. Please add <EOH> therefore in the first line.

So we have new advice; the trouble with it is that this produces a file that does not conform to the ADIF specification. (I am the first to admit that the ADIF specification falls far short of the precision that one expects in a professional document; nevertheless, on this point it seems unambiguous.)

Referring to the specification, we see:

IV.A.2. ADI File Structure

An ADI file begins with an optional Header followed by one or more Records:

Header
Record
Record
...
Record

and:

IV.A.3. ADI Header

A Header begins with any character other than < and terminates with a case-insensitive End-Of-Header tag:

<EOH>
If the first character in an ADI file is <, it contains no Header....

It follows directly from these requirements that a file that begins with "<EOH>" is not a compliant ADIF file (1: it begins with "<" and hence contains no header; 2: the "<EOH"> string is an End-of-Header tag, which is within, and marks the end of, a Header... which, by definition, the file does not contain). The example in the specification makes it clear that an "<EOH>" tag is part of the header and, as the text says, "terminates" it.

Nevertheless, having been advised to do so, I recreated the original file (without the altered frequency fields), but inserted the string "<EOH>" as a line at the start of the file (and added a blank line afterwards). The file now began (with line numbers added):

     1    <EOH>
     2    
     3    <BAND:3>20m
     4    <CALL:5>RZ3FW
     5    <FREQ:2>14
     6    <MODE:2>CW
     7    <QSL_RCVD:1>Y
     8    <QSO_DATE:8>20100718
     9    <RST_RCVD:3>449
    10    <RST_SENT:3>559
    11    <STATION_CALLSIGN:4>N7DR
    12    <TIME_ON:6>013800
    13    <EOR>

And, yes! it worked, despite the file's failure to conform to the ADIF requirements.

But some of the QSO data were incomplete on DCL: several of the QSOs were accompanied by an error message:

 "ignored, as tag's value too long (max. 5 characters allowed in DCL)"

It transpired that there is an undocumented limit of 5 characters to the RST_SENT and RST_RCVD ADIF tags imposed by DCL (the ADIF specification imposes no limit). So any entry such as "559/579" or "59 plus 20dB", or anything of that ilk is rejected by DCL.

So I created another non-compliant ADIF file, but with all the RST values limited to strings that DCL would find acceptable, with the intention that this file would represent my application.

In passing, I note that I raised these as issues with Martina, and she promptly responded:

we are currently in the process of reprogramming the DCL. As part of 
this, we will certainly also fix the problems mentioned.
However, this change will certainly take some time.

That at least gave me some comfort that others would not be subjected to the kind of mess that I was dealing with. However, later (see below), she also said:

there will be no more changes to the current DCL (e.g. function for 
deleting QSOs).

So either there will be changes, or there won't. Take your pick.

Going back to the increasingly-wearing application process: at this point now I naturally wanted to delete the QSOs currently in the DCL system, prior to uploading the actual QSOs to be used for the application. There was no obvious indication in DCL as to how to go about deleting data, so I asked Martina (by this time I was, understandably, I think,  beginning to get pretty annoyed at the programming underlying DCL and the lack of user-facing documentation -- at least, English documentation -- for the system). She replied:

... the [delete] function can be found in "im- and export" in the job
list. There is a red X at the right end of each line, which you can use
to delete the corresponding upload.

I had actually already tried that already, but I did it again, in case I had made some error earlier. But no, clicking the (undocumented) red X did not remove any QSOs from the logbook.

Back to Martina again, and this time I received the response:

of course you are right and the variant described does absolutely not 
work in your case. 

She continued:

The red X only deletes the QSOs that are not in confirmed status. As you 
have marked them all as T-QSL, no connections will be deleted. This only 
works for QSOs in status "x", but yours have the status "w" for worked

I'm sorry I didn't think of that at the time.

In this case, I also have no way of deleting the QSOs in our database. 
The only possibility would be to delete all QSOs manually in the logbook.

At this point, I wondered if I was going quietly insane... a logbook system that allows you to add hundreds or thousands of QSOs at a time, but requires you to delete QSOs one at a time? (In fact, it occurred to me that the simplest way to delete a non-trivial number of QSOs might be simply to delete the account, and then create a new one; but, given the experience so far, I guessed that the system might well not allow that, and I was -- at this point -- still sufficiently interested in completing the application process that I didn't want to run that risk.) So I soldiered on and sat in front of the computer for a while, tediously deleting all the QSOs that I had uploaded, seriatim. That at least worked. I then uploaded a clean file (I was going to write "ADIF file", but of course that would be wrong, since it didn't conform to the ADIF specification.) DCL accepted the file (hooray!). But then what do I do?

On the DCL site, I clicked various buttons in an attempt to find something that would tell me what to do next in order to get the system to understand that I wanted to apply for WAE-TOP. I must have done something right, although I didn't see anything that told me that an application had been created, because when I broke down and asked Martina what I needed to do next, she said:

You already have the WAE under "Award Request" - "All Requests".
If you click on the pencil to the right of the request, you will be 
taken to your award request.

 and:

At the top of the page in the blue area, the option "submit application" 
is no longer grayed out.
You can apply for the diploma and contact the diploma manager in the 
Notes section to check your T-QSL.

I didn't really understand that last part, but I found the "submit application" button and pressed it. It was nuclear what that actually did, but at that point I assumed that my application had been entered into the system. (I admit that I still don't understand why the application was present, as Martina pointed out, under "Award Request - All Requests", which, to a native speaker of English means that a request is present in the system, when another step was required. As best as I can tell, the system treats "request" and "application" as two different things, which a lawyer would be tempted to regard as a "distinction without a difference", at least in English.)

Working on the assumption that I had now created a valid request for WAE-TOP, I waited for something or someone to tell me what happens next.

What happened next was that four days later I received an e-mail from DF8AA (Wolfgang), the WAE award manager. So that was good. What was less good was what it said, which was:

I have received your WAE application.
The points are sufficient for the WAE TOP, but many QSOs still need to 
be checked, red in the application.
I see that you are a user of Clublog. It is possible to import confirmed 
QSOs into the DCL. This can make the verification procedure easier.

In the DCL, go to >Logbook > Club log import
follow the further instructions.

We will then see what further steps are necessary.

My first thought was along the lines of, "well, the old system was about as easy as possible; why was that abandoned?" ; and my second thought was, "And if I hadn't ever used Clublog, what happens?" Indeed, I have many physical QSLs for QSOs that are not on Clublog; what if I had needed to use some of those for this application? And that concluding sentence in the e-mail was a bit ominous: I mean, is this process not well-defined, even if it's undocumented? Further, it wasn't at all clear to me how much help Clublog data would be anyway; Clublog doesn't seem, as far as I can tell, to provide a simple way to test whether a collection of QSOs have been confirmed by the other parties, so I couldn't test this thesis, but a priori it seems unlikely that a collection of ~380 QSOs with Europe will have an overwhelming number of confirmations within Clublog.

Of course, I didn't raise any of these issues. Instead, I went to the DCL site and clicked as instructed in the e-mail.

This brought up a DCL page that, among other things, requested my Clublog password! Notice what I said there: this was a DCL request for me to enter a password for a service run by a different entity -- the request did not come from Clublog. There was a word in common use in the 1990s: gobsmacked. That pretty much sums up my reaction to this: do the programmers of DCL seriously expect people to provide them with their passwords for other services? Apparently, they do. (And just as mind-boggling, one can reasonably assume merely from the existence of the request that a non-trivial fraction of people are actually willing to do this. Is it any wonder that computer fraud is a vast, thriving business?) [The question also crossed my mind whether, given the EU's strict data laws, it is even legal for an entity in the EU to require me to surrender a password to another entity in order to use a service of the first entity. I don't know.]

Not being a complete idiot (although I freely admit that sometimes I do a reasonable impersonation of one; but I do try not to make it a lifestyle), I did not "follow the further instructions", per DF8AA's e-mail. Instead, I responded that I would not disclose a password to access my account on another service, and if that was actually a requirement for continuing the application, then I would just give up.

You can tell that by this time I had pretty much reached my breaking point ... how ridiculous is it to replace a process that worked fine for decades with something that is, by any measure, and according to my experience, at least several times as much effort... and following which, even after all this work, I still hadn't reached the point where the application was approved and I could legitimately expect a plaque for my shack wall? Surely, part of sponsoring an award is to make it as easy as possible for applicants, is it not? (Which doesn't necessarily mean mean that every applicant has to use the same process; but it does mean that, having achieved the necessary QSOs and received confirmation of those QSOs, in whatever form, the barrier to submitting the confirmations should be as low as possible.)

I admit that what I hoped for at this point was a statement that I could always just use the old application system, and, indeed, the next e-mail from Wolfgang said:

I also accept a confirmed GCR list.
Nevertheless, it is much quicker to import 210 confirmed QSOs from
Clublog into the DCL than to check each card individually. But anyway,
it's not my time.
Please send the confirmed GCR list as PDF to me.

I can't say that I understood the second sentence, but no matter: at last I was given permission to submit confirmations in the old way. I printed out a list and had a couple of local hams check it and sign the application, and e-mailed it to DF8AA.

And, a month or so later, I received in the mail:

So, in the end it's a happy story. At least, mostly happy. But the experience has certainly soured me on DARC awards. For several years I have been applying for, and receiving, DLD at various levels on 20m, 15m and 10m, all done the old-fashioned way. As long as that is acceptable, I'll continue to do so -- although it's getting harder and harder to find DOKs that I haven't already confirmed -- but I am fearful that at some point DLD may become DCL-only, in which case that will be the end of my fun of working DLs and collecting DOKs and DL QSLs. But for now at least, I can continue to enjoy the chase, and savour every new DOK.


No comments:

Post a Comment

Note: Only a member of this blog may post a comment.