D. R. Evans (N7DR): 2016

2016-07-29

CY9C

Reception of the CY9C DXpedition by the RBN.

First, continent-by-continent:

And now the six zones with the largest number of reception reports:

Notes

I was away for most of this DXpedition, and so missed them on 80m, where they would have been a new one. They were audible in the evening on the last two nights, but the QRN level from thunderstorms was very high. They were copyable and worth calling at sunset, but unfortunately both evenings they were QRX for about 20 minutes just at the time of their peak.

"The Second Book" available in electronic formats

My most recent literary novel, The Second Book, is now available in several electronic formats:

Amazon for the Kindle ($3.50);
Smashwords in several formats ($3.50);
Barnes and Noble for the NOOK ($3.50);
iTunes store for Apple devices ($3.99).

2016-05-07

New augmented logs for CQ WW from 2005 to 2015

I have added two more columns to the flags on each line of the CQ WW augmented logs (the MD5 hash of the file is: 6e8d27f709f13b1851e3bb134592d336).

The additions to the standard CQ WW Cabrillo QSO line are now (changes in bold):

The letter "A" or "U" indicating "assisted" or "unassisted"
A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
Band
A set of ten flags, each encoded as T/F:

a. QSO is confirmed by a log from the second party
b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party)
c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party)
d. the call of the second party is unique
e. QSO appears to be a NIL
f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest
g. QSO appears to be a country mult
h. QSO appears to be a zone mult
i. the QSO is a zone bust (i.e., the received zone appears to be a bust)
j. the QSO is a reverse zone bust (i.e. the second party appears to have bust the zone of the first party)

Note that the encoding of some of the flags requires subjective decisions to be made as to whether the flag should be true or false; consequently, and because CQ has yet to understand the importance of making their scoring code public, the value of a flag for a specific QSO line in some circumstances might not match the value that CQ would assign. (Also, CQ has more data available in the form of check logs, which are not made public.)

2016-04-17

Using Adrive with rsync and SSH keys

Jump directly to solution (lower on this page).

My experience setting up an Adrive account to use rsync with SSH keys

When copy.com announced that they would no longer offer a hosting service that would allow me to make large files publicly available, I looked around for other choices. Eventually, I chose Adrive, partly because they had a good price for the amount of storage I am likely to need for this purpose, and partly because they claim that one can use their storage using ssh-based tools (in particular, rsync), which would make management easy. (I note in passing that for online storage and management of private information, I use and recommend rsync.net.)

On the Adrive login page, it says:

SCP, SFTP and Rsync
Access all your files from anywhere using SCP, SFTP and Rsync including server-side SSH key management.

which certainly sounds promising.

The instructions for using rsync are simple:

rsync files_to_upload username@rsync.adrive.com:.

Wonderful, except that that doesn't work. Worse, any attempt to use rsync causes a request for a password to be displayed, despite the site's mention of support for "server-side SSH key management". I tried to deal with the ssh problem first.

They provide apparently detailed instructions as to how to generate a key for use with Adrive and how to upload the key so that it will be used.

The trouble is, the instructions don't result in keys that are used; trying to use rsync still prompts the user for the password.

Having discovered all this, I entered the world of Adrive non-support. The support page tells Premium users:

Please browse through our FAQ to see if your question or problem has already been addressed.

Well, it didn't take more than a few moments to see that any question of a technical nature is noticeably absent from the FAQ.

The support page continues:

Search the ADrive Forum to find answers within our community, or to post a new question, technical support issue, or concern.

It turns out that there is a forum dedicated to SCP, SFTP and Rsync issues. Of course, actually posting to the forum, even though one is a Premium customer with an Adrive username and password, is, as one is beginning to expect with this company, not as simple as it should be: one has to register anew to obtain permission to post to the form.

Having accomplished reregistration, I posted my ssh question, and waited for a response. And waited. And waited. After three weeks without any reply, I decided that enough was enough, and it was time to file an official support ticket. I did this via the Adrive web form.

Very soon, I received an acknowledgement via e-mail:

Thank you for submitting your support issue to ADrive. Our Support Team has received your submission. We are reviewing your question and will work to resolve it as soon as possible. If you have not already done so, please browse through our FAQ <http://www.adrive.com/info/faq> and our Forum <http://www.adrive.com/forum> to see if your question or issue has already been addressed.

Unfortunately, I received no actual useful response in the days that followed.

After a few days,. I tried again, resubmitting the issue. I received the same acknowledgement, and, as before, no reponse that actually addressed the issue.

I then tried directly e-mailing support@adrive.com. That produced no response at all.

At that point, I finally was forced to the conclusion that Adrive "support" does not actually exist in any meaningful manner, even for paying customers. The claims of support on the web site, are, it seems from my experience, intended to make the unwary believe that there is a professional support team available, when it appears that in fact any message sent to Adrive support merely ends up in /dev/null.

So at that point I decided to spend the time to resolve the issue myself (although, of course, a genuine support team could have given me the solution more or less instantly).

Forcing Adrive to use an SSH key

The solution, or at least the solution that worked here, isn't hard, but it is rather obscure.

After creating a key pair and uploading the public key, create a file called ~/.ssh/config, and insert the following lines:

Host rsync.adrive.com
IdentityFile <path to key>

That should force rsync to use the correct key.

The correct form of the rsync command

There remained two problems on my system: firstly, every time I tried to use rsync, I would receive the message:

X11 forwarding request failed on channel 0;

and, secondly, the command
rsync files_to_upload username@rsync.adrive.com:.
didn't actually transfer anything (hardly surprising, since files_to_upload can't possibly mean what it says).

These problems can both be solved by using the following form of the rsync command:

rsync -rvt --delete -e "ssh -x" <directory-to-upload>/ "username@rsync.adrive.com:."

where username is replaced with your Adrive username, including the commat symbol. The ssh -x portion removes the error message about X11. The rest of the command causes rsync recursively to transfer the files in <directory-to-upload> to Adrive. The --delete option causes files that have been removed locally also to be removed on Adrive. (Using -rqt instead of -rvt will cause rsync to operate silently, which may be more useful than -rqt when the command is used in scripts).

2016-03-18

Most-Logged Stations in CQ WW SSB

As I mentioned in my last post, I thought that it might be interesting to use the public CQ WW logs to see which calls are logged most often in CQ WW, and how that changes from year to year. In this post I'll tabulate the SSB results.

As before, the first column is the callsign. The second column is the total number of times that the call appears in logs. That is, if a station worked HC8N on six bands, that will increment the value in the second column of the HC8N row by six. The third column is the percentage of logs that contain the callsign at least once.

CQ WW 2005:

Callsign	Appearances	% logs
CT3YA	6510	73
LZ9W	5616	66
V26B	5601	61
6Y2Z	5481	57
DF0CG	5359	65
RW2F	5259	63
OT5A	5251	65
MD4K	5007	60
9A7A	4999	62
DQ0Q	4994	62

CQ WW 2006:

Callsign	Appearances	% logs
DR1A	6414	71
DF0HQ	5905	67
MD4K	5707	60
CT3YA	5676	66
OT6A	5548	64
V26B	5539	59
EA8AH	5515	65
3V6T	5397	61
9A7A	5356	63
CT9L	5287	63

CQ WW 2007:

Callsign	Appearances	% logs
DR1A	6813	66
AO8A	6582	65
DF0HQ	6488	63
V26B	6426	57
CQ9K	6325	65
TS6A	6257	60
OT5A	6020	61
HC8N	5917	55
C50C	5866	57
LZ9W	5851	59

CQ WW 2008:

Callsign	Appearances	% logs
AO8A	6008	63
DF0HQ	5722	62
OT5A	5357	60
LZ9W	5205	57
P33W	4940	56
D4C	4898	54
CT9L	4714	54
6Y1V	4692	48
IR4X	4631	55
CN3A	4513	54

CQ WW 2009:

Callsign	Appearances	% logs
DR1A	7995	68
DF0HQ	7402	63
9A1P	7324	64
LZ9W	7089	60
YT9X	6976	61
AO8A	6769	60
OT5A	6361	59
OH0Z	6334	56
OL7R	6226	58
EE2W	6039	59

CQ WW 2010:

Callsign	Appearances	% logs
DR1A	7986	66
CR3A	7893	62
EF8R	7875	62
CN3A	7778	63
DF0HQ	7158	60
LZ9W	7036	59
A73A	6690	53
CR2X	6120	53
PJ2T	5973	45
OT5A	5894	55

CQ WW 2011:

Callsign	Appearances	% logs
DR1A	10118	66
C5A	9450	65
9A1P	9314	61
C37N	8705	56
CR3L	8323	59
OT5A	8211	58
LZ9W	8156	56
K3LR	8040	55
A73A	7487	53
DF0HQ	7291	52

CQ WW 2012:

Callsign	Appearances	% logs
DR1A	10734	65
HK1NA	10171	62
C5A	10060	63
LZ9W	9798	60
9A1A	9779	61
DF0HQ	9328	59
OT5A	9279	58
CN3A	9029	60
V26B	8704	55
PJ4X	8703	56

CQ WW 2013:

Callsign	Appearances	% logs
ES9C	12863	70
D4C	12274	69
DR1A	11230	66
HK1NA	10631	61
II9P	10559	63
IG9Y	9991	59
PJ2T	9896	59
K3LR	9452	57
LZ9W	9445	57
CN3A	9142	60

CQ WW 2014:

Callsign	Appearances	% logs
II9P	11128	66
HK1NA	10676	62
LZ9W	10366	60
CN3A	9526	62
DF0HQ	9393	57
CN2AA	8648	58
OT5A	8607	55
K3LR	8409	54
P33W	8250	55
P40L	8246	54

CQ WW 2015:

Callsign	Appearances	% logs
CN2AA	15100	78
CN2R	10017	63
9A1A	9872	61
CN3A	9693	61
DF0HQ	9237	59
LZ9W	9045	56
HK1NA	8936	54
PJ2T	8456	51
K3LR	8310	54
EF8R	8254	57

We can also look at ten-year spans:

CQ WW 2005 to 2014:

Callsign	Appearances	% logs
LZ9W	68565	55
DF0HQ	66790	55
DR1A	65618	52
OT5A	64050	54
K3LR	61458	52
CN3A	59763	53
PJ2T	59700	47
LY7A	56378	50
V26B	54656	46
P33W	50855	45

CQ WW 2006 to 2015:

Callsign	Appearances	% logs
DF0HQ	76024	59
LZ9W	71994	54
CN3A	69456	58
OT5A	66397	53
K3LR	65974	52
DR1A	65622	49
PJ2T	63768	47
LY7A	58901	49
P33W	58520	49
V26B	56197	45

2016-03-15

Most-Logged Stations in CQ WW CW

I thought that it might be interesting to use the public CQ WW logs to see which calls are logged most often in CQ WW, and how that changes from year to year.

Looking at CW first (SSB will be a separate post):

CQ WW 2005:

Callsign	Appearances	% logs
HC8N	7302	73
PJ2T	6405	66
LZ9W	6329	75
IH9P	5881	69
TZ5A	5378	67
RW2F	5293	69
DQ0Q	5169	69
DF0CG	5106	66
EA6IB	5074	66
UU7J	5062	65

The first column is the callsign. The second column is the total number of times that the call appears in logs. That is, if a station worked HC8N on six bands, that will increment the value in the second column of the HC8N row by six. The third column is the percentage of logs that contain the callsign at least once.

CQ WW 2006:

Callsign	Appearances	% logs
HC8N	8170	73
IH9P	8167	75
5A7A	7715	74
EA8EW	7405	64
TZ5A	7111	70
Z38N	7105	73
CT9L	6753	73
EA6IB	6590	70
DF0HQ	6311	70
DR1A	6160	69

CQ WW 2007:

Callsign	Appearances	% logs
LZ9W	7797	73
EF8M	7443	72
3X5A	7395	66
EA6IB	6721	68
HC8N	6705	60
D4C	6522	63
HI3A	6274	58
PJ4A	6181	62
DF0HQ	6172	65
P3F	6138	64

CQ WW 2008:

Callsign	Appearances	% logs
D4C	7710	71
HC8N	7515	64
CT9L	7193	71
3X5A	6870	63
LZ9W	6691	70
DF0HQ	6351	67
EE2W	6160	54
KP2M	6009	58
P3F	5982	64
PJ2T	5888	57

CQ WW 2009:

Callsign	Appearances	% logs
HC8GR	8066	61
LX7I	7693	68
DR1A	7300	67
CR3L	7208	67
EE2W	6917	64
LZ9W	6776	64
DF0HQ	6736	63
6Y1V	6408	56
EA8URL	6268	60
UU7J	6119	58

CQ WW 2010:

Callsign	Appearances	% logs
C5A	9102	68
EA2EA	8940	71
LZ9W	8790	70
9A1A	8286	69
CR3L	8157	68
DF0HQ	8025	67
PJ2T	7748	55
YT2W	7634	67
KC1XX	7141	57
LX7I	7119	62

CQ WW 2011:

Callsign	Appearances	% logs
C5A	11477	73
DR1A	10731	70
9A1A	10399	70
LZ9W	9739	67
CR3L	9509	68
PJ2T	9474	62
SK3W	9354	65
DF0HQ	9221	65
TK4W	8555	61
C6AAW	8534	57

CQ WW 2012:

Callsign	Appearances	% logs
D4C	14304	79
C5A	12357	73
HK1NA	11133	67
Z38N	10900	67
CR3L	10696	68
9A1A	10420	69
DR1A	10122	65
LZ9W	10090	67
PJ4A	8995	59
PJ2T	8773	57

CQ WW 2013:

Callsign	Appearances	% logs
D4C	15607	83
ES9C	12575	74
9A1A	11606	73
ED6A	11477	69
HK1NA	11264	68
PJ2T	11058	66
DR1A	10666	66
LZ9W	10250	67
CR3L	10103	67
IR9Y	9471	63

CQ WW 2014:

Callsign	Appearances	% logs
CN2AA	16754	83
9A1A	12234	73
LZ9W	11784	70
HK1NA	11660	68
9K2HN	11499	70
PJ2T	10433	63
EA5KA	10412	66
DR1A	10151	64
DF0HQ	10093	65
K3LR	9911	61

CQ WW 2015:

Callsign	Appearances	% logs
D4C	11129	68
9A1A	11047	72
LZ9W	10375	68
PJ2T	9498	58
CR3L	9456	63
CN2AA	8934	59
LX7I	8933	62
YT5A	8827	62
P33W	8764	60
DF0HQ	8537	61

We can also look at ten-year spans:

CQ WW 2005 to 2014:

Callsign	Appearances	% logs
LZ9W	81702	68
PJ2T	75453	59
DF0HQ	70559	60
DR1A	66706	55
K3LR	64391	54
W3LPL	63800	53
9A1A	62050	53
LX7I	61869	56
LY7A	61258	57
PJ4A	60797	52

CQ WW 2006 to 2015:

Callsign	Appearances	% logs
LZ9W	85748	67
DF0HQ	79095	64
PJ2T	78546	58
9A1A	73092	58
PJ4A	68977	56
W3LPL	68458	54
K3LR	68360	55
LX7I	67527	57
D4C	67448	48
DR1A	66737	52

Online Files Now Hosted By Adrive

I have been using copy.com to host the files I make available online. Unfortunately, that service is going to close in a few weeks' time.

So I have copied the files over to the Adrive hosting service.

The relevant files are to be found at the following new locations:

RBN data: https://www.adrive.com/public/cQwkEB/rbn
CQ WW logs: https://www.adrive.com/public/Ws5CQw/cq-ww
CQ WPX logs: https://www.adrive.com/public/BaAVjK/cq-wpx

I am sorry for any inconvenience caused by this change.

The files will continue to be available at their original locations on copy.com until that service is closed.

2016-02-16

"Nowhere to Run" available in electronic formats

My Christian action novel (for want of a better description), Nowhere to Run, is now available in several electronic formats:

Amazon for the Kindle ($3.50);
Smashwords in several formats ($3.50);
Barnes and Noble for the NOOK ($3.50);
iTunes store for Apple devices ($3.99).

Don't ask me why Apple charges more than the price I set. They seem to do this with all my books.

Most of the above links also allow you to read a sample, typically 10 to 15 percent of the book.

This book was inspired by events in the life of a fellow member of the Rotary Club of Boulder (Colorado) and his wife.

You can, of course, still purchase a print version of the book.

2016-01-24

Plots from CQ WW logs

The availability of the CQ WW public logs covering the period 2005 to 2015 allows us to look at various trends over that period.

The number of logs shows that for both CW and SSB the total number of entrants has been more or less stable for the most recent years, following a period of increase:

Breaking it down between unassisted and assisted, we see:

The number of assisted entrants shows a remarkably constant increase from year to year, whereas the unassisted numbers are close to constant. It will be interesting to see whether the assisted lines cross the unassisted lines next year.

The geographical distribution of entrants on CW has changed little over the period:

As I have noted before, on SSB the percentage of logs from western EU has decreased somewhat, and the percentage from JA/HL and zone 28 has slightly increased.

Of more interest (well, I think so, anyway) than the numbers of logs is the number of QSOs in the logs:

This shows the perhaps not-unexpected reality that as the number of assisted logs has increased, the number of QSOs in the top 10 percent of logs (as determined by number of QSOs in the log) has steadily declined. The median number of QSOs (represented by the coloured near-horizontal lines) has varied little, with only SSB Assisted showing a clear change in recent years, although both CW and SSB Assisted show a more or less gentle trend downwards in the median number of QSOs in the logs.

One unexpected fact emerges: the median log in CQ WW CW has consistently contained about a hundred more QSOs than has the median CQ WW SSB log. I'm not sure why this should be, but it is a robust result.

Now, how about whether the CQ WW contest is actually gaining in popularity, as is often claimed (without any supporting evidence for the declaration, as far as I know). How many people are actually getting on the air and making QSOs in the contest?

Different people may set the bar as to how many times a call has to appear in the logs of other stations at different values. For example, if a call appears only once, it might well be a bust rather than a real participant. Even if a call appears, say, twenty times, that might merely reflect a busted spot that was logged by twenty people who did not check the call.

So we can produce a series of nearly-parallel lines, setting the required number of appearances at different values.

On CW, it makes no substantive difference which definition of participation one chooses: the number of participants is essentially unchanged from year to year. Any increase in recent years is marginal at best.

On SSB, there is indication of a slight decline in recent years.

Finally, it's of some interest to plot the year-by-year number of distinct QSOs made on each band. By "distinct" I mean that a QSO both sides of which are logged by entrants counts just once.

For CW:

And for SSB:

In both cases, one can see the importance of 10m in determining the total number of distinct QSOs. However, while 10m is the most dominant band on SSB in years of good propagation, that is not true on CW: on CW, in years of good propagation, 10m, 15m, 20m and 40m provide essentially identical contributions to the total.

We also see that on CW, 20m and 40m provide nearly identical numbers throughout the cycle; on SSB, 20m and 15m have this characteristic.

The relative lack of dominance of 10m on CW is likely due to the timing of the two contests (indeed, I wish that both CQ WPX and CQ WW would swap the SSB and CW events each year; with the current calendar, CW suffers in both types of contest). By the end of November, the number of hours of common daylight between EU and NA are simply too few for 10m to be dominant.

"Nowhere to Run" available on Kindle

My book Nowhere to Run is now available for the Kindle. You may read a sample (from the hardcopy version).

2016-01-23

Obtaining Magnetometer Data

The Problem

Until a few days ago, it was possible to obtain near-real-time ("NRT") magnetometer data files from directories housed below http://magweb.cr.usgs.gov/data/magnetometer/. Attempts to reach that location, and the subordinate files that held the actual NRT data, now return:

The requested service is temporarily unavailable. Please try later.

After waiting for several days for something more useful to be returned, I finally realised that temporarily here seems to be government-speak for permanently.

There is still a way to obtain NRT magnetometer data, but it's vastly more complicated than simply downloading data from a URL. I do not know why the sites that make the data available do not hide all the complexity from users and simply provide a URL as an interface, but the fact is that, at least for now, they do not, and we now have to go through a rather tortuous process to obtain data that was, until a few days ago, essentially trivial to obtain.

Obtaining the data in the absence of a URL

These are the steps that I had to take on debian stable; other Linux distributions and other operating systems will likely differ in details, but the basic idea should be the same.

First, one needs to install python, since the data are accessed via a python script. Debian, like most Linux distributions installs python by default.
Next, install the packages python-numpy, python-scipy and python-flake8 from the distribution repositories.

Now, install obspy by following the instructions at: https://github.com/obspy/obspy/wiki/Installation-on-Linux-via-Apt-Repository. If using a different distribution or operating system, follow the pertinent instructions at https://github.com/obspy/obspy/wiki.

Now that python and the necessary libraries are available, we need to download the actual software package that does the work; we get this from: https://github.com/usgs/geomag-algorithms.

I note that the installation instructions on the site read, in their entirety:
First time install. Walk through dependencies and other considerations.

If one looks inside the downloaded package, there is no sign of the standard INSTALL file. Neither is there a README or README.FIRST file. There is a README.md file, but its contents merely repeat the above "instructions" from the site. So I simply unpacked the package in a reasonable location and used it as-is. I am sure that that is not what one is supposed to do, but in the absence of explicit, detailed installation instructions, it will have to do.

After unpacking that software somewhere suitable, we have access to the script we need, which is geomag.py in the .../bin directory.

The documentation is... well, the politest word I can think of is "sparse". As unfortunately seems to be so often the case when trying to use government data, it is hard to escape the conclusion that they don't really want it to be used. They certainly don't seem to go to any effort to explain the details of the package in a way that is useful to users. So we're pretty much on our own.

The command geomag.py -h offers a lot of output, most of it incomprehensible without further documentation. However, the following command (typed on one line) produces output in a format very similar to that stored at the URL that in the past provided one-minute Boulder magnetometer data for a particular day:

geomag.py --inchannels H D Z F --observatory BOU --starttime 2016-01-23T00:00:00Z --endtime 2016-01-23T23:59:00Z --input-edge cwbpub.cr.usgs.gov --output-iaga-file output-file-name

From this point one can process the file output-file-name appropriately, in a way similar to the manner in which the old URL-based data could be processed. For example, I have a widget on my desktop that displays the last couple of hours of data from the Boulder magnetometer, using the above method to obtain data every five minutes:

CQ WW logs, 2005 - 2015

The logs for the 2015 running of the CQ WW CW contest are now available from CQ. This means that logs from the period 2005 to 2015 are now available for both SSB and CW.

As before, I have created a compressed file that contains all the cleaned QSO lines from the Cabrillo files from all the logs for all the years for which data are available. The MD5 checksum of this file is: 84864c3d7a7f80653243a35ae014f0dc.

I have also created an augmented file, in compressed format, that adds useful data to each QSO. Each QSO line in the augmented file includes an additional four columns, with the following meanings:

The letter "A" or "U" indicating "assisted" or "unassisted"
A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
Band
A set of eight flags, each encoded as T/F:

a. QSO is confirmed by a log from the second party
b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party)
c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party)
d. the call of the second party is unique
e. QSO appears to be a NIL
f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest
g. QSO appears to be a country mult
h. QSO appears to be a zone mult

The MD5 checksum of this file is: f2ab0dc3af3a4b5ed3f1846953c4d03b.

Note that the flags in the augmented data are calculated from the raw data independently of CQ. This is because:

for reasons I cannot guess, CQ does not make the actual scoring code available ;
the checklogs are not public, and hence represent additional data that CQ can use in determining the values of the flags.

CQ WW Videos Updated

I have updated the set of CQ WW video maps on my youtube channel (channel N7DR). These video maps cover all the years for which public logs are currently available.

To access individual videos directly:

2016-01-07

CQ WW SSB 2015 logs

The logs for the 2015 running of the CQ WW SSB contest are now available from CQ.

As usual, I have created a compressed file that contains all the cleaned QSO lines from the Cabrillo files from all the logs for all the years for which data are available. As the 2015 data for the CW running of the contest are not yet available, this file contains only the SSB data. Currently, the file covers all the QSOs in the years from 2005 to 2015. The MD5 checksum of this file is: b532ba3307583c379c826314b81a5d04.

I have also created an augmented file, in compressed format, that adds useful data to each QSO. Each QSO line in the augmented file includes an additional four columns, with the following meanings:

The letter "A" or "U" indicating "assisted" or "unassisted"
A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
Band
A set of eight flags, each encoded as T/F:

a. QSO is confirmed by a log from the second party
b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party)
c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party)
d. the call of the second party is unique
e. QSO appears to be a NIL
f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest
g. QSO appears to be a country mult
h. QSO appears to be a zone mult

The MD5 checksum of this file is: 1e3f1862d4c8126a5e85639368941d3a.

Note that the flags in the augmented data are calculated from the raw data independently of CQ. This is because:

for reasons I cannot guess, CQ does not make the actual scoring code available ;
the checklogs are not public, and hence represent additional data that CQ can use in determining the values of the flags.

Obviously, a huge number of analyses can be performed with these various files. There follow a few that interested me.

Geographical Participation

How has the geographical distribution of entries changed over time?

So there seem to be no major variations over time: the percentage of logs from western EU has decreased somewhat, and the percentage from JA/HL and zone 28 has slightly increased.

In terms of raw numbers of logs, almost everywhere with substantive participation has shown a reasonably consistent increase:

Popularity

I not-infrequently come across statements to the effect that contesting in general, and CQ WW in particular, are increasingly popular. Usually, no evidence for the statement is provided, as if it were self-evident; the only purported evidence I have seen is referral to the fact that the number of entries is increasing -- which is manifestly not the same as an increase in the popularity.

By definition, popularity demands some measure of people (or, in our case, the simple proxy of callsigns). So we can look at the number of calls in the logs as a function of time:

I find this graph particularly interesting, not just because it shows that the popularity of CQ WW SSB appears to have peaked a few years ago, but also because it shows that the result is quite robust regardless of how many times one deems it necessary for a call to appear before that call is deemed a participant.

Activity

We can also look at the change in activity as a function of year. Activity depends on the number of people participation, and on how many QSOs those people make:

Here the words "distinct QSOs" is intended to convey that a QSO is counted once, even if both participants have contributed a log.

Not unexpectedly, this plot shows that the total number of QSOs is dominated by conditions, and, in particular, the state of 10m during the contest.

2016-01-02

2015 RBN data

All the postings to the Reverse Beacon Network in 2015, along with the postings from prior years, are now available in the directory https://copy.com/smnSEO4RoQuTj4mC.

Some simple annual statistics for the period 2009 to 2015 follow (the 2009 numbers cover only part of that year, as the RBN was instantiated partway through the year).

Total posts:

2009: 5,007,040
2010: 25,116,810
2011: 49,705,539
2012: 71,584,195
2013: 92,875,152
2014: 108,862,505
2015: 116,754,643

Total posting stations:

2009: 151
2010: 265
2011: 320
2012: 420
2013: 473
2014: 515
2015: 512

Total posted callsigns:

2009: 143,724
2010: 266,189
2011: 271,133
2012: 308,010
2013: 353,952
2014: 398,293
2015: 433,812

Obviously, much more comprehensive statistics may be derived rather easily from the files in the directory.