2019-06-11

Statistics from 2018 CQ WW SSB and CQ WW CW logs

A huge number of analyses can be performed with the various public CQ WW logs (cq-ww-2005--2018-augmented.xz; see here for details of the augmented format) for the period from 2005 to 2018.

There follow a few analyses that have interested me. There is plenty of scope to use the files for further analyses.

 

Number of Logs


The raw number of submitted logs for SSB has been relatively flat for several years; the logs submitted for CW continues to show a fairly steady annual increase:

 
One not infrequently reads statements to the effect that the popularity of contests such as CQ WW has been increasing for the past several years. Certainly it is true that, for CW, the number of logs has generally increased year-on-year, but the above plot shows that the same cannot be truthfully said for SSB, for which the number of logs has shown no systematic variation for the last half-dozen years.

 

Popularity


By definition, popularity requires some measure of people (or, in our case, the simple proxy of callsigns) -- there is no reason to believe, a priori, that the number of received logs as shown above is related in any particular way to the popularity of a contest.

So we look at the number of calls in the logs as a function of time, rather than positing any kind of well-defined positively correlated relationship between log submission and popularity (actually, the posts I have seen don't even bother to posit such a relationship: they are silent on the matter, thereby simply seeming to presume that the reader will assume one). 

However, the situation isn't as simple as it might be, because of the presence of busted calls in logs. If a call appears in the logs just once (or some small number of times), it is more likely to be a bust rather an actual participant. Where to set a cut-off a priori in order to discriminate between busts and actual calls is unclear; but we can plot the results of choosing several such values. 

First, for SSB:


Regardless of how many logs a call has to appear in before we regard it as a legitimate callsign, the popularity of CQ WW SSB in the past few years has fallen to a level rarely (if ever) seen in the public logs. It is certainly difficult to argue, on the basis of the above plot, that this contest is now more popular than it was at a similar point in the last solar cycle -- indeed, it appears, on its face, that the opposite is true.

[I note that a reasonable argument can be made that the number of uniques will be more or less proportional to the number of QSOs made (I have not tested that hypothesis; I leave it as an exercise for the interested reader to determine whether it is true), but there is no obvious reason why the same would be true for, for example, callsigns that appear in, say, ten or more logs.]

Moving to CW:


we see a similar story to SSB, except that any decrease in participation since the same point in the last cycle appears to be very small: participation in the CW event in the current inter-cycle doldrums seems to be more or less the same as at the corresponding point in the last cycle.

 

Geographical Participation


How has the geographical distribution of entries changed over time?

Again looking at SSB first: 


Zone 28 continues to show a gradual but sustained increase in the number of logs submitted. Nevertheless, compared to the behemoths like zones 14 and 15, the number of logs from zones such as 11 or 28 is miniscule. This can be seen more clearly if we plot the percentage of logs received from each zone as a function of time:


On CW, most zones evidence a long-term increase:


But the relative increase seems to be spread more or less evenly across all zones, with the percentages of logs from each zone barely changing over the years 2005 to 2018:

 

Activity


Total activity in a contest depends both on the number of people who participate and on how many QSOs each of those people makes. We can use the public logs to count the total number of distinct QSOs in the logs (that is, each QSO is counted only once, even if both participants have submitted a log).

For SSB: 


The total number of distinct QSOs in the current inter-cycle doldrums is essentially the same as at the same point in the last solar cycle, although if last year is anything to go by, the next year or two may show even fewer QSOs than the last solar minimum.

And for CW:


On this mode there appears to be an underlying upward trend (on which the effect of the solar cycle is superimposed). Despite the claims I see that CW is an obsolete technology in serious decline, the actual evidence, at least from this, the largest contest of the year, is quite the opposite. (This is a good reminder that when someone makes a claim whose truth is not self-evident, one should examine the underlying data for oneself. I have found that all too often it transpires that no defensible evidence has been put forward for the conclusion being drawn.) The evidence certainly seems to indicate that CW activity is faring better than activity on SSB, at least insofar as CQ WW is concerned.

 

Running and Calling


On SSB, the ongoing gradual shift towards stations strongly favouring either running or calling, rather than splitting their effort between the two types of operation, continued in 2018:




I have not investigated the cause of the continued decrease in the percentage of stations strongly favouring running, although the public logs could readily be used to distinguish possibilities that spring to mind, such as more SO2R operation, more multi-operator stations, and/or a reluctance of stations to forego the perceived advantages of spots from cluster networks.

On CW, the split between callers and runners continues to be much less bimodal on CW than on SSB (on SSB, fully 30% of entrants have no run QSOs; on CW, the equivalent number is below 10%). Indeed, the difference in call/run behaviour on the two modes (and the difference in the way that the behaviour has changed over time) is profound, and probably worthy of further investigation. CW continues to appear to have what would seem to be a much healthier split between the two operating styles:






Inter-Zone QSOs


We can show the number of inter-zone QSOs, both band-by-band and in total. In these plots, the number of QSOs is accumulated every ten minutes, so there are six points per hour.


As expected at this point in the cycle, there were a negligible number of QSOs on 10m, in either the SSB or the CW events.



In 2018, activity decreased substantially on 15m as compared to 2017. We certainly seem to be very close to the bottom of the cycle. Perhaps by next year there will be a slight improvement in conditions.


Especially on CW, 20m was the place to be, on both days showing more activity on CW than in any prior year for which we have data.


As usual, CW dominates on 40m (and the other low bands). Usually, the bulk of CW activity is in the first six hours of the contest; in 2018 the last few hours also showed strong activity.


80m was also dominated by CW, with, as usual, the bulk of activity in the first six hours.



160m paints a similar story to 80m, although the raw QSO counts are much smaller. 2018 appears to be the strongest year on record for CW activity on 160m in the first six hours of the contest.


The overall picture seems to be one of typical low bottom-of-cycle activity.

2019-02-12

Zones and Distance in CQ WW: 2017

Using the data from the CQ WW public logs, we can examine the distribution of distance for QSOs as a function of zone.

Below is a series of figures showing this distribution integrated over all bands and, separately, band by band for the CQ WW SSB and CQ WW CW contests for 2017.

Each plot shows a colour-coded distribution of the distance of QSOs for each zone, with the data for SSB appearing above the data for CW within each zone.

For every half-QSO in a given zone, the distance of the QSO is calculated; in ths way, the total  number of half-QSOs in bins of width 500 km is accumulated. Once all the QSOs for a particular contest have been binned in this manner, the distribution for each zone is normalised to total 100% and the result coded by colour and plotted. The mean distance for each zone and mode is denoted by a small white rectangle added to the underlying distance distribution.

As usual, only QSOs for which logs have been provided by both parties, and which show no bust of either callsign or zone number are included. Bins coloured black are those for which no QSOs are present at the relevant distance.

The resulting plots are reproduced below. I find that they display in a compact format a wealth of data that is informative and often unexpected.








2019-02-11

Zones and Distance in CQ WW: 2016

Using the data from the CQ WW public logs, we can examine the distribution of distance for QSOs as a function of zone.

Below is a series of figures showing this distribution integrated over all bands and, separately, band by band for the CQ WW SSB and CQ WW CW contests for 2016.

Each plot shows a colour-coded distribution of the distance of QSOs for each zone, with the data for SSB appearing above the data for CW within each zone.

For every half-QSO in a given zone, the distance of the QSO is calculated; in ths way, the total  number of half-QSOs in bins of width 500 km is accumulated. Once all the QSOs for a particular contest have been binned in this manner, the distribution for each zone is normalised to total 100% and the result coded by colour and plotted. The mean distance for each zone and mode is denoted by a small white rectangle added to the underlying distance distribution.

As usual, only QSOs for which logs have been provided by both parties, and which show no bust of either callsign or zone number are included. Bins coloured black are those for which no QSOs are present at the relevant distance.

The resulting plots are reproduced below. I find that they display in a compact format a wealth of data that is informative and often unexpected.








2019-02-06

Revised Augmented Logs including RBN data for CQ WW CW and SSB Contests, 2005 to 2018

Revised augmented versions of the logs for the CQ WW CW and SSB contests are now available for the period 2005 to 2018. These logs for the first time incorporate data from outside the logs: they contain a column based on data from the Reverse Beacon Network (RBN).

Links to the augmented logs may be followed here.

The augmented logs contain the same information as cleaned logs, but with the addition of some useful (derived) information on each line. The information added to each line comprises:
  1. The sequence of four characters that are the same for each entry in a particular log:
    •  a. letter "A" or "U" indicating "assisted" or "unassisted"
    •  b. letter "Q", "L", "H" or "U", indicating respectively QRP, low power, high power or unknown power level
    •  c. letter "S", "M", "C" or "U", indicating respectively a single-operator, multi-operator, checklog or unknown operator category [ the contest organisers have stated that checklogs are not made public, but in fact at least some of them from the early years have been, hence the need for the "C" category ]
    •  d. character "1", "2", "+" or "U", indicating respectively that the number of transmitters is one, two, unlimited or unknown
  2. A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves subsequent processors of the file considerable time to have the number readily available in the file without having to calculate it each time.)
  3. Band
  4. A set of fourteen flags, each -- apart from column k and column n -- encoded as T/F: 
    • a. QSO is confirmed by a log from the second party 
    • b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party) 
    • c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party) 
    • d. the call of the second party is unique 
    • e. QSO appears to be a NIL 
    • f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest 
    • g. QSO appears to be a country mult 
    • h. QSO appears to be a zone mult 
    • i. QSO is a zone bust (i.e., the received zone appears to be a bust)
    • j. QSO is a reverse zone bust (i.e. the second party appears to have bust the zone of the first party)
    • k. This entry has three possible values rather than just T/F:
      • T: QSO appears to be made during a run by the first party
      • F: QSO appears not to be made during a run by the first party
      • U: the run status is unknown because insufficient frequency information is available in the first party's log
    • l. QSO is a dupe
    • m. QSO is a dupe in the second party's log
    • n. RBN information (see below)
  5. If the QSO is a reverse bust, the call logged by the second party; otherwise, the placeholder "-"
  6. If the QSO is an ordinary bust, the correct call that should have been logged by the first party; otherwise, the placeholder "-"
  7. If the QSO is a reverse zone bust, the zone logged by the second party; otherwise, the placeholder "-"
  8.  If the QSO is an ordinary zone bust, the correct zone that should have been logged by the first party; otherwise, the placeholder "-"

RBN Information


In the CW contests from 2009 onwards, the RBN was active, automatically spotting the frequency at which any station calling CQ was transmitting. To reflect possible use of RBN information, the augmented files now include a fourteenth column. For the sake of uniformity, this column is present in all the augmented files, regardless of whether the RBN actually contributed useful information to a particular contest.

Each QSO has one of several characters in the fourteenth column of flags. These characters should be interpreted as follows:

'-'
  No useful RBN-derived information is available for this QSO.

'0'
  The worked station (i.e., the second call on the log line) appears to have begun to CQ on this frequency within (roughly) 60 seconds prior to the QSO.

'A' to 'Z'
  For the nth letter of the alphabet: the worked station appears to have been CQing on this frequency for (roughly) n minutes prior to the QSO.

'+'
  The worked station appears to have been CQing for more than 26 minutes on this frequency.

'<'
  Because the the RBN is distributed, and because each contest entrant station has its own clock, there is generally a skew between the reading of the clock of the station making the QSO and the timestamp from the RBN at which it believes a posting was made (indeed, it's unclear from the RBN's [lack of] documentation exactly how the timestamp on an individual RBN posting is to be interpreted). If the character '<' appears in the the RBN column, it indicates that the raw values of the clocks suggest that the QSO took place up to two minutes before the RBN reported the worked station commencing to CQ at this frequency. When this occurs, the most likely interpretation is that there is non-negligible skew between the two clocks, and the station was actually worked almost as soon as a CQ was posted by the RBN. But it might also mean that the entrant was simply lucky and found the CQing station just as it fired up on a new frequency.

Notes:
  • The encoding of some of the flags requires subjective decisions to be made as to whether the flag should be true or false; consequently, and because CQ has yet to understand the importance of making their scoring code public, the value of a flag for a specific QSO line in some circumstances might not match the value that CQ would assign. (Also, CQ has more data available in the form of check logs, which are generally not made public.)
  • I made no attempt to deduce or infer the run status of a QSO in the second party's log (if such exists), regardless of the status in the first party's log. This allows one cleanly to perform correct statistical analyses anent the number of QSOs made by running stations merely by excluding QSOs marked with a U in column k.
  • No attempt is made to detect the case in which both participants of a QSO bust the other station's call. This is a problematic situation because of the relatively high probability of a false positive unless both stations log the frequency as opposed to the band. (Also, on bands on which split-frequency QSOs are common, the absence of both transmit and receive frequency is a problem.) Because of the likelihood of false positives, it seems better, given the presumed rarity of double-bust QSOs, that no attempt be made to mark them.
  • The entries for the zones in the case of zone or reverse zone busts are normalised to two-digit values.

2019-02-05

Zones and Distance in CQ WW: 2015

Using the data from the CQ WW public logs, we can examine the distribution of distance for QSOs as a function of zone.

Below is a series of figures showing this distribution integrated over all bands and, separately, band by band for the CQ WW SSB and CQ WW CW contests for 2015.

Each plot shows a colour-coded distribution of the distance of QSOs for each zone, with the data for SSB appearing above the data for CW within each zone.

For every half-QSO in a given zone, the distance of the QSO is calculated; in ths way, the total  number of half-QSOs in bins of width 500 km is accumulated. Once all the QSOs for a particular contest have been binned in this manner, the distribution for each zone is normalised to total 100% and the result coded by colour and plotted. The mean distance for each zone and mode is denoted by a small white rectangle added to the underlying distance distribution.

As usual, only QSOs for which logs have been provided by both parties, and which show no bust of either callsign or zone number are included. Bins coloured black are those for which no QSOs are present at the relevant distance.

The resulting plots are reproduced below. I find that they display in a compact format a wealth of data that is informative and often unexpected.








2019-02-04

Zones and Distance in CQ WW: 2014

Using the data from the CQ WW public logs, we can examine the distribution of distance for QSOs as a function of zone.

Below is a series of figures showing this distribution integrated over all bands and, separately, band by band for the CQ WW SSB and CQ WW CW contests for 2014.

Each plot shows a colour-coded distribution of the distance of QSOs for each zone, with the data for SSB appearing above the data for CW within each zone.

For every half-QSO in a given zone, the distance of the QSO is calculated; in ths way, the total  number of half-QSOs in bins of width 500 km is accumulated. Once all the QSOs for a particular contest have been binned in this manner, the distribution for each zone is normalised to total 100% and the result coded by colour and plotted. The mean distance for each zone and mode is denoted by a small white rectangle added to the underlying distance distribution.

As usual, only QSOs for which logs have been provided by both parties, and which show no bust of either callsign or zone number are included. Bins coloured black are those for which no QSOs are present at the relevant distance.

The resulting plots are reproduced below. I find that they display in a compact format a wealth of data that is informative and often unexpected.