2020-01-18

Video Maps of CQ WW CW QSOs, 2005 to 2019

I have updated the set of CQ WW video maps on my youtube channel (channel N7DR) to include the logs from the 2019 running of CQ WW CW. These video maps cover all the years for which public logs are currently available (2005 to 2019).

To access individual videos directly:

2020-01-16

HF Beacons and the Reverse Beacon Network, 2019

Below is  a table of the twenty fixed-frequency stations most often posted by the RBN in 2019.

Position Station Frequency (kHz) Number of Posts
1 YU7QF 14017 108,006
2 W6WX 14100 68,976
3 I1MMR 7026 68,942
4 YV5B 14100 67,984
5 AA1K 1821 67,707
6 CS3B 14100 67,537
7 4U1UN 14100 66,934
8 EW7LO 7008 66,578
9 HA8RM 7009 47,940
10 DO1EK 3541 46,295
11 DK4AN 3569 44,395
12 DL2IAD 7024 39,752
13 UA3KW 14007 39,633
14 DK5JPL 3541 39,062
15 4X6TU 14100 37,580
16 SQ6JAN 3565 36,354
17 OH2B 14100 35,678
18 F4DIA 7010 34,904
19 HA8RM 3509 34,328
20 N4BP 14022 34,041

Notes:
  1. Frequencies are rounded to the nearest kHz;
  2. I am unsure how the U.S. stations in the list can be legal, since the FCC's regulations appear to limit [unattended] HF beacons to a portion of 10m.
  3. It is my memory that the original HF beacons were all located on 28 MHz, so that listeners could be made aware of an opening. It is noticeable that not a single one of the stations on the list above is on 10m: the vast majority are on bands that can reasonably be expected to support some kind of non-local propagation at almost all times (which is probably the very reason that they are posted by the RBN so often -- but one does wonder what the putative purpose of such a beacon is);
Below are figures showing, for each of the stations in the table above, the signal strength as reported by the ten RBN stations that most frequently posted each individual beacon station.

In the following figures:
  1. The ordinate for each of the strip charts ranges between 0 dB and the value shown as FSD (i.e., full scale deflection) near the bottom right-hand corner; in this case, the maximum value of each strip is therefore 80 dB.
  2. The value plotted in this manner is the value denoted SNR by the RBN.  Remember that the RBN has an odd definition of SNR.
  3. The abscissa is divided into a number of bins of equal duration. On each plot there are 100 such bins; because the duration covered by each plot is one year, each bin therefore covers about 3½ days.
  4. At the bottom of each strip chart is a coloured bar. Each bin in these bars is coloured so as to represent the total number of times that the RBN station spotted the beacon in the period covered by the bin.The colour legend for each figure is to the right of the figure.
  5. For the period covered by each 3½-day bin, the lower quartile of SNR readings is coloured grey, the upper quartile is coloured white, and the middle two quartiles are coloured blue.
  6. The vertical order of the various RBN stations is determined solely by the chronological order in which each station first spotted the beacon.





















2020-01-15

Summary File for RBN data, 2009 to 2019

The complete set of RBN data for 2009 to the end of 2019, after uncompression, is some 80GB in size. As not all analyses need the complete dataset, I have constructed a summary file (rbn-summary-data.xz) that contains an overview of the data and which is sufficient for many kinds of analysis that do not depend on the details of individual posts to the RBN. (The basic script used to generate this summary file may be found here; the actual summary file is created by running the basic script for each individual year from 2009 to 2019 and concatenating the results after removing the header line from all except the first year.)

The summary file, after being uncompressed, comprises a single large table of values separated by white space. The name of each column (there are twelve columns in all) is on the first row. The columns are:
  1. band: a string that identifies the band pertaining to this row. Typical values are "15m" or "160m"; if a row contains data that are not distinguished by band, then the characters "NA" are used.
  2. mode: a string that identifies the mode pertaining to this row. Typical values are "CW" or "RTTY"; if a row contains data that are not distinguished by mode, then the characters "NA" are used.
  3. type: a single character that identifies whether the data on this row are for a period of a year ("A"), a month ("M") or a day ("D").
  4. year: the numeric four-digit value of the year to which the current row pertains.
  5. month: the numeric value of the month (January = 1, etc.) of the data in this row. If the data are of type A or D, then this element has the value "NA".
  6. doy: the numeric value of the day number of the year (January 1st = 1, etc.). The maximum value in each year is 366 (even if the year is not a leap year). In the event that the year is not a leap year, the data in columns 7, 8 and 9 will be set to 0 when doy is 366. If the data are of type A or M, then this element has the value "NA".
  7. posts: the total number of posts recorded by the RBN for the band, mode and period identified by the first six columns. 
  8. calls: the total number of distinguishable calls recorded by the RBN for the band, mode and period identified by the first six columns. 
  9. posters: the total number of distinguishable posters recorded by the RBN for the band, mode and period identified by the first six columns. 
  10. scatter: the value of a scatter metric that characterises the geography of the RBN for the band, mode and period identified by the first six columns. The scatter metric is the sum of all possible distance pairs of good posters (measure in km), divided by the number of distance pairs.
  11. good posters: the total number of distinguishable posters recorded by the RBN for the band, mode and period identified by the first six columns, and for which location data are available from the RBN.
  12.  grid metric: the total number of G(15, 100) grid cells that contain good posters.
For example, the first two lines of the summary file are (presented here as a table, in order to make it easier to view on more devices):

band NA
mode NA
type A
year 2009
month NA
doy NA
posts 5007040
calls 143724
posters 151
scatter 5541
good_posters 150
grid_metric 22

This tells us that the first line of actual data in the file comprises annual data for the year 2009, with no separation by band or mode. In 2009, we see that there were 50,007,040 posts of 143,724 callsigns by 151 posters; the scatter metric, which is a measure of the geographic dispersion of the posters on the RBN., was 5,541; 150 different posters contributed the data, spread across 22 distinct G(15, 100) grid cells.

The summary file allows rather rapid analysis of the RBN overview statistics. For example, a plot of the daily number of posts covering the period from the inception of the RBN to the end of 2019 -- 

 -- can be generated on an ordinary desktop PC in about five seconds. From this plot, for example, we can immediately see that the largest number of daily posts occurred during the 2019 running of the CQ WW CW contest in late November (the second-highest cluster of peaks is for the CQ WPX contest, and the third is for the ARRL DX CW contest); also, the burst of activity that coincides with weekends is unmistakable.

2020-01-12

2020-01-11

Video Maps of CQ WW SSB QSOs, 2005 to 2019

I have updated the set of CQ WW video maps on my youtube channel (channel N7DR) to include the logs from the 2019 running of CQ WW SSB. These video maps cover all the years for which public logs are currently available (2005 to 2019).

To access individual videos directly:

2020-01-06

Reverse Beacon Network Actvity: 2009-2019

I here show various plots of the G(15, 100) grid-based scatter metric, G(15, 100), for the Reverse Beacon Network (RBN), using data from the inception of the RBN up to the end of 2019.

As in the past I note that a reasonable a priori case can be made on the basis of propagation characteristics that somewhat different metrics in the G(Δ, n) series might be better representations of RBN coverage on some of the bands. However, rather than make this into a full-scale research project, I shall here simply continue to use the G(15, 100) metric on the basis that it seems "good enough" on all bands.

RBN Posting Stations as a Function of Time


We begin by looking simply at how the number of per-band posters to the RBN has varied since the RBN's inception. (NB Throughout this post, we ignore posters for which the location is not recorded by the RBN; plots for which the abscissa is time show one datum per month.)

First, a plot of the total number of posters as a function of time:


This can be more compactly represented, along with similar per-band data for 160m through 10m (excluding 60m):


G(15, 100) as a Function of Time


Turning now to the geographical distribution of the posting stations, we can display the mensal values of G(15, 100) in a similar manner:



These figures seem to make rather clearly the rather depressing point that since early 2017 there has been no substantive or sustained increase in either the number or geographical distribution of the stations posting to the RBN.

G(15, 100) as a Function of the Number of Posters


Finally, we can combine the mensal values of G(15, 100) and the number of posters. Firstly, including all bands:

 
The summary plot for these data is slightly different, as the ordinate is multi-valued for some values of the abscissa. So, in this summary plot, we take the mean value of G(15, 100) in bins of width equivalent to ten posters, and plot rectangles in the equivalent colours:


All in all, a rather unhappy picture emerges, in which the RBN, after expanding and increasing coverage rather nicely for the better part of a decade, became essentially static in early 2017 and has effectively failed to expand numerically or in geographical coverage since then.

2020-01-03

2019 RBN data

All the postings to the Reverse Beacon Network in 2019, along with the postings from prior years, are now available in this directory.

Some simple annual statistics for the period 2009 to 2019 follow (the 2009 numbers cover only part of that year, as the RBN was instantiated partway through that year).

Total posts:
2009:   5,007,040
2010:  25,116,810
2011:  49,705,539
2012:  71,584,195
2013:  92,875,152
2014:  108,862,505
2015:  116,385,762
2016:  111,027,068
2017:  117,973,111
2018:  131,930,432
2019:  135,558,461
  Total posting stations:
2009: 151
2010: 265
2011: 320
2012: 420
2013: 473
2014: 515
2015: 511
2016: 590
2017: 625
2018: 550
2019: 583
 Total posted distinct callsigns:
2009: 143,724
2010: 266,189
2011: 271,133
2012: 308,010
2013: 353,952
2014: 398,293
2015: 433,197
2016: 375,613
2017: 356,461
2018: 361,058
2019: 337,246
Obviously, statistics that are considerably more comprehensive may be derived rather easily from the files in the directory.

Note that if you intend to use the databaseß´s reported signal strengths in an analysis, you should be sure that you understand the ramifications of what the RBN means by SNR.