2017-01-04

New CQ WW SSB video maps

I have updated the set of CQ WW SSB video maps on my youtube channel (channel N7DR). These video maps cover all the years for which public CQ WW SSB logs are currently available (2005 to 2016).

To access individual videos directly:

2017-01-03

CQ WW SSB 2016 logs available

The logs for the 2016 running of the CQ WW SSB contest are now available from CQ. The logs for the CW contest are not yet available.

The logs are also available in compressed form that may be more easily and quickly downloaded here.

I have created a compressed file that contains all the cleaned QSO lines from the Cabrillo files from all the SSB logs for all the years for which data are available. Currently, the file covers all the SSB QSOs in the years from 2005 to 2016. The MD5 checksum of this file is: ac555db15417fa76466ed636936e0efe.

I have also created an augmented file, in compressed format, that adds useful data to each QSO. Each QSO line in the augmented file includes an additional four columns, with the following meanings:

  1. The letter "A" or "U" indicating "assisted" or "unassisted"
  2. A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
  3. Band
  4. A set of ten flags, each encoded as T/F: 
    • a. QSO is confirmed by a log from the second party 
    • b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party) 
    • c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party) 
    • d. the call of the second party is unique 
    • e. QSO appears to be a NIL 
    • f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest 
    • g. QSO appears to be a country mult 
    • h. QSO appears to be a zone mult 
    • i. the QSO is a zone bust (i.e., the received zone appears to be a bust)
    • j. the QSO is a reverse zone bust (i.e. the second party appears to have bust the zone of the first party)
The MD5 checksum of this file is: 4acae023af4ea41903249f54446dd79a.
  
Note that the flags in the augmented data are calculated from the raw data independently of the CQ contest committee. This is because:
  1. CQ still does not make the actual scoring code available ;
  2. the checklogs are not public, and hence represent additional data that CQ can use in determining the values of the flags.


2017-01-02

2016 RBN data

All the postings to the Reverse Beacon Network in 2016, along with the postings from prior years, are now available in the directory https://www.adrive.com/public/cQwkEB/rbn.

Some simple annual statistics for the period 2009 to 2016 follow (the 2009 numbers cover only part of that year, as the RBN was instantiated partway through that year).

Total posts:
2009:   5,007,040
2010:  25,116,810
2011:  49,705,539
2012:  71,584,195
2013:  92,875,152
2014:  108,862,505
2015:  116,385,762
2016:  111,027,068
 Total posting stations:
2009: 151
2010: 265
2011: 320
2012: 420
2013: 473
2014: 515
2015: 511
2016: 590
 Total posted callsigns:
2009: 143,724
2010: 266,189
2011: 271,133
2012: 308,010
2013: 353,952
2014: 398,293
2015: 433,197
2016: 375,613
Obviously, statistics that are considerably more comprehensive may be derived rather easily from the files in the directory.

Note that if you intend to use the databaseß´s reported signal strengths in an analysis, you should be sure that you understand the ramifications of what the RBN means by SNR.

2016-07-29

CY9C


Reception of the CY9C DXpedition by the RBN. 

First, continent-by-continent:


And now the six zones with the largest number of reception reports:

Notes

I was away for most of this DXpedition, and so missed them on 80m, where they would have been a new one. They were audible in the evening on the last two nights, but the QRN level from thunderstorms was very high. They were copyable and worth calling at sunset, but unfortunately both evenings they were QRX for about 20 minutes just at the time of their peak.

"The Second Book" available in electronic formats




My most recent literary novel, The Second Book, is now available in several electronic formats:
Don't ask me why Apple charges more than the price I set. They seem to do this with all my books.

Most of the above links also allow you to read a sample, typically 10 to 15 percent of the book.

You can, of course, still purchase a print version of the book.

2016-05-07

New augmented logs for CQ WW from 2005 to 2015

I have added two more columns to the flags on each line of the CQ WW augmented logs (the MD5 hash of the file is: 6e8d27f709f13b1851e3bb134592d336).

 The additions to the standard CQ WW Cabrillo QSO line are now (changes in bold):
  1. The letter "A" or "U" indicating "assisted" or "unassisted"
  2. A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
  3. Band
  4. A set of ten flags, each encoded as T/F: 
    • a. QSO is confirmed by a log from the second party 
    • b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party) 
    • c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party) 
    • d. the call of the second party is unique 
    • e. QSO appears to be a NIL 
    • f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest 
    • g. QSO appears to be a country mult 
    • h. QSO appears to be a zone mult 
    • i. the QSO is a zone bust (i.e., the received zone appears to be a bust)
    • j. the QSO is a reverse zone bust (i.e. the second party appears to have bust the zone of the first party)
Note that the encoding of some of the flags requires subjective decisions to be made as to whether the flag should be true or false; consequently, and because CQ has yet to understand the importance of making their scoring code public, the value of a flag for a specific QSO line in some circumstances might not match the value that CQ would assign. (Also, CQ has more data available in the form of check logs, which are not made public.)

2016-04-17

Using Adrive with rsync and SSH keys

Jump directly to solution (lower on this page).

 

My experience setting up an Adrive account to use rsync with SSH keys 


When copy.com announced that they would no longer offer a hosting service that would allow me to make large files publicly available, I looked around for other choices. Eventually, I chose Adrive, partly because they had a good price for the amount of storage I am likely to need for this purpose, and partly because they claim that one can use their storage using ssh-based tools (in particular, rsync), which would make management easy. (I note in passing that for online storage and management of private information, I use and recommend rsync.net.)

On the Adrive login page, it says:
SCP, SFTP and Rsync
Access all your files from anywhere using SCP, SFTP and Rsync including server-side SSH key management.
which certainly sounds promising.

The instructions for using rsync are simple:
  rsync files_to_upload username@rsync.adrive.com:. 
Wonderful, except that that doesn't work. Worse, any attempt to use rsync causes a request for a password to be displayed, despite the site's mention of support for "server-side SSH key management". I tried to deal with the ssh problem first.

They provide apparently detailed instructions as to how to generate a key for use with Adrive and how to upload the key so that it will be used.

The trouble is, the instructions don't result in keys that are used; trying to use rsync still prompts the user for the password.

Having discovered all this, I entered the world of Adrive non-support. The support page tells Premium users:
Please browse through our FAQ to see if your question or problem has already been addressed. 
Well, it didn't take more than a few moments to see that any question of a technical nature is noticeably absent from the FAQ.

The support page continues:
Search the ADrive Forum to find answers within our community, or to post a new question, technical support issue, or concern.
It turns out that there is a forum dedicated to SCP, SFTP and Rsync issues. Of course, actually posting to the forum, even though one is a Premium customer with an Adrive username and password, is, as one is beginning to expect with this company, not as simple as it should be: one has to register anew to obtain permission to post to the form.

Having accomplished reregistration, I posted my ssh question, and waited for a response. And waited. And waited. After three weeks without any reply, I decided that enough was enough, and it was time to file an official support ticket. I did this via the Adrive web form.

Very soon, I received an acknowledgement via e-mail:
Thank you for submitting your support issue to ADrive. Our Support Team has received your submission. We are reviewing your question and will work to resolve it as soon as possible. If you have not already done so, please browse through our FAQ <http://www.adrive.com/info/faq> and our Forum <http://www.adrive.com/forum> to see if your question or issue has already been addressed.
Unfortunately, I received no actual useful response in the days that followed.

After a few days,. I tried again, resubmitting the issue. I received the same acknowledgement, and, as before, no reponse that actually addressed the issue.

I then tried directly e-mailing support@adrive.com. That produced no response at all.

At that point, I finally was forced to the conclusion that Adrive "support" does not actually exist in any meaningful manner, even for paying customers. The claims of support on the web site, are, it seems from my experience, intended to make the unwary believe that there is a professional support team available, when it appears that in fact any message sent to Adrive support merely ends up in /dev/null.

So at that point I decided to spend the time to resolve the issue myself (although, of course, a genuine support team could have given me the solution more or less instantly).


Forcing Adrive to use an SSH key


The solution, or at least the solution that worked here, isn't hard, but it is rather obscure.

After creating a key pair and uploading the public key, create a file called ~/.ssh/config, and insert the following lines:

  Host rsync.adrive.com
    IdentityFile <path to key>


That should force rsync to use the correct key.

The correct form of the rsync command 


There remained two problems on my system: firstly, every time I tried to use rsync, I would receive the message:
X11 forwarding request failed on channel 0; 
and, secondly, the command
  rsync files_to_upload username@rsync.adrive.com:.
didn't actually transfer anything (hardly surprising, since files_to_upload can't possibly mean what it says).

These problems can both be solved by using the following form of the rsync command:
rsync -rvt --delete -e "ssh -x" <directory-to-upload>/ "username@rsync.adrive.com:."
where username is replaced with your Adrive username, including the commat symbol. The ssh -x portion removes the error message about X11. The rest of the command causes rsync recursively to transfer the files in <directory-to-upload> to Adrive. The --delete option causes files that have been removed locally also to be removed on Adrive. (Using -rqt instead of -rvt will cause rsync to operate silently, which may be more useful than -rqt when the command is used in scripts).