CQ WW CW logs for 2021 have been added to this repository of public CQ WW logs.
2022-01-01
2021-12-16
CQ WW SSB logs for 2021
CQ WW SSB logs for 2021 have been added to this repository of public CQ WW logs.
2021-12-02
Debian Bullseye, Samba and WIndows XP
Following an upgrade here from Debian Buster to Bullseye, I discovered that a Windows XP computer on my home network could no longer access a samba share on the upgraded system, even though (or possibly because, although I doubt it) I did not allow the upgrade to change the /etc/samba/smb.conf file. After a bit of messing around, I discovered that the solution is easy, although apparently undocumented as yet, at least officially. I added the line:
server min protocol = NT1
to the [global] portion of the file, and then restarted samba:
systemctl restart smbd.service
From that point on, I found that the XP machine was once more able to reach the samba shares correctly.
I note that the inability to browse Bullseye samba shares from XP machines that previously could browse Buster shares on the same server machine has been reported as Debian bug 948671, with the above workaround now documented in an associated submission by me.
2021-10-12
CQ WPX SSB and CW logs for 2021
CQ WPX SSB and CW logs for 2021 have been added to this repository of public CQ WPX logs.
2021-08-26
Unofficial Station Reports, ARRL DX SSB and CW, 2018 to 2021
Using the public logs, it is rather easy to generate unofficial station-by-station reports for the entrants in the ARRL DX contests.
The ARRL generates official reports and generally makes these reports available individually to each entrant. But these are not made public. The unofficial reports, while not necessarily identical to the official ones, may therefore hold some interest.
The
unofficial reports may differ from the official ones because the
contest committee has access to checklogs, which are not made public.
Also, there are various pathological occurrences in logs that require a
decision to be made as to how to classify one or more QSOs; the rules by
which such decisions are made are not public, so the decisions that I
made when constructing the unofficial reports may well be different from
those made by the ARRL. Nevertheless, pathological logs
(or pathological QSOs within a log) are relatively rare, so these
decisions should affect a relatively small percentage of logs and QSOs.
(Typical examples [there are many more] of circumstances in which
decisions must made be are: by how much may clocks be skewed and a QSO
still be considered valid? what to do if the transmitted callsign
changes for some number of QSOs in the contest? what do to if more than
one entrant claims to have used the same transmitted callsign?)
The complete set of unofficial reports for the CW and SSB versions of the ARRL DX contest for the years 2018 to 2021 may be found in appropriately named files in this directory.
I note that despite explicitly informing me that they would do so (in 2017), the ARRL have never made public the logs that they hold for the ARRL DX contests for years prior to 2018.
One note regarding interpretation of the information in these unofficial reports: all the fields should be self-explanatory, except that in the listing for EXCHANGE BUSTS, some values are enclosed in parentheses: this indicates that the worked station did not submit a log, and the value of the exchange sent by that station was deduced from QSO: lines in the logs of other entrants.
For example, the report for HL2ZN in the 2021 ARRL DX CW contest contains the line (the line below may be wrapped on your display):
QSO: 14000 CW 2021-02-21 2245 HL2ZN 599 0500 N7DR 599 KY [ (CO) ]
This indicates that we can deduce that HL2ZN probably bust N7DR's exchange, even though N7DR did not send in a log: HL2ZN recorded N7DR's state as KY, even though N7DR probably sent CO (indeed, I did send CO).
2021-08-17
Cleaned and Augmented Logs for ARRL DX CW and SSB contests, 2018 to 2021
Raw logs
The raw logs for the ARRL DX CW and SSB contests are now available for 2018, 2019, 2020 and 2021 in this directory.
Cleaned Logs
Cleaned logs for the same years may also be downloaded from the directory. The cleaned logs are combined into a single file; but data for individual stations and years may trivially be extracted from the combined file.
The cleaned logs are the result of processing the QSO: lines from the entrants' submitted Cabrillo files (as [gratuitously] modified by the ARRL) to ensure that all fields contain valid values and all the data match the column-specific standard format for this contest.
Any line containing illegal data in a field has simply been removed. Also, only the QSO: lines are retained, so that each line in the file can be processed easily. All QTH multipliers are rendered as two letters, and the power is rendered as four digits, regardless of how the submitted log recorded these two fields; this should simplify processing the logs by scripts or programs, as should the use of fixed-length records in these cleaned files.
Augmented Logs
Links to the augmented logs for the same years may likewise be downloaded from the directory. The augmented logs are combined into a single file; but data for individual stations and years may trivially be extracted from the combined file.
The augmented logs for the ARRL DX contests contain the same information as the cleaned logs, but with the addition of some useful (derived) information on each line. The information added to each line comprises:
- The sequence of four characters that are the same for each entry in a particular log:
- a. letter "A" or "U" indicating "assisted" or "unassisted"
- b. letter "Q", "L", "H" or "U", indicating respectively QRP, low power, high power or unknown power level
- c. letter "S", "M", "C" or "U", indicating respectively a single-operator, multi-operator, checklog or unknown operator category
- d. character "1", "2", "+" or "U", indicating respectively that the number of transmitters is one, two, unlimited or unknown
- A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves subsequent processors of the file considerable time to have the number readily available in the file without having to calculate it each time.)
- Band
- A set of fourteen flags, each -- apart from column k and column n -- encoded as T/F:
- a. QSO is confirmed by a log from the second party
- b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party)
- c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party)
- d. the call of the second party is unique
- e. QSO appears to be a NIL
- f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest
- g. QSO appears to be a country mult (may be T for W/VE stations only)
- h. QSO appears to be a state/province mult (may be T for DX stations only)
- i. QSO is an exchange bust (i.e., the received exchange appears to be a bust)
- j. QSO is a reverse exchange bust (i.e. the second party appears to have bust the exchange of the first party)
- k. This entry has three possible values rather than just T/F:
- T: QSO appears to be made during a run by the first party
- F: QSO appears not to be made during a run by the first party
- U: the run status is unknown because insufficient frequency information is available in the first party's log
- l. QSO is a dupe
- m. QSO is a dupe in the second party's log
- n. RBN information (see below)
- If the QSO is a reverse bust, the call logged by the second party; otherwise, the placeholder "-"
- If the QSO is an ordinary bust, the correct call that should have been logged by the first party; otherwise, the placeholder "-"
- If the QSO is a reverse exchange bust, the exchange logged by the second party; otherwise, the placeholder "-"
- If the QSO is an ordinary exchange bust, the correct exchange that should have been logged by the first party; otherwise, the placeholder "-"
RBN Information
In CW contests from 2009 onwards, the RBN has been active, automatically spotting the frequency at which any station calling CQ was transmitting. To reflect possible use of RBN information, the augmented files include a fourteenth column. For the sake of uniformity, this column is present in all the augmented files, regardless of whether the RBN actually contributed useful information to a particular contest.
Each QSO has one of several characters in the fourteenth column of flags. These characters should be interpreted as follows:
'-'
No useful RBN-derived information is available for this QSO.
'0'
The worked station (i.e., the second call on the log line) appears to have begun to CQ on this frequency within (roughly) 60 seconds prior to the QSO.
'A' to 'Z'
For the nth letter of the alphabet: the worked station appears to have been CQing on this frequency for (roughly) n minutes prior to the QSO.
'+'
The worked station appears to have been CQing for more than 26 minutes on this frequency.
'<'
Because the the RBN is distributed, and because each contest entrant station has its own clock, there is generally a skew between the reading of the clock of the station making the QSO and the timestamp from the RBN at which it believes a posting was made (indeed, it's unclear from the RBN's [lack of] documentation exactly how the timestamp on an individual RBN posting is to be interpreted). If the character '<' appears in the the RBN column, it indicates that the raw values of the clocks suggest that the QSO took place up to two minutes before the RBN reported the worked station commencing to CQ at this frequency. When this occurs, the most likely interpretation is that there is non-negligible skew between the two clocks, and the station was actually worked almost as soon as a CQ was posted by the RBN. But it might also mean that the entrant was simply lucky and found the CQing station just as it fired up on a new frequency.
Notes:
- The encoding of some of the flags requires subjective decisions to be made as to whether the flag should be true or false; consequently, and because the ARRL has yet to understand the importance of making the scoring code public, the value of a flag for a specific QSO line in some circumstances might not match the value that the ARRL has assigned. (Also, the ARRL has additional, non-public, data available.)
- I made no attempt to deduce or infer the run status of a QSO in the second party's log (if such exists), regardless of the status in the first party's log. This allows one cleanly to perform correct statistical analyses anent the number of QSOs made by running stations merely by excluding QSOs marked with a U in column k.
- No attempt is made to detect the case in which both participants of a QSO bust the other station's call. This is a problematic situation because of the relatively high probability of a false positive unless both stations log the frequency as opposed to the band. (Also, on bands on which split-frequency QSOs are common, the absence of both transmit and receive frequency is a problem.) Because of the likelihood of false positives, it seems better, given the presumed rarity of double-bust QSOs, that no attempt be made to mark them.
- The entries for the exchanges in the case of exchange or reverse exchange busts are normalised to two-letter or four-digit values in the same manner as described above for the exchanges in the cleaned logs.
2021-06-07
Creating Local FCC Database From Version 6 Data
This is a followup to this post, required because the FCC has changed the format of their files from version 5 to version 6.
The contents of the eight FCC files are now (as described in the version 6 document):
Amateur
[AM] -- unchanged from version 5
1 Record Type [AM] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 EBF Number varchar(30)
5 Call Sign char(10)
6 Operator Class char(1)
7 Group Code char(1)
8 Region Code tinyint
9 Trustee Call Sign char(10)
10 Trustee Indicator char(1)
11 Physician Certification char(1)
12 VE Signature char(1)
13 Systematic Call Sign Change char(1)
14 Vanity Call Sign Change char(1)
15 Vanity Relationship char(12)
16 Previous Call Sign char(10)
17 Previous Operator Class char(1)
18 Trustee Name varchar(50)
Comments
[CO] -- unchanged from version 5
1 Record Type [CO] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 Call Sign char(10)
5 Comment Date mm/dd/yyyy
6 Description varchar(255)
7 Status Code char(1)
8 Status Date mm/dd/yyyy
Entity
[EN] -- unchanged from version 5
1 Record Type [EN] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 EBF Number varchar(30)
5 Call Sign char(10)
6 Entity Type char(2)
7 Licensee ID char(9)
8 Entity Name varchar(200)
9 First Name varchar(20)
10 MI char(1)
11 Last Name varchar(20)
12 Suffix char(3)
13 Phone char(10)
14 Fax char(10)
15 Email varchar(50)
16 Street Address varchar(60)
17 City varchar(20)
18 State char(2)
19 Zip Code char(9)
20 PO Box varchar(20)
21 Attention Line varchar(35)
22 SGIN char(3)
23 FCC Registration Number (FRN) char(10)
24 Applicant Type Code char(1)
25 Applicant Type Code Other char(40)
26 Status Code char(1)
27 Status Date mm/dd/yyyy
28 3.7 GHz License Type char(1)
29 Linked Unique System Identifier numeric(9,0)
30 Linked Call Sign char(10)
Application/License Header
[HD]
1 Record Type [HD] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 EBF Number varchar(30)
5 Call Sign char(10)
6 License Status char(1)
7 Radio Service Code char(2)
8 Grant Date mm/dd/yyyy
9 Expired Date mm/dd/yyyy
10 Cancellation Date mm/dd/yyyy
11 Eligibility Rule Num char(10)
12 Reserved char(1)
13 Alien char(1)
14 Alien Government char(1)
15 Alien Corporation char(1)
16 Alien Officer char(1)
17 Alien Control char(1)
18 Revoked char(1)
19 Convicted char(1)
20 Adjudged char(1)
21 Reserved char(1)
22 Common Carrier char(1)
23 Non Common Carrier char(1)
24 Private Comm char(1)
25 Fixed char(1)
26 Mobile char(1)
27 Radiolocation char(1)
28 Satellite char(1)
29 Developmental or STA or Demonstration char(1)
30 InterconnectedService char(1)
31 Certifier First Name varchar(20)
32 Certifier MI char(1)
33 Certifier Last Name varchar(20)
34 Certifier Suffix char(3)
35 Certifier Title char(40)
36 Female char(1)
37 Black or African-American char(1)
38 Native American char(1)
39 Hawaiian char(1)
40 Asian char(1)
41 White char(1)
42 Hispanic char(1)
43 Effective Date mm/dd/yyyy
44 Last Action Date mm/dd/yyyy
45 Auction ID integer
46 Broadcast Services - Regulatory Status char(1)
47 Band Manager - Regulatory Status char(1)
48 Broadcast Services - Type of Radio Service char(1)
49 Alien Ruling char(1)
50 Licensee Name Change char(1)
51 Whitespace Indicator char(1)
52 Operation/Performance Requirement Choice char(1)
53 Operation/Performance Requirement Answer char(1)
54 Discontinuation of Service char(1)
55 Regulatory Compliance char(1)
56 900 MHz Eligibility Certification char(1)
57 900 MHz Transition Plan Certification char(1)
58 900 MHz Return Spectrum Certification char(1)
59 900 MHz Payment Certification char(1)
History
[HS] -- unchanged from version 5
1 Record Type [HS] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 Call Sign char(10)
5 Log Date mm/dd/yyyy
6 Code char(6)
License Attachment
[LA] -- unchanged from version 5
1 Record Type [LA] char(2)
2 Unique System Identifier numeric(9,0)
3 Call Sign char(10)
4 Attachment Code char(1)
5 Attachment Description varchar(60)
6 Attachment Date mm/dd/yyyy
7 Attachment File Name varchar(60)
8 Action Performed char(1)
Special Condition
[SC] -- unchanged from version 5
1 Record Type [SC] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 EBF Number varchar(30)
5 Call Sign char(10)
6 Special Condition Type char(1)
7 Special Condition Code int
8 Status Code char(1)
9 Status Date mm/dd/yyyy
License Free Form Special ConditionThe following extract from the code that creates the output database maps these on a one-to-one basis to internal identifiers (the four new fields in the [HD] records don't seem to be important -- at least for now -- so there is no change in the output as compared to the processing of version 5 files, although the new fields are processed):
Position Data Element Definition
[SF] -- unchanged from version 5
1 Record Type [SF] char(2)
2 Unique System Identifier numeric(9,0)
3 ULS File Number char(14)
4 EBF Number varchar(30)
5 Call Sign char(10)
6 License Free Form Type char(1)
7 Unique License Free Form Identifier numeric(9,0)
8 Sequence Number integer
9 License Free Form Condition varchar(255)
10 Status Code char(1)
11 Status Date mm/dd/yyyy
[AM]
RECORD_TYPE,[CO]
ID,
ULS_NUMBER,
EBF_NUMBER,
CALLSIGN,
OPERATOR_CLASS,
GROUP_CODE,
REGION_CODE,
TRUSTEE_CALLSIGN,
TRUSTEE_INDICATOR,
PHYSICIAN_CERTIFICATION,
VE_SIGNATURE,
SYSTEMATIC_CALLSIGN_CHANGE,
VANITY_CALLSIGN_CHANGE,
VANITY_RELATIONSHIP,
PREVIOUS_CALLSIGN,
PREVIOUS_OPERATOR_CLASS,
TRUSTEE_NAME
RECORD_TYPE,[EN]
ID,
ULS_NUMBER,
CALLSIGN,
COMMENT_DATE,
DESCRIPTION,
STATUS_CODE,
STATUS_DATE
RECORD_TYPE,[HD]
ID,
ULS_NUMBER,
EBF_NUMBER,
CALLSIGN,
ENTITY_TYPE,
LICENSE_ID,
ENTITY_NAME,
FIRST_NAME,
MIDDLE_INITIAL,
LAST_NAME,
SUFFIX,
PHONE,
FAX,
EMAIL,
STREET_ADDRESS,
CITY,
STATE,
ZIP_CODE,
PO_BOX,
ATTENTION_LINE,
SGIN,
FRN,
APPLICANT_TYPE_CODE,
APPLICANT_TYPE_CODE_OTHER,
STATUS_CODE,
LICENSE_TYPE_37,
LINKED_ID,
LINKED_CALLSIGN,
RECORD_TYPE,
ID,
ULS_NUMBER,
EBF_NUMBER,
CALLSIGN,
LICENSE_STATUS,
RADIO_SERVICE_CODE,
GRANT_DATE,
EXPIRED_DATE,
CANCELLATION_DATE,
ELIGIBILITY_RULE_NUM,
RESERVED_1,
ALIEN,
ALIEN_GOVERNMENT,
ALIEN_CORPORATION,
ALIEN_OFFICER,
ALIEN_CONTROL,
REVOKED,
CONVICTED,
ADJUDGED,
RESERVED_2,
COMMON_CARRIER,
NON_COMMON_CARRIER,
PRIVATE_COMM,
FIXED,
MOBILE,
RADIOLOCATION,
SATELLITE,
DEVELOPMENTAL_STA_DEMONSTRATION,
INTERCONNECTED_SERVICE,
CERTIFIER_FIRST_NAME,
CERTIFIER_MIDDLE_INITIAL,
CERTIFIER_LAST_NAME,
CERTIFIER_SUFFIX,
CERTIFIER_TITLE,
FEMALE,
BLACK_AFRICAN_AMERICAN,
NATIVE_AMERICAN,
HAWAIIAN,
ASIAN,
WHITE,
HISPANIC,
EFFECTIVE_DATE,
LAST_ACTION_DATE,
AUCTION_ID,
BROADCAST_SERVICES_REGULATORY_STATUS,
BAND_MANAGER_REGULATORY_STATUS,
BROADCAST_SERVICES_SERVICE_TYPE,
ALIEN_RULING,
LICENSEE_NAME_CHANGE,
WHITESPACE_INDICATOR,
REQUIREMENT_CHOICE,
REQUIREMENT_ANSWER,
DISCONTINUED_SERVICE,
REGULATORY_COMPLIANCE,
ELIGIBILITY_900_MHZ,
TRANSITION_PLAN_900_MHZ,
RETURN_SPRCTRUM_900_MHZ,
PAYMENT_900_MHZ
RECORD_TYPE,[LA]
ID,
ULS_NUMBER,
CALLSIGN,
LOG_DATE,
CODE
RECORD_TYPE,[SC]
ID,
CALLSIGN,
ATTACHMENT_CODE,
ATTACHMENT_DESCRIPTION,
ATTACHMENT_DATE,
ATTACHMENT_FILENAME,
ACTION_PERFORMED
RECORD_TYPE,[SF]
ID,
ULS_NUMBER,
EBF_NUMBER,
CALLSIGN,
SPECIAL_CONDITION_TYPE,
SPECIAL_CONDITION_CODE,
STATUS_CODE,
STATUS_DATE
RECORD_TYPE,
ID,
ULS_NUMBER,
EBF_NUMBER,
CALLSIGN,
LICENSE_FREEFORM_TYPE,
UNIQUE_LICENSE_FREEFORM_ID,
SEQUENCE_NUMBER,
LICENSE_FREEFORM_CONDITION,
STATUS_CODE,
STATUS_DATE
The 50 output fields selected from the above lists are (arranged in groups of ten for easy counting):
ID,The contents of these fields are based on the original equivalent entries in the original data files. The entries for the fields are subject to the following transformations before being written to the output file:
CALLSIGN,
OPERATOR_CLASS,
GROUP_CODE,
REGION_CODE,
TRUSTEE_CALLSIGN,
TRUSTEE_INDICATOR,
SYSTEMATIC_CALLSIGN_CHANGE,
VANITY_CALLSIGN_CHANGE,
VANITY_RELATIONSHIP,
PREVIOUS_CALLSIGN,
PREVIOUS_OPERATOR_CLASS,
TRUSTEE_NAME,
COMMENT_DATE,
DESCRIPTION,
CO_STATUS_CODE, (i.e., STATUS_CODE from [CO])
CO_STATUS_DATE, (i.e., STATUS_DATE from [CO])
ENTITY_NAME,
FIRST_NAME,
MIDDLE_INITIAL,
LAST_NAME,
SUFFIX,
PHONE,
FAX,
EMAIL,
STREET_ADDRESS,
CITY,
STATE,
ZIP_CODE,
PO_BOX,
ATTENTION_LINE,
FRN,
APPLICANT_TYPE_CODE,
APPLICANT_TYPE_CODE_OTHER,
EN_STATUS_CODE, (i.e., STATUS_CODE from [EN])
EN_STATUS_DATE, (i.e., STATUS_DATE from [EN])
LICENSE_STATUS,
RADIO_SERVICE_CODE,
GRANT_DATE,
EXPIRED_DATE,
CANCELLATION_DATE,
ELIGIBILITY_RULE_NUM,
REVOKED,
CONVICTED,
ADJUDGED,
EFFECTIVE_DATE,
LAST_ACTION_DATE,
LICENSEE_NAME_CHANGE,
LINKED_ID,
LINKED_CALLSIGN
- The entry is converted to upper case;
- Any line feeds (yes, the FCC allows line feeds within a field) are converted to the four-character sequence: <LF>;
- Leading and trailing spaces are removed;
- If the field is a date, it is converted from FCC format (mm/dd/yyyy) to ISO 8601 extended format: YYYY-MM-DD.
The full source code to generate the output file may be downloaded here.
To create the binary from the source code, go to the directory that contains the makefile and type:
make fcc-dbThis should generate the executable program as: bin/fcc-db. The program may be executed from within the bin directory as:
fcc-db [directory]where [directory] is the name of the directory that contains the input FCC AM.dat, CO.dat, EN.dat and HD.dat files. Those files should be processed and the output written to stdout.
For what it's worth, it takes somewhat less than 15 seconds for the program to execute to completion on my desktop computer if stdout is redirected to an output file.