2017-06-26

New Augmented Logs for CQ WW, 2005 to 2016

Now available are new augmented versions of the public logs for CQ WW CW and SSB for the period 2005 to 2016.

The cleaned logs are the result of processing the QSO: lines from the entrants' submitted Cabrillo files to ensure that all fields contain valid values and all the data match the format required in the rules. Any line containing illegal data in a field (for example, a zone number greater than 40, or a date/time stamp that is outside the contest period) has simply been removed. Also, only the QSO: lines are retained, so that each line in the file can be processed easily. The MD5 checksum for the file of cleaned logs is: 1b47059d1f2431b55d89a5eb954a05cc.

The augmented logs contain the same information as the cleaned logs, with the addition of some useful information on each line. The MD5 checksum for the compressed (~800 MB) file of augmented logs is: 1e981765c9c1d8edb76fb18d2cd32460. The information added to each line now includes two new fields: the callsign copied by the second party in the case that the second party bust the cull of the first party; amd the correct callsign of the second party in the case that the first party bust the second party's call.

In all, the addition fields in the augmented file comprise:
  1. The letter "A" or "U" indicating "assisted" or "unassisted"
  2. A four-digit number representing the time if the contact in minutes measured from the start of the contest. (I realise that this can be calculated from the other information on the line, but it saves a lot of time to have the number readily available in the file without having to calculate it each time.)
  3. Band
  4. A set of eleven flags, each -- apart from column k -- encoded as T/F: 
    • a. QSO is confirmed by a log from the second party 
    • b. QSO is a reverse bust (i.e., the second party appears to have bust the call of the first party) 
    • c. QSO is an ordinary bust (i.e., the first party appears to have bust the call of the second party) 
    • d. the call of the second party is unique 
    • e. QSO appears to be a NIL 
    • f. QSO is with a station that did not send in a log, but who did make 20 or more QSOs in the contest 
    • g. QSO appears to be a country mult 
    • h. QSO appears to be a zone mult 
    • i. QSO is a zone bust (i.e., the received zone appears to be a bust)
    • j. QSO is a reverse zone bust (i.e. the second party appears to have bust the zone of the first party)
    • k. This entry has three possible values rather than just T/F:
      • T: QSO appears to be made during a run by the first party
      • F: QSO appears not to be made during a run by the first party
      • U: the run status is unknown because insufficient frequency information is available in the first party's log 
  5. If the QSO is a reverse bust, the call logged by the second party; otherwise, the placeholder "-"
  6. If the QSO is an ordinary bust, the correct call that should have been logged by the first party; otherwise, the placeholder "-"
Notes:
  • The encoding of some of the flags requires subjective decisions to be made as to whether the flag should be true or false; consequently, and because CQ has yet to understand the importance of making their scoring code public, the value of a flag for a specific QSO line in some circumstances might not match the value that CQ would assign. (Also, CQ has more data available in the form of check logs, which are not made public.)
  • I made no attempt to deduce the run status of a QSO in the second party's log (if such exists), regardless of the status in the first party's log. This allows one cleanly to perform correct statistical analyses anent the number of QSOs made by running stations merely by excluding QSOs marked with a U in column k.
  • No attempt is made to detect the case in which both participants of a QSO bust the other station's call. This is a problematic situation because of the relatively high probability of a false positive unless both stations log the frequency as opposed to the band. (Also, on bands on which split-frequency QSOs are common, the absence of both transmit and receive frequency is a problem.) Because of the likelihood of false positives, it seems better, given the presumed rarity of double-bust QSOs, that no attempt be made to mark them.

2017-06-23

Call Busts and Reverse Busts in CQ WW, 2007 to 2016

These tables show the stations with the most busts and reverse busts in CQ WW (both CW and SSB) over the period from 2006 to 2015. These tables are based on the augmented versions of the CQ WW public logs.

2007 to 2016 SSB -- Most Busts
Position Call QSOs Busts % Busts
1 OT5A 103,392 1,379 1.3
2 LZ9W 111,781 1,316 1.2
3 PJ2T 122,169 1,202 1.0
4 A73A 77,486 1,103 1.4
5 CN3A 117,724 1,069 0.9
6 HG1S 69,495 1,038 1.5
7 LY7A 74,808 842 1.1
8 JA7YRR 43,287798 1.8
9 PT2CM 37,005 766 2.1
10 ZW5B 49,388 757 1.5


2007 to 2016 SSB -- Most Reverse Busts
Position Call QSOs Reverse Busts % Reverse Busts
1 DF0HQ 116,808 2,264 1.9
2 JA3YBK 63,251 1,297 2.1
3 WE3C 64,056 1,158 1.8
4 K3LR 111,602 1,054 0.9
5 HG1S 69,495 1,023 1.5
6 S52ZW 46,405 959 2.1
7 GM2T 47,757 944 2.0
8 CN2R 76,763789 1.0
9 HK1NA 67,593 788 1.2
10 CN3A 117,724 769 0.7


2007 to 2016 SSB -- Highest Percentage of Busts (≥500 QSOs)
Position Call QSOs % Busts
1 K2JMY 2,129 12.6
2 PV8ADI 1,157 12.2
3 EA7JQT 650 10.9
4 OH1FJ 523 10.9
5 YO7LYM 666 9.5
6 UY3CC 1,008 9.3
7 PY2QM 519 8.7
8 K8TS 8558.7
9 HA6VA 501 8.6
10 PU2TRX 1,151 7.6


2007 to 2016 SSB -- Highest Percentage of Reverse Busts (≥500 QSOs)
Position Call QSOs % Reverse Busts
1 ON46CQ 566 81.8
2 TA2YA 508 26.8
3 CW90A 1,815 23.4
4 TA2IK 558 16.8
5 KH6SP/W5 532 13.0
6 BA8AG 1,011 12.2
7 HE70FG 1,597 9.1
8 BI8FZA 7968.5
9 G2B 2,274 8.0
10 BW2/KU1CW 1,445 8.0

In tables of reverse busts, one sometimes finds what seems like an unreasonable number of reverse busts (as, in this table, for ON46CQ). This is generally caused by a discrepancy between the call actually sent by the listed station and the one recorded as being sent in at least some QSOs in the station's log.

2007 to 2016 CW -- Most Busts
Position Call QSOs Busts % Busts
1 PJ2T 126,638 1,213 1.0
2 PV8ADI 10,710 1,181 11.0
3 LZ9W 116,593 1,074 0.9
4 HG1S 55,298 1,020 1.8
5 D4C 108,787 930 0.9
6 PJ4A 107,384 863 0.8
7 PI4CC 50,599 861 1.7
8 LY7A 71,986790 1.1
9 NR4M 66,448 748 1.1
10 LX7I 87,480 724 0.8


2007 to 2016 CW -- Most Reverse Busts
Position Call QSOs Reverse Busts % Reverse Busts
1 DF0HQ 107,871 2,929 2.7
2 JS3CTQ 25,148 2,710 10.8
3 DR1A 79,239 1,528 1.9
4 K3LR 96,138 1,375 1.4
5 IR4X 55,382 1,375 2.5
6 W2FU 71,886 1,369 1.9
7 W0AIH 38,767 1,219 3.1
8 ES9C 49,6501,208 2.4
9 V26K 63,344 1,143 1.8
10 4O3A 67,238 1,121 1.7


2007 to 2016 CW -- Highest Percentage of Busts (≥500 QSOs)
Position Call QSOs % Busts
1 AD7XG 787 19.6
2 EA2GC 507 18.1
3 4Z5FW 663 16.1
4 AE3D 948 16.0
5 YO7LYM 1,423 15.7
6 OH2MO 554 15.5
7 JA3AHY 597 15.4
8 YU1NIM 66614.7
9 BD3MV 1,089 14.5
10 EA3AJP 542 14.4


2007 to 2016 CW -- Highest Percentage of Reverse Busts (≥500 QSOs)
Position Call QSOs % Reverse Busts
1 G3RWF 1,103 85.4
2 HA8FW 1,256 71.7
3 RZ3VO 2,025 44.2
4 YT65A 1,394 30.6
5 OG55W 2,958 27.9
6 5K0A 2,882 20.6
7 DP65HSC 574 15.2
8 SB0A 1,21514.1
9 YP0HQ 2,223 11.2
10 JS3CTQ 25,148 10.8



2017-06-16

Call Busts and Reverse Busts in CQ WW, 2006 to 2015

These tables show the stations with the most busts and reverse busts in CQ WW (both CW and SSB) over the period from 2006 to 2015. These tables are based on the augmented versions of the CQ WW public logs.

2006 to 2015 SSB -- Most Busts
Position Call QSOs Busts % Busts
1 OT5A 97,034 1,288 1.3
2 PJ2T 118,501 1,158 1.0
3 LZ9W 101,023 1,136 1.1
4 A73A 68,730 1,002 1.5
5 HG1S 69,607 1,000 1.4
6 CN3A 112,280 992 0.9
7 LY7A 81,094 918 1.1
8 ZW5B 53,742821 1.5
9 JA7YRR 45,139 804 1.8
10 RT6A 57,189 764 1.3


2006 to 2015 SSB -- Most Reverse Busts
Position Call QSOs Reverse Busts % Reverse Busts
1 DF0HQ 118,568 2,269 1.9
2 JA3YBK 63,760 1,271 2.0
3 WE3C 60,343 1,129 1.9
4 K3LR 112,067 1,041 0.9
5 HG1S 69,607 1,028 1.5
6 S52ZW 48,525 997 2.1
7 GM2T 50,104 981 2.0
8 HK1NA 67,593788 1.2
9 JA5FDJ 35,747 749 2.1
10 NQ4I 57,892 742 1.3


2006 to 2015 SSB -- Highest Percentage of Busts (≥500 QSOs)
Position Call QSOs % Busts
1 K2JMY 1,943 12.8
2 PV8ADI 816 12.7
3 EA7JQT 650 10.9
4 OH1FJ 523 10.9
5 UY3CC 698 9.9
6 YO7LYM 587 9.9
7 UA3PI 562 8.7
8 PY2QM 5198.7
9 K8TS 855 8.7
10 HA6VA 501 8.6


2006 to 2015 SSB -- Highest Percentage of Reverse Busts (≥500 QSOs)
Position Call QSOs % Reverse Busts
1 ON46CQ 566 81.8
2 TA2YA 508 26.8
3 CW90A 1,815 23.4
4 TA2IK 558 16.8
5 KH6SP/W5 532 13.0
6 BA8AG 1,011 12.2
7 HE70FG 1,597 9.1
8 YB6DE 6729.1
9 PU2UAF 959 9.1
10 BW2/KU1CW 1,445 8.0

In tables of reverse busts, one sometimes finds what seems like an unreasonable number of reverse busts (as, in this table, for ON46CQ). This is generally caused by a discrepancy between the call actually sent by the listed station and the one recorded as being sent in at least some QSOs in the station's log.

2006 to 2015 CW -- Most Busts
Position Call QSOs Busts % Busts
1 PJ2T 120,427 1,162 1.0
2 HG1S 55,427 1.019 1.8
3 PV8ADI 8,653 964 11.1
4 LZ9W 110,650 954 0.9
5 LY7A 78,797 862 1.1
6 D4C 103,842 832 0.8
7 PJ4A 105,271 798 0.8
8 NR4M 64,608743 1.2
9 PI4CC 42,863 742 1.7
10 LX7I 86,643 707 0.8


2006 to 2015 CW -- Most Reverse Busts
Position Call QSOs Reverse Busts % Reverse Busts
1 DF0HQ 108,986 2,844 2.6
2 JS3CTQ 23,970 2,499 10.4
3 DR1A 88,958 1,692 1.9
4 IR4X 63,518 1,525 2.4
5 K3LR 97,236 1,336 1.4
6 W2FU 72,251 1,316 1.8
7 W0AIH 39,060 1,282 3.3
8 V26K 61,8601,126 1.8
9 4O3A 62,917 1,063 1.7
10 NR4M 64,608 1,059 1.6


2006 to 2015 CW -- Highest Percentage of Busts (≥500 QSOs)
Position Call QSOs % Busts
1 AD7XG 594 21.7
2 EA2GC 507 18.1
3 YO7LYM 1,189 17.5
4 AD4TR 738 17.2
5 AE3D 948 16.0
6 WF5W 762 15.7
7 EA2NA 1,306 15.5
8 OH2MO 55415.5
9 JA3AHY 597 15.4
10 BD3MV 1,089 14.5


2006 to 2015 CW -- Highest Percentage of Reverse Busts (≥500 QSOs)
Position Call QSOs % Reverse Busts
1 G3RWF 1,103 85.4
2 HA8FW 1,256 71.7
3 RZ3VO 2,025 44.2
4 YT65A 1,394 30.6
5 OG55W 1,935 29.0
6 HA505BA 660 20.8
7 5K0A 2,882 20.6
8 YO7HHI 80418.3
9 SB0A 1,215 14.1
10 YP0HQ 2,223 11.2



2017-06-12

More Rationalisation of the LANL GPS Charged-Particle Dataset

Earlier posts in this series:
We continue to rationalise the data by removing fields that are unused or provide no useful information, and marking invalid values.

Stage 16: remove records with invalid values of Rad_Re


The documentation for Rad_Re (for both the BDD instrument and the CXD instrument) is:

Column Variable name type Dim. description
4 Rad_Re double 1 (radius of satellite)/Rearth

By definition, any value less than unity for this field must be an error (actually, any value less than about four is an error, but it turns out that are no records with values between one and four). We therefore remove records with invalid values for this field:

  for file in ns*
  do
    awk '$4 > 1 {print $0}' $file > ../gps-stage-16/$file
  done

At this point, the number of records in the dataset for each satellite is:

Satellite Stage 16 Records
ns41 1,990,340
ns48 1,105,300
ns53 1,331,017
ns54 1,938,603
ns55 1,054,569
ns56 1,680,369
ns57 1,082,028
ns58 1,174,629
ns59 1,515,843
ns60 1,495,007
ns61 1,469,803
ns62 774,976
ns63 651,078
ns64 343,643
ns65 480,017
ns66 446,139
ns67 327,174
ns68 304,964
ns69 262,021
ns70 110,260
ns71 220,694
ns72 181,519
ns73 145,292

Stage 17: remove proton_flux_fit from the CXD data


The documentation for the CXD field proton_flux_fit is:


Column Variable name type Dim. description
97-127 proton_flux_fit double 31 intended to be proton flux at 31 energies, not filled currently

As the documentation indicates that this field is not filled with useful data, we remove it:

  for file in ns[567]*
  do 
    awk \ '{$97=$98=$99=$100=$101=$102=$103=$104=$105=$106=$107=$108=$109=$110=$111=$112=$113=$114=$115=$116=$117=$118=$119=$120=$121=$122=$123=$124=$125=$126=$127=""; print $0}' \
  $file | tr -s " " | sed 's/ $//' > ../gps-stage-17/$file
  done

The data table for the CXD satellites (ns53 to ns73) now looks like this:

Column Variable name type Dim. description
1 decimal_day double 1 GPS time, a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years.
2 Geographic_Latitude double 1 Latitude of satellite (deg)
3 Geographic_Longitude double 1 Longitude of satellite (deg)
4 Rad_Re double 1 (radius of satellite)/Rearth
5-15 rate_electron_measured double 11 Measured rate (Hz) in each of the 11 CXD electron channels
16-20 rate_proton_measured double 5 Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21 LEP_thresh double 1 LEP threshold in E1 channels (0 means low, 1 means high)
22 collection_interval int 1 dosimeter collection period (seconds)
23 year int 1 year (e.g. 2015)
24 decimal_year double 1 decimal year = year + (decimal_day-1.0)/(days in year)
25 SVN_number int 1 SVN number of satellite
26 b_coord_radius double 1 radius from earth's dipole axis (earth radii)
27 b_coord_height double 1 height above the earth's dipole equatorial plane (earth radii)
28 magnetic_longitude double 1 Magnetic longitude (degrees)
29 L_shell double 1 L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30 L_LGM_TS04IGRF double 1 LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31 L_LGM_OP77IGRF double 1 LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32 L_LGM_T89CDIP double 1 LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33 bfield_ratio double 1 Bsatellite/Bequator
34 local_time double 1 magnetic local time (0-24 hours)
35 utc_lgm double 1 UTC (0-24 hours)
36 b_sattelite double 1 B field at satellite (gauss)
37 b_equator double 1 B field at equator (on this field line I think) (gauss)
38-48 electron_background double 11 estimated background in electron channels E1-E11 (Hz)
49-53 proton_background double 5 estimated background in proton channels P1-P5 (Hz)
54 proton_activity int 1 =1 if there is significant proton activity
55 proton_temperature_fit double 1 characteristic momentum -- R0 in the expression given above (MeV/c)
56 proton_density_fit double 1 N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
57 electron_temperature_fit double 1 electron temperature from a one Maxwellian fit (MeV)
58 electron_density_fit double 1 electron number density from a one Maxwellian fit (cm-3)
59-69 model_counts_electron_fit_pf double 11 E1-E11 rates due to proton background based on proton flux fit -- currently not filled (all -1's)
70-74 model_counts_proton_fit_pf double 5 P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
75-85 model_counts_electron_fit double 11 E1-E11 rates from the 9-parameter electron flux model
86-90 model_counts_proton_fit double 5 P1-P5 rates from electron background -- currently not filled (all -1's)
91-96 proton_integrated_flux_fit double 6 integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
97-126 integral_flux_instrument double 30 (based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
127-156 integral_flux_energy double 30 energies for the integral of integral_flux_instrument (MeV)
157-171 electron_diff_flux_energy double 15 energies for the fluxes in electron_diff_flux_energy (MeV)
172-186 electron_diff_flux double 15 (based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
187-195 Efitpars double 9 fit parameters for 9 parameter electron fit


Stage 18: remove model_counts_proton_fit from the CXD data


The documentation for the CXD field model_counts_proton_fit is:

Column Variable name type Dim. description
86-90 model_counts_proton_fit double 5 P1-P5 rates from electron background -- currently not filled (all -1's)

As the documentation indicates that this field is not filled with useful data, we remove it:

  for file in ns[567]*
  do 
    awk '{$86=$87=$88=$89=$90=""; print $0}' $file \
  | tr -s " " | sed 's/ $//' > ../gps-stage-18/$file
  done

The data table for the CXD satellites (ns53 to ns73) now looks like this:

Column Variable name type Dim. description
1 decimal_day double 1 GPS time, a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years.
2 Geographic_Latitude double 1 Latitude of satellite (deg)
3 Geographic_Longitude double 1 Longitude of satellite (deg)
4 Rad_Re double 1 (radius of satellite)/Rearth
5-15 rate_electron_measured double 11 Measured rate (Hz) in each of the 11 CXD electron channels
16-20 rate_proton_measured double 5 Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21 LEP_thresh double 1 LEP threshold in E1 channels (0 means low, 1 means high)
22 collection_interval int 1 dosimeter collection period (seconds)
23 year int 1 year (e.g. 2015)
24 decimal_year double 1 decimal year = year + (decimal_day-1.0)/(days in year)
25 SVN_number int 1 SVN number of satellite
26 b_coord_radius double 1 radius from earth's dipole axis (earth radii)
27 b_coord_height double 1 height above the earth's dipole equatorial plane (earth radii)
28 magnetic_longitude double 1 Magnetic longitude (degrees)
29 L_shell double 1 L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30 L_LGM_TS04IGRF double 1 LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31 L_LGM_OP77IGRF double 1 LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32 L_LGM_T89CDIP double 1 LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33 bfield_ratio double 1 Bsatellite/Bequator
34 local_time double 1 magnetic local time (0-24 hours)
35 utc_lgm double 1 UTC (0-24 hours)
36 b_sattelite double 1 B field at satellite (gauss)
37 b_equator double 1 B field at equator (on this field line I think) (gauss)
38-48 electron_background double 11 estimated background in electron channels E1-E11 (Hz)
49-53 proton_background double 5 estimated background in proton channels P1-P5 (Hz)
54 proton_activity int 1 =1 if there is significant proton activity
55 proton_temperature_fit double 1 characteristic momentum -- R0 in the expression given above (MeV/c)
56 proton_density_fit double 1 N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
57 electron_temperature_fit double 1 electron temperature from a one Maxwellian fit (MeV)
58 electron_density_fit double 1 electron number density from a one Maxwellian fit (cm-3)
59-69 model_counts_electron_fit_pf double 11 E1-E11 rates due to proton background based on proton flux fit -- currently not filled (all -1's)
70-74 model_counts_proton_fit_pf double 5 P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
75-85 model_counts_electron_fit double 11 E1-E11 rates from the 9-parameter electron flux model
86-91 proton_integrated_flux_fit double 6 integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
92-121 integral_flux_instrument double 30 (based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
122-151 integral_flux_energy double 30 energies for the integral of integral_flux_instrument (MeV)
152-166 electron_diff_flux_energy double 15 energies for the fluxes in electron_diff_flux_energy (MeV)
167-181 electron_diff_flux double 15 (based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
182-190 Efitpars double 9 fit parameters for 9 parameter electron fit

Stage 19: remove model_counts_electron_fit_pf from the CXD data


The documentation for the CXD field model_counts_electron_fit_pf is:

Column Variable name type Dim. description
59-69 model_counts_electron_fit_pf double 11 E1-E11 rates due to proton background based on proton flux fit -- currently not filled (all -1's)

As the documentation indicates that this field is not filled with useful data, we remove it:

  for file in ns[567]*
  do 
    awk '{$59=$60=$61=$62=$63=$64=$65=$66=$67=$68=$69=""; print $0}' \
$file | tr -s " " | sed 's/ $//' > ../gps-stage-19/$file
  done

The data table for the CXD satellites (ns53 to ns73) now looks like this:

Column Variable name type Dim. description
1 decimal_day double 1 GPS time, a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years.
2 Geographic_Latitude double 1 Latitude of satellite (deg)
3 Geographic_Longitude double 1 Longitude of satellite (deg)
4 Rad_Re double 1 (radius of satellite)/Rearth
5-15 rate_electron_measured double 11 Measured rate (Hz) in each of the 11 CXD electron channels
16-20 rate_proton_measured double 5 Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21 LEP_thresh double 1 LEP threshold in E1 channels (0 means low, 1 means high)
22 collection_interval int 1 dosimeter collection period (seconds)
23 year int 1 year (e.g. 2015)
24 decimal_year double 1 decimal year = year + (decimal_day-1.0)/(days in year)
25 SVN_number int 1 SVN number of satellite
26 b_coord_radius double 1 radius from earth's dipole axis (earth radii)
27 b_coord_height double 1 height above the earth's dipole equatorial plane (earth radii)
28 magnetic_longitude double 1 Magnetic longitude (degrees)
29 L_shell double 1 L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30 L_LGM_TS04IGRF double 1 LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31 L_LGM_OP77IGRF double 1 LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32 L_LGM_T89CDIP double 1 LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33 bfield_ratio double 1 Bsatellite/Bequator
34 local_time double 1 magnetic local time (0-24 hours)
35 utc_lgm double 1 UTC (0-24 hours)
36 b_sattelite double 1 B field at satellite (gauss)
37 b_equator double 1 B field at equator (on this field line I think) (gauss)
38-48 electron_background double 11 estimated background in electron channels E1-E11 (Hz)
49-53 proton_background double 5 estimated background in proton channels P1-P5 (Hz)
54 proton_activity int 1 =1 if there is significant proton activity
55 proton_temperature_fit double 1 characteristic momentum -- R0 in the expression given above (MeV/c)
56 proton_density_fit double 1 N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
57 electron_temperature_fit double 1 electron temperature from a one Maxwellian fit (MeV)
58 electron_density_fit double 1 electron number density from a one Maxwellian fit (cm-3)
59-63 model_counts_proton_fit_pf double 5 P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
64-74 model_counts_electron_fit double 11 E1-E11 rates from the 9-parameter electron flux model
75-80 proton_integrated_flux_fit double 6 integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
81-110 integral_flux_instrument double 30 (based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
111-140 integral_flux_energy double 30 energies for the integral of integral_flux_instrument (MeV)
141-155 electron_diff_flux_energy double 15 energies for the fluxes in electron_diff_flux_energy (MeV)
156-170 electron_diff_flux double 15 (based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
171-179 Efitpars double 9 fit parameters for 9 parameter electron fit

Stage 20: mark invalid values of L_shell for the BDD experiment


I have been informed in a private communication that, for ns41 and ns48, all calculated values of L_shell greater than 25 should be set to 25, and this value is to be interpreted as "too large to calculate".  Therefore, for these two satellites, we set all values of L_shell that currently have the value 25 or greater, to "NA".

  for file in ns4*
  do 
    awk '$28 >= 25 {$28="NA"}; {print $0}' $file > ../gps-stage-20/$file
  done

Time to checkpoint the data again. The checkpointed data file has the MD5 checksum 830dfdcc200fd4b2c0ba41be9e6edf0e.

The data table for ns41 and ns48 now looks like this:

Column Variable name type Dim. Description
1 decimal_day double 1 GPS time -- a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years
2 Geographic_Latitude double 1 Latitude of satellite (deg)
3 Geographic_Longitude double 1 Longitude of satellite (deg)
4 Rad_Re double 1 (radius of satellite)/Rearth
5-12 rate_electron_measured double 8 Measured rate (Hz) in each of the 8 BDD electron channels (E1-E8)
13-20 rate_proton_measured double 8 Measured rate (Hz) in each of the 8 BDD proton channels (P1-P8)
21 collection_interval int 1 dosimeter collection period (seconds)
22 year int 1 year (e.g. 2015)
23 decimal_year double 1 decimal year = year + (decimal_day-1.0)/(days in year)
24 svn_number int 1 SVN number of satellite
25 b_coord_radius double 1 radius from earth's dipole axis (earth radii)
26 b_coord_height double 1 height above the earth's dipole equatorial plane (earth radii)
27 magnetic_longitude double 1 Magnetic longitude (degrees)
28 L_shell double 1 L_shell (earth radii) -- I do not clearly understand the origin of the calculation, but it seems to be a dipole field/T-89
29 bfield_ratio double 1 Bsatellite/Bequator
30 local_time double 1 magnetic local time (0-24 hours)
31 b_sattelite double 1 B field at satellite (gauss)
32 b_equator double 1 B field at equator (on this field line I think) (gauss)
33-40 electron_background double 8 estimated background in electron channels E1-E8 (Hz)
41-48 proton_background double 8 estimated background in proton channels P1-P8 (Hz)
49 proton_activity int 1 =1 if there is significant proton activity
50 electron_temperature double 1 electron temperature from a one Maxwellian fit (MeV)
51 electron_density_fit double 1 electron number density from a one Maxwellian fit (cm-3)
52-59 model_counts_electron_fit double 8 E1-E8 rates from the 2-parameter Maxwellian fit to the electron data
60-67 dtc_counts_electron double 8 Dead time corrected electron rates (from data, not fit)
68-97 integral_flux_instrument double 30 (based on 2 parameter Maxwellian fit) integral of electron flux above integral_flux_energy[i] particles/(cm2sec)
98-127 integral_flux_energy double 30 energies for the integral of integral_flux_instrument (MeV)
128-142 electron_diff_flux_energy double 15 energies for the fluxes in electron_diff_flux_energy (MeV)
143-157 electron_diff_flux double 15 (based on 2 parameter Maxwellian fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))

And for the remaining satellites:

Column Variable name type Dim. description
1 decimal_day double 1 GPS time, a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years.
2 Geographic_Latitude double 1 Latitude of satellite (deg)
3 Geographic_Longitude double 1 Longitude of satellite (deg)
4 Rad_Re double 1 (radius of satellite)/Rearth
5-15 rate_electron_measured double 11 Measured rate (Hz) in each of the 11 CXD electron channels
16-20 rate_proton_measured double 5 Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21 LEP_thresh double 1 LEP threshold in E1 channels (0 means low, 1 means high)
22 collection_interval int 1 dosimeter collection period (seconds)
23 year int 1 year (e.g. 2015)
24 decimal_year double 1 decimal year = year + (decimal_day-1.0)/(days in year)
25 SVN_number int 1 SVN number of satellite
26 b_coord_radius double 1 radius from earth's dipole axis (earth radii)
27 b_coord_height double 1 height above the earth's dipole equatorial plane (earth radii)
28 magnetic_longitude double 1 Magnetic longitude (degrees)
29 L_shell double 1 L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30 L_LGM_TS04IGRF double 1 LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31 L_LGM_OP77IGRF double 1 LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32 L_LGM_T89CDIP double 1 LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33 bfield_ratio double 1 Bsatellite/Bequator
34 local_time double 1 magnetic local time (0-24 hours)
35 utc_lgm double 1 UTC (0-24 hours)
36 b_sattelite double 1 B field at satellite (gauss)
37 b_equator double 1 B field at equator (on this field line I think) (gauss)
38-48 electron_background double 11 estimated background in electron channels E1-E11 (Hz)
49-53 proton_background double 5 estimated background in proton channels P1-P5 (Hz)
54 proton_activity int 1 =1 if there is significant proton activity
55 proton_temperature_fit double 1 characteristic momentum -- R0 in the expression given above (MeV/c)
56 proton_density_fit double 1 N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
57 electron_temperature_fit double 1 electron temperature from a one Maxwellian fit (MeV)
58 electron_density_fit double 1 electron number density from a one Maxwellian fit (cm-3)
59-63 model_counts_proton_fit_pf double 5 P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
64-74 model_counts_electron_fit double 11 E1-E11 rates from the 9-parameter electron flux model
75-80 proton_integrated_flux_fit double 6 integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
81-110 integral_flux_instrument double 30 (based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
111-140 integral_flux_energy double 30 energies for the integral of integral_flux_instrument (MeV)
141-155 electron_diff_flux_energy double 15 energies for the fluxes in electron_diff_flux_energy (MeV)
156-170 electron_diff_flux double 15 (based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
171-179 Efitpars double 9 fit parameters for 9 parameter electron fit