More Sanity Checking of the LANL GPS Charged-Particle Dataset

Prior posts in this series may be found at:

• Original data
• Stages 1 to 4
• Revised stages 1 to 5  
• Stages 6 to 10 
• Stages 11 to 15
• Stages 16 to 20  
• Stages 21 to 25
• Stages 26 to 30 

In this post I continue looking at the values of fields for the CXD experiment carried on ns53 through ns73.

Sanity check for L_shell

The values for L_shell include some NA values (created during earlier processing), representing locations for which the L-shell calculation is not meaningful. It is fairly debatable whether these should be removed, so as to ensure that the user cannot make the mistake of failing to check for NA values. On balance, though, it seems better to leave these values in the dataset, since the actual data from the satellite are not tainted.

Ignoring the NA values, the values of L_shell lie in the following ranges:

L_shell
Satellite	Minimum	Maximum
53	4.079050e+00	6.885292e+01
54	4.073716e+00	7.250067e+01
55	4.067056e+00	7.125280e+01
56	4.081161e+00	7.127625e+01
57	4.113699e+00	7.339611e+01
58	4.088591e+00	7.147417e+01
59	4.058853e+00	7.154358e+01
60	4.083813e+00	7.114283e+01
61	4.061549e+00	7.002652e+01
62	4.082852e+00	6.885380e+01
63	4.076490e+00	7.115145e+01
64	4.105269e+00	7.008071e+01
65	4.068625e+00	6.727069e+01
66	4.084220e+00	6.957386e+01
67	4.108083e+00	7.065966e+01
68	4.108754e+00	7.211983e+01
69	4.106989e+00	6.798032e+01
70	4.100414e+00	6.645398e+01
71	4.103767e+00	7.137340e+01
72	4.103526e+00	7.117196e+01
73	4.096220e+00	6.786675e+01

These look reasonable.

Sanity check for L_LGM_TS04IGRF

The values for L_LGM_TS04IGRF include some NA values (created during earlier processing), representing locations for which the L-shell calculation is not meaningful. It is fairly debatable whether these should be removed, so as to ensure that the user cannot make the mistake of failing to check for NA values. On balance, though, it seems better to leave these values in the dataset, since the actual data from the satellite are not tainted.

Ignoring the NA values, the values of L_LGM_TS04IGRF lie in the following ranges:

L_LGM_TS04IGRF
Satellite	Minimum	Maximum
53	3.947573e+00	6.927239e+01
54	3.901199e+00	6.919588e+01
55	3.936277e+00	6.879617e+01
56	3.792127e+00	7.015551e+01
57	4.008310e+00	7.110403e+01
58	3.900022e+00	7.005057e+01
59	4.007891e+00	6.847356e+01
60	4.003237e+00	7.221038e+01
61	3.902738e+00	6.978320e+01
62	3.912530e+00	7.011196e+01
63	3.971503e+00	6.971264e+01
64	4.024707e+00	6.769095e+01
65	4.015073e+00	7.188151e+01
66	4.007551e+00	6.856100e+01
67	4.050529e+00	6.987180e+01
68	3.995247e+00	6.942418e+01
69	4.048874e+00	6.902460e+01
70	4.046164e+00	6.815424e+01
71	4.071240e+00	6.757528e+01
72	3.999839e+00	6.450151e+01
73	4.044781e+00	6.871470e+01

These look reasonable.

Sanity check for L_LGM_OP77IGRF

The values for L_LGM_OP77IGRF include some NA values (created during earlier processing), representing locations for which the L-shell calculation is not meaningful. It is fairly debatable whether these should be removed, so as to ensure that the user cannot make the mistake of failing to check for NA values. On balance, though, it seems better to leave these values in the dataset, since the actual data from the satellite are not tainted. I note that the description of this field claims that it is "not currently filled". This is obviously not true.

Ignoring the NA values, the values of L_LGM_OP77IGRF lie in the following ranges:

L_LGM_OP77IGRF
Satellite	Minimum	Maximum
53	4.113965e+00	3.660894e+01
54	4.106457e+00	3.617539e+01
55	4.100884e+00	3.591780e+01
56	4.113720e+00	3.660868e+01
57	4.146636e+00	3.850798e+01
58	4.120502e+00	3.787992e+01
59	4.091215e+00	3.613692e+01
60	4.117398e+00	3.684020e+01
61	4.094553e+00	3.684505e+01
62	4.114965e+00	3.617936e+01
63	4.108885e+00	3.639508e+01
64	4.138934e+00	3.603204e+01
65	4.103389e+00	3.535154e+01
66	4.116871e+00	3.776598e+01
67	4.144117e+00	3.279393e+01
68	4.145199e+00	2.837767e+01
69	4.138738e+00	3.545018e+01
70	4.132139e+00	3.260918e+01
71	4.136635e+00	3.643418e+01
72	4.137440e+00	3.532261e+01
73	4.127833e+00	3.632726e+01

These look (mostly) reasonable; but note that the maximum valid values are considerably less than for the other magnetic field models, and also the low maximum for ns68, which may indicate an issue worthy of more investigation with the data for that satellite.

Sanity check for L_LGM_T89CDIP

The values for L_LGM_T89CDIP include some NA values (created during earlier processing), representing locations for which the L-shell calculation is not meaningful. It is fairly debatable whether these should be removed, so as to ensure that the user cannot make the mistake of failing to check for NA values. On balance, though, it seems better to leave these values in the dataset, since the actual data from the satellite are not tainted.

Ignoring the NA values, the values of L_LGM_T89CDIP lie in the following ranges:

L_LGM_T89CDIP
Satellite	Minimum	Maximum
53	4.156665e+00	7.046841e+01
54	4.134309e+00	7.297807e+01
55	4.141375e+00	7.265581e+01
56	4.147715e+00	7.362871e+01
57	4.171050e+00	7.021322e+01
58	4.160699e+00	6.858366e+01
59	4.138150e+00	7.196883e+01
60	4.132394e+00	7.063522e+01
61	4.135012e+00	7.077780e+01
62	4.164926e+00	6.988359e+01
63	4.155301e+00	7.192335e+01
64	4.165912e+00	6.925883e+01
65	4.153215e+00	6.977235e+01
66	4.163230e+00	6.761487e+01
67	4.174354e+00	6.961524e+01
68	4.172899e+00	6.893099e+01
69	4.178059e+00	7.078880e+01
70	4.174908e+00	7.211150e+01
71	4.179994e+00	7.007543e+01
72	4.171566e+00	7.102184e+01
73	4.177882e+00	6.814721e+01

These look reasonable.

Sanity check for bfield_ratio

The definition of bfield_ratio is:

Column	Variable name	type	Dim.	description
33	bfield_ratio	double	1	Bsatellite/Bequator

 This appears to mean:

Column	Variable name	type	Dim.	description
33	bfield_ratio	double	1	Ratio of magnetic field at the satellite to the magnetic field along the field line to the equator.

The recorded value of this field is not obvious (especially since it is naturally highly dependent on the field model one uses).

Looking at the values of the bfield_ratio, one quickly sees many NA values and, worse, values of the form -n.nnnnnne+99 that make no sense at all. It is therefore not obvious that this field was ever subject to any kind of quality control.

In private communication with the LANL team, I discovered that:

1. There is no in situ satellite instrument for measuring magnetic fields;
2. All fields related to magnetic measurement are calculated, not measured.
3. The primary reference for the calculation is: https://github.com/drsteve/LANLGeoMag.

It does not, therefore, seem sensible to retain these fields from the records (i.e., in particular, bfield_ratio, b_sattelite [which I remove with some relief, for obvious reasons] and b_equator [removed with relief for reasons that are almost as obvious if one looks at the official documentation for the field]. If a user wishes to calculate the values of equivalent fields for himself, he may do so, using the then-current versions of the routines cited above or similar routines. (Although it is obviously troubling that apparently it was possible to obtain nonsense results from the published routines, at least as of the date when the original 1.3 dataset was released. So, if a user calculates magnetic values for himself, he is urged to check their sanity before relying on them.)

Stage 31: Remove bfield_ratio

for file in ns[567]*
  do 
    awk '{$33=""; print $0}' $file | tr -s " " | sed 's/ $//' > ../gps-stage-31/$file
  done

The records for ns53 to ns73 now look like this:

Column	Variable name	type	Dim.	description
1	decimal_day	double	1	GPS time: a decimal number in the range [1, 367) in leap years or [1, 366) otherwise, representing the day of the year (1-Jan 00:00 to 31-Dec 24:00).
2	Geographic_Latitude	double	1	Latitude of satellite (°, N +ve)
3	Geographic_Longitude	double	1	Longitude of satellite (°, E +ve, measured from Greenwich meridian)
4	Rad_Re	double	1	Distance from centre of Earth, in units of Earth radii.
5-15	rate_electron_measured	double	11	Measured rate (Hz) in each of the 11 CXD electron channels
16-20	rate_proton_measured	double	5	Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21	LEP_thresh	double	1	LEP threshold in E1 channels, in keV
22	collection_interval	int	1	dosimeter collection period (seconds)
23	year	int	1	year (e.g. 2015)
24	decimal_year	double	1	decimal year = year + (decimal_day-1.0) / (days in year)
25	SVN	int	1	Satellite Vehicle Number
26	b_coord_radius	double	1	Distance from dipole axis, in units of Earth radii.
27	b_coord_height	double	1	Distance from dipole equatorial plane, in units of Earth radii (N +ve).
28	magnetic_longitude	double	1	Magnetic longitude (degrees)
29	L_shell	double	1	L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30	L_LGM_TS04IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31	L_LGM_OP77IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32	L_LGM_T89CDIP	double	1	LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33	local_time	double	1	magnetic local time (0-24 hours)
34	utc_lgm	double	1	UTC (0-24 hours)
35	b_sattelite	double	1	B field at satellite (gauss)
36	b_equator	double	1	B field at equator (on this field line I think) (gauss)
37-47	electron_background	double	11	estimated background in electron channels E1-E11 (Hz)
48-52	proton_background	double	5	estimated background in proton channels P1-P5 (Hz)
53	proton_activity	int	1	=1 if there is significant proton activity
54	proton_temperature_fit	double	1	characteristic momentum -- R0 in the expression given above (MeV/c)
55	proton_density_fit	double	1	N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
56	electron_temperature_fit	double	1	electron temperature from a one Maxwellian fit (MeV)
57	electron_density_fit	double	1	electron number density from a one Maxwellian fit (cm-3)
58-62	model_counts_proton_fit_pf	double	5	P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
63-73	model_counts_electron_fit	double	11	E1-E11 rates from the 9-parameter electron flux model
74-79	proton_integrated_flux_fit	double	6	integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
80-109	integral_flux_instrument	double	30	(based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
110-139	integral_flux_energy	double	30	energies for the integral of integral_flux_instrument (MeV)
140-154	electron_diff_flux_energy	double	15	energies for the fluxes in electron_diff_flux_energy (MeV)
155-169	electron_diff_flux	double	15	(based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
170-178	Efitpars	double	9	fit parameters for 9 parameter electron fit

Stage 32: Remove b_sattelite

  for file in ns[567]*
  do 
    awk '{$35=""; print $0}' $file | tr -s " " | sed 's/ $//' > ../gps-stage-32/$file
  done

The records for ns53 to ns73 now look like this:

Column	Variable name	type	Dim.	description
1	decimal_day	double	1	GPS time: a decimal number in the range [1, 367) in leap years or [1, 366) otherwise, representing the day of the year (1-Jan 00:00 to 31-Dec 24:00).
2	Geographic_Latitude	double	1	Latitude of satellite (°, N +ve)
3	Geographic_Longitude	double	1	Longitude of satellite (°, E +ve, measured from Greenwich meridian)
4	Rad_Re	double	1	Distance from centre of Earth, in units of Earth radii.
5-15	rate_electron_measured	double	11	Measured rate (Hz) in each of the 11 CXD electron channels
16-20	rate_proton_measured	double	5	Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21	LEP_thresh	double	1	LEP threshold in E1 channels, in keV
22	collection_interval	int	1	dosimeter collection period (seconds)
23	year	int	1	year (e.g. 2015)
24	decimal_year	double	1	decimal year = year + (decimal_day-1.0) / (days in year)
25	SVN	int	1	Satellite Vehicle Number
26	b_coord_radius	double	1	Distance from dipole axis, in units of Earth radii.
27	b_coord_height	double	1	Distance from dipole equatorial plane, in units of Earth radii (N +ve).
28	magnetic_longitude	double	1	Magnetic longitude (degrees)
29	L_shell	double	1	L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30	L_LGM_TS04IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31	L_LGM_OP77IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32	L_LGM_T89CDIP	double	1	LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33	local_time	double	1	magnetic local time (0-24 hours)
34	utc_lgm	double	1	UTC (0-24 hours)
35	b_equator	double	1	B field at equator (on this field line I think) (gauss)
36-46	electron_background	double	11	estimated background in electron channels E1-E11 (Hz)
47-51	proton_background	double	5	estimated background in proton channels P1-P5 (Hz)
52	proton_activity	int	1	=1 if there is significant proton activity
53	proton_temperature_fit	double	1	characteristic momentum -- R0 in the expression given above (MeV/c)
54	proton_density_fit	double	1	N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
55	electron_temperature_fit	double	1	electron temperature from a one Maxwellian fit (MeV)
56	electron_density_fit	double	1	electron number density from a one Maxwellian fit (cm-3)
57-61	model_counts_proton_fit_pf	double	5	P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
62-72	model_counts_electron_fit	double	11	E1-E11 rates from the 9-parameter electron flux model
73-78	proton_integrated_flux_fit	double	6	integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
79-108	integral_flux_instrument	double	30	(based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
109-138	integral_flux_energy	double	30	energies for the integral of integral_flux_instrument (MeV)
139-153	electron_diff_flux_energy	double	15	energies for the fluxes in electron_diff_flux_energy (MeV)
154-168	electron_diff_flux	double	15	(based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
169-177	Efitpars	double	9	fit parameters for 9 parameter electron fit

Stage 33: Remove b_equator

  for file in ns[567]*
  do 
    awk '{$35=""; print $0}' $file | tr -s " " | sed 's/ $//' > ../gps-stage-32/$file
  done

The records for ns53 to ns73 now look like this:

Column	Variable name	type	Dim.	description
1	decimal_day	double	1	GPS time: a decimal number in the range [1, 367) in leap years or [1, 366) otherwise, representing the day of the year (1-Jan 00:00 to 31-Dec 24:00).
2	Geographic_Latitude	double	1	Latitude of satellite (°, N +ve)
3	Geographic_Longitude	double	1	Longitude of satellite (°, E +ve, measured from Greenwich meridian)
4	Rad_Re	double	1	Distance from centre of Earth, in units of Earth radii.
5-15	rate_electron_measured	double	11	Measured rate (Hz) in each of the 11 CXD electron channels
16-20	rate_proton_measured	double	5	Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21	LEP_thresh	double	1	LEP threshold in E1 channels, in keV
22	collection_interval	int	1	dosimeter collection period (seconds)
23	year	int	1	year (e.g. 2015)
24	decimal_year	double	1	decimal year = year + (decimal_day-1.0) / (days in year)
25	SVN	int	1	Satellite Vehicle Number
26	b_coord_radius	double	1	Distance from dipole axis, in units of Earth radii.
27	b_coord_height	double	1	Distance from dipole equatorial plane, in units of Earth radii (N +ve).
28	magnetic_longitude	double	1	Magnetic longitude (degrees)
29	L_shell	double	1	L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30	L_LGM_TS04IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31	L_LGM_OP77IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field (not currently filled)
32	L_LGM_T89CDIP	double	1	LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33	local_time	double	1	magnetic local time (0-24 hours)
34	utc_lgm	double	1	UTC (0-24 hours)
35-45	electron_background	double	11	estimated background in electron channels E1-E11 (Hz)
46-50	proton_background	double	5	estimated background in proton channels P1-P5 (Hz)
51	proton_activity	int	1	=1 if there is significant proton activity
52	proton_temperature_fit	double	1	characteristic momentum -- R0 in the expression given above (MeV/c)
53	proton_density_fit	double	1	N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
54	electron_temperature_fit	double	1	electron temperature from a one Maxwellian fit (MeV)
55	electron_density_fit	double	1	electron number density from a one Maxwellian fit (cm-3)
56-60	model_counts_proton_fit_pf	double	5	P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
61-71	model_counts_electron_fit	double	11	E1-E11 rates from the 9-parameter electron flux model
72-77	proton_integrated_flux_fit	double	6	integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
78-107	integral_flux_instrument	double	30	(based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
108-137	integral_flux_energy	double	30	energies for the integral of integral_flux_instrument (MeV)
138-152	electron_diff_flux_energy	double	15	energies for the fluxes in electron_diff_flux_energy (MeV)
153-167	electron_diff_flux	double	15	(based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
168-176	Efitpars	double	9	fit parameters for 9 parameter electron fit

Sanity check for local_time

The definition of local_time is:

Column	Variable name	type	Dim.	description
33	local_time	double	1	magnetic local time (0-24 hours)

 There are a couple of problems with this definition:

1. The field name gives no indication that refers to a magnetic value, so should be renamed magnetic_local_time;
2. It provides no indication of exactly how magnetic local time is defined (and several mutually exclusive definitions are extant).

However, providing that one does not ascribe great accuracy to this value, it might conceivably be useful, so it is not unreasonable to retain it, with some emendations.

Firstly, we redefine it slightly (although unfortunately being forced to retain the ambiguity as to its precise meaning):

Column	Variable name	type	Dim.	description
33	magnetic_local_time	double	1	magnetic local time [0-24) hours

The actual values of this field lie in the following ranges:

magnetic_local_time
Satellite	Minimum	Maximum
53	7.845492e-06	2.399998e+01
54	7.956284e-06	2.400000e+01
55	3.351804e-05	2.399994e+01
56	1.586655e-05	2.399998e+01
57	4.068317e-05	2.399998e+01
58	2.519977e-05	2.399999e+01
59	4.977282e-06	2.400000e+01
60	7.915747e-06	2.399999e+01
61	2.649532e-06	2.400000e+01
62	1.991213e-05	2.400000e+01
63	3.556485e-06	2.400000e+01
64	1.172278e-05	2.399996e+01
65	1.809704e-05	2.399994e+01
66	4.205324e-05	2.399998e+01
67	4.227346e-05	2.399994e+01
68	2.200456e-05	2.399982e+01
69	3.367094e-05	2.399992e+01
70	1.883996e-05	2.399991e+01
71	1.952489e-04	2.399984e+01
72	1.620355e-04	2.399977e+01
73	6.868977e-05	2.399989e+01

There are obvious issues with these values:

1. They are not normalised to [0-24)
2. The numbers are reported to different precision, depending on their magnitude (and, for small numbers, they are reported to an unjustifiable precision).

These matters are easily fixed by the following script, do-stage-34.py:

#!/usr/bin/env python
# -*- coding: utf8 -*-

import re
import sys

filename = sys.argv[1]

field_nr = 32    # magnetic_local_time (wrt 0)

with open(filename) as records:
  for line in records:
    fields = line.split()
    value_str = fields[field_nr]
    value = float(value_str)
  
    if (value >= 24):
      value = value - 24
    
    output_format = '{:8.5f}'
    
    new_value_str = output_format.format(value)
  
    fields[field_nr] = new_value_str
    newline = " ".join(fields)
 
    print newline

Stage 34: Reformat values of magnetic_local_time

Execute do-stage-34.py on all the relevant satellites:

for file in ns[567]*
do  
  ./do-stage-34.py $file > ../gps-stage-34/$file
done

The maxima and minima now have these (reasonable) values:

magnetic_local_time
Satellite	Minimum	Maximum
53	0.00001	23.99998
54	0.00000	23.99998
55	0.00003	23.99994
56	0.00002	23.99998
57	0.00004	23.99998
58	0.00003	23.99999
59	0.00000	23.99999
60	0.00001	23.99999
61	0.00000	23.99999
62	0.00000	23.99996
63	0.00000	23.99999
64	0.00001	23.99996
65	0.00002	23.99994
66	0.00004	23.99998
67	0.00004	23.99994
68	0.00002	23.99982
69	0.00003	23.99992
70	0.00002	23.99991
71	0.00020	23.99984
72	0.00016	23.99977
73	0.00007	23.99989

Sanity check for utc_lgm

The definition of utc_lgm [it is unclear why the field is not called simply utc] is:

Column	Variable name	type	Dim.	description
34	utc_lgm	double	1	UTC (0-24 hours)

 We offer a small correction:

Column	Variable name	type	Dim.	description
34	utc_lgm	double	1	UTC [0-24) hours

The actual values of this field lie in the following ranges:

utc_lgm
Satellite	Minimum	Maximum
53	5.567111e-03	2.396917e+01
54	5.567111e-03	2.398610e+01
55	1.916133e-02	2.399556e+01
56	6.108889e-03	2.397277e+01
57	1.248933e-02	2.399583e+01
58	2.527778e-02	2.398555e+01
59	5.563556e-03	2.399222e+01
60	1.949778e-03	2.395944e+01
61	9.177333e-03	2.398278e+01
62	9.993333e-03	2.398639e+01
63	7.219556e-03	2.399306e+01
64	1.554756e-02	2.399083e+01
65	2.001156e-02	2.399666e+01
66	3.249156e-02	2.399500e+01
67	1.055556e-02	2.398972e+01
68	1.111778e-02	2.395138e+01
69	1.638756e-02	2.399332e+01
70	4.445378e-02	2.399693e+01
71	1.221156e-02	2.399445e+01
72	2.527778e-02	2.396389e+01
73	4.111778e-02	2.397445e+01

The numbers are obviously reported to a precision that is a function of their magnitude. This is easily fixed by the following script, do-stage-35.py:

#!/usr/bin/env python
# -*- coding: utf8 -*-

import re
import sys

filename = sys.argv[1]

field_nr = 33    # utc_lgm (wrt 0)

with open(filename) as records:
  for line in records:
    fields = line.split()
    value_str = fields[field_nr]
    value = float(value_str)
    
    output_format = '{:8.5f}'
    
    new_value_str = output_format.format(value)
  
    fields[field_nr] = new_value_str
    newline = " ".join(fields)
 
    print newline

Note, however, that the minima are orders of magnitude greater than the originally reported values for the field magnetic_local_time. There is  a slight inconsistency in the mechanisms for reporting these two types of time. It is hard to see how this will make any difference when actually using the dataset, but I note it in passing.

Stage 35: Reformat values of utc_lgm

Execute do-stage-34.py on all the relevant satellites:

for file in ns[567]*
do  
  ./do-stage-35.py $file > ../gps-stage-35/$file
done

The maxima and minima now have these values:

utc_lgm
Satellite	Minimum	Maximum
53	0.00557	23.96917
54	0.00557	23.98610
55	0.01916	23.99556
56	0.00611	23.97277
57	0.01249	23.99583
58	0.02528	23.98555
59	0.00556	23.99222
60	0.00195	23.95944
61	0.00918	23.98278
62	0.00999	23.98639
63	0.00722	23.99306
64	0.01555	23.99083
65	0.02001	23.99666
66	0.03249	23.99500
67	0.01056	23.98972
68	0.01112	23.95138
69	0.01639	23.99332
70	0.04445	23.99693
71	0.01221	23.99445
72	0.02528	23.96389
73	0.04112	23.97445

A checkpoint copy of the stage 35 files is available here, with MD5 checksum b85d41e615126e3912bd50aa55a4af54.

The data table for ns41 and ns48 still looks like this:

Column	Variable name	type	Dim.	Description
1	decimal_day	double	1	GPS time -- a number from 1 (1-Jan 00:00) to 366 (31-Dec 24:00) or 367 in leap years
2	Geographic_Latitude	double	1	Latitude of satellite (deg)
3	Geographic_Longitude	double	1	Longitude of satellite (deg)
4	Rad_Re	double	1	(radius of satellite)/Rearth
5-12	rate_electron_measured	double	8	Measured rate (Hz) in each of the 8 BDD electron channels (E1-E8)
13-20	rate_proton_measured	double	8	Measured rate (Hz) in each of the 8 BDD proton channels (P1-P8)
21	collection_interval	int	1	dosimeter collection period (seconds)
22	year	int	1	year (e.g. 2015)
23	decimal_year	double	1	decimal year = year + (decimal_day-1.0)/(days in year)
24	svn_number	int	1	SVN number of satellite
25	b_coord_radius	double	1	radius from earth's dipole axis (earth radii)
26	b_coord_height	double	1	height above the earth's dipole equatorial plane (earth radii)
27	magnetic_longitude	double	1	Magnetic longitude (degrees)
28	L_shell	double	1	L_shell (earth radii) -- I do not clearly understand the origin of the calculation, but it seems to be a dipole field/T-89
29	bfield_ratio	double	1	Bsatellite/Bequator
30	local_time	double	1	magnetic local time (0-24 hours)
31	b_sattelite	double	1	B field at satellite (gauss)
32	b_equator	double	1	B field at equator (on this field line I think) (gauss)
33-40	electron_background	double	8	estimated background in electron channels E1-E8 (Hz)
41-48	proton_background	double	8	estimated background in proton channels P1-P8 (Hz)
49	proton_activity	int	1	=1 if there is significant proton activity
50	electron_temperature	double	1	electron temperature from a one Maxwellian fit (MeV)
51	electron_density_fit	double	1	electron number density from a one Maxwellian fit (cm-3)
52-59	model_counts_electron_fit	double	8	E1-E8 rates from the 2-parameter Maxwellian fit to the electron data
60-67	dtc_counts_electron	double	8	Dead time corrected electron rates (from data, not fit)
68-97	integral_flux_instrument	double	30	(based on 2 parameter Maxwellian fit) integral of electron flux above integral_flux_energy[i] particles/(cm2sec)
98-127	integral_flux_energy	double	30	energies for the integral of integral_flux_instrument (MeV)
128-142	electron_diff_flux_energy	double	15	energies for the fluxes in electron_diff_flux_energy (MeV)
143-157	electron_diff_flux	double	15	(based on 2 parameter Maxwellian fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))

And for the remaining satellites we now have (I have slightly edited the description of the magnetic_longitude and L_LGM_OP77IGRF fields):

Column	Variable name	type	Dim.	description
1	decimal_day	double	1	GPS time: a decimal number in the range [1, 367) in leap years or [1, 366) otherwise, representing the day of the year (1-Jan 00:00 to 31-Dec 24:00).
2	Geographic_Latitude	double	1	Latitude of satellite (°, N +ve)
3	Geographic_Longitude	double	1	Longitude of satellite (°, E +ve, measured from Greenwich meridian)
4	Rad_Re	double	1	Distance from centre of Earth, in units of Earth radii.
5-15	rate_electron_measured	double	11	Measured rate (Hz) in each of the 11 CXD electron channels
16-20	rate_proton_measured	double	5	Measured rate (Hz) in each of the 5 CXD proton channels (P1-P5)
21	LEP_thresh	double	1	LEP threshold in E1 channels, in keV
22	collection_interval	int	1	dosimeter collection period (seconds)
23	year	int	1	year (e.g. 2015)
24	decimal_year	double	1	decimal year = year + (decimal_day-1.0) / (days in year)
25	SVN	int	1	Satellite Vehicle Number
26	b_coord_radius	double	1	Distance from dipole axis, in units of Earth radii.
27	b_coord_height	double	1	Distance from dipole equatorial plane, in units of Earth radii (N +ve).
28	magnetic_longitude	double	1	Magnetic longitude (°)
29	L_shell	double	1	L shell: McIlwain calculation according to model with T89 External Field, IGRF Internal Field.
30	L_LGM_TS04IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, TS04 External Field, IGRF Internal Field.
31	L_LGM_OP77IGRF	double	1	LanlGeoMag L-shell McIlwain calculation, OP77 External Field, IGRF Internal Field
32	L_LGM_T89CDIP	double	1	LanlGeoMag L-shell McIlwain calculation, T89 External Field, Centered Dipole Internal Field
33	magnetic_local_time	double	1	magnetic local time [0-24) hours
34	utc_lgm	double	1	UTC [0-24) hours
35-45	electron_background	double	11	estimated background in electron channels E1-E11 (Hz)
46-50	proton_background	double	5	estimated background in proton channels P1-P5 (Hz)
51	proton_activity	int	1	=1 if there is significant proton activity
52	proton_temperature_fit	double	1	characteristic momentum -- R0 in the expression given above (MeV/c)
53	proton_density_fit	double	1	N0 parameter in fit to proton flux ((protons/(cm2 sec sr MeV))
54	electron_temperature_fit	double	1	electron temperature from a one Maxwellian fit (MeV)
55	electron_density_fit	double	1	electron number density from a one Maxwellian fit (cm-3)
56-60	model_counts_proton_fit_pf	double	5	P1-P5 rate from proton fit (using proton_temperature_fit, proton_density_fit)
61-71	model_counts_electron_fit	double	11	E1-E11 rates from the 9-parameter electron flux model
72-77	proton_integrated_flux_fit	double	6	integral of proton flux (based on fit) above 10, 15.85, 25.11, 30, 40, 79.43 MeV (proton kinetic energy)
78-107	integral_flux_instrument	double	30	(based on 9 parameter fit) integral of electron flux above integral_flux_energy[i] particles/(cm2 sec)
108-137	integral_flux_energy	double	30	energies for the integral of integral_flux_instrument (MeV)
138-152	electron_diff_flux_energy	double	15	energies for the fluxes in electron_diff_flux_energy (MeV)
153-167	electron_diff_flux	double	15	(based on 9 parameter fit) electron flux at energies electron_diff_flux[i] (particle/(cm2 sr MeV sec))
168-176	Efitpars	double	9	fit parameters for 9 parameter electron fit