RainQC Job Manager daily report 2023-07-07
The completeness numbers seem to be falling. --Tom Thomas G. Dietterich, Distinguished Professor Voice: 541-737-5559 School of Electrical Engineering FAX: 541-737-1300 and Computer Science URL: eecs.oregonstate.edu/~tgd US Mail: 1148 Kelley Engineering Center Office: 2067 Kelley Engineering Center Oregon State Univ., Corvallis, OR 97331-5501 From: Rainqc-jobman <rainqc-jobman-bounces@engr.oregonstate.edu> On Behalf Of slater@oregonstate.edu Sent: Thursday, July 6, 2023 23:23 To: rainqc-jobman@engr.oregonstate.edu Subject: [Rainqc-jobman] RainQC Job Manager daily report 2023-07-07 Current UTC date: 2023-07-07 -> scoring models for previous day: 2023-07-06 --------------------------------------------------------------------------------- Daily Model Data Completeness Check: data completeness 50% | complete models: 142 of 273 (52.01%) data completeness 60% | complete models: 141 of 273 (51.65%) data completeness 70% | complete models: 137 of 273 (50.18%) data completeness 75% | complete models: 137 of 273 (50.18%) data completeness 80% | complete models: 137 of 273 (50.18%) data completeness 85% | complete models: 133 of 273 (48.72%) data completeness 90% | complete models: 127 of 273 (46.52%) data completeness 95% | complete models: 127 of 273 (46.52%) data completeness 100% | complete models: 115 of 273 (42.12%) ---------------------------- station status | total: 313, delayed: 120, offline 24h: 64, offline week: 53 | battery, min: 0, max: 100, mean: 65.8, std dev: 24.87 | battery, common values: [(100, 167), (0, 64), (93, 4), (96, 4), (76, 4)] | battery <= mean, common countries: [('KE', 28), ('GH', 21), ('ML', 9), ('UG', 7), ('TG', 5)] ---------------------------- 86 LOW DATA (< 0.9) and 73 NO DATA weather stations impacted 146 RainQC models LOW/NO data station impact on models: [('TA00715', 8), ('TA00568', 8), ('TA00242', 8), ('TA00196', 7), ('TA00414', 6), ('TA00413', 6), ('TA00243', 5), ('TA00190', 4), ('TA00361', 4), ('TA00367', 4), ('TA00368', 4), ('TA00530', 4), ('TA00565', 4), ('TA00543', 4), ('TA00700', 4), ('TA00032', 3), ('TA00222', 3), ('TA00041', 3), ('TA00267', 3), ('TA00178', 3), ('TA00066', 3), ('TA00094', 3), ('TA00115', 3), ('TA00126', 3), ('TA00154', 3), ('TA00482', 3), ('TA00385', 3), ('TA00289', 3), ('TA00436', 3), ('TA00430', 3), ('TA00024', 2), ('TA00050', 2), ('TA00171', 2), ('TA00101', 2), ('TA00118', 2), ('TA00148', 2), ('TA00164', 2), ('TA00487', 2), ('TA00684', 2), ('TA00232', 2), ('TA00271', 2), ('TA00314', 2), ('TA00691', 2), ('TA00339', 2), ('TA00373', 2), ('TA00451', 2), ('TA00462', 2), ('TA00471', 2), ('TA00594', 2), ('TA00001', 1), ('TA00278', 1), ('TA00014', 1), ('TA00031', 1), ('TA00044', 1), ('TA00062', 1), ('TA00077', 1), ('TA00091', 1), ('TA00095', 1), ('TA00123', 1), ('TA00157', 1), ('TA00219', 1), ('TA00650', 1), ('TA00229', 1), ('TA00237', 1), ('TA00260', 1), ('TA00276', 1), ('TA00287', 1), ('TA00286', 1), ('TA00290', 1), ('TA00336', 1), ('TA00350', 1), ('TA00362', 1), ('TA00333', 1), ('TA00369', 1), ('TA00382', 1), ('TA00389', 1), ('TA00422', 1), ('TA00432', 1), ('TA00493', 1), ('TA00529', 1), ('TA00533', 1), ('TA00535', 1), ('TA00652', 1), ('TA00655', 1), ('TA00677', 1), ('TA00702', 1)] ----------------------------------------------------------- Processed daily jobs for UTC date: 2023-07-06 Start time: 2023-07-07T05:33:22+00:00 End time : 2023-07-07T06:22:57+00:00 Elapsed time HH:MM:SS: 0:49:36 --------------------- Before job processing job table stats: Total 'success' count: 127 Total 'failure' count: 137 Total record count: 1123 Job history table record count: 77312 Scoring job record table record count: 301 --------------------- After job processing job table stats: Total 'success' count: 133 (flag=2 count: 8) (flag 2->1 downgrades: 6) | 'success' count for 2023-07-06: 117 (flag=2 count: 8) | 'success' count for 2023-07-05: 14 (flag=2 count: 0) | 'success' count for 2023-07-04: 0 (flag=2 count: 0) | 'success' count for 2023-07-03: 0 (flag=2 count: 0) | 'success' count for 2023-07-02: 0 (flag=2 count: 0) | 'success' count for 2023-07-01: 2 (flag=2 count: 0) | 'success' count for 2023-06-30: 0 (flag=2 count: 0) Anomalies (flag=2): TA00006 2023-07-06 | score: 119.883 (thresh: 73.464) -- 'pr' t: 1.582 mm n: (20.378 mm, 44 km), (7.161 mm, 65 km), (73.073 mm, 75 km), (0.102 mm, 82 km), (6.783 mm, 84 km) TA00013 2023-07-06 | score: 317.347 (thresh: 195.678) -- 'pr' t: 0.000 mm n: (4.237 mm, 25 km), (21.692 mm, 26 km), (0.102 mm, 40 km), (7.161 mm, 44 km) TA00061 2023-07-06 | score: 174.822 (thresh: 80.285) -- 'pr' t: 0.000 mm n: (0.0 mm, 23 km), (12.19 mm, 34 km), (9.138 mm, 35 km) TA00110 2023-07-06 | score: 246.050 (thresh: 190.539) -- 'pr' t: 0.034 mm n: (21.692 mm, 49 km), (4.999 mm, 117 km) TA00140 2023-07-06 | score: 118.721 (thresh: 94.060) -- 'pr' t: 31.055 mm n: (5.92 mm, 35 km), (44.064 mm, 39 km), (0.0 mm, 68 km) TA00141 2023-07-06 | score: 307.212 (thresh: 126.729) -- 'pr' t: 5.920 mm n: (31.055 mm, 35 km), (44.064 mm, 56 km), (2.584 mm, 66 km), (26.76 mm, 70 km) TA00274 2023-07-06 | score: 193.030 (thresh: 107.907) -- 'pr' t: 44.064 mm n: (31.055 mm, 39 km) TA00587 2023-07-06 | score: 264.657 (thresh: 134.554) -- 'pr' t: 0.000 mm n: (9.138 mm, 12 km), (12.19 mm, 20 km), (0.0 mm, 23 km) -------- Total 'failure' count: 143 Total record count: 1132 Job history table record count: 77576 Scoring job record table record count: 302 -----------------------------------------------------------
Possibly poorer battery health due to more regions in the network having cooler temperatures and lower average daily incident shortwave solar energy? Average temp and solar energy for Kenya look low in July, for instance: https://weatherspark.com/countries/KE<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fweatherspark.com%2Fcountries%2FKE&data=05%7C01%7Crainqc-jobman%40engr.oregonstate.edu%7Cb9674e69d8974291f9ed08db7f14cb27%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638243499498387860%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zfBp5q2Af%2FHbAsho3J9fjqGFkv9BaP9FBArFtt8OY0M%3D&reserved=0> Of course, I don't know how these averages compare to current local conditions. Or if these are the key factors impacting the batteries. Perhaps it is dust on the solar panels from the 'great migration' :-) From: Rainqc-jobman <rainqc-jobman-bounces@engr.oregonstate.edu> On Behalf Of Dietterich, Thomas Sent: Friday, July 7, 2023 9:16 AM To: RainQC Job Manager <rainqc-jobman@engr.oregonstate.edu> Subject: Re: [Rainqc-jobman] RainQC Job Manager daily report 2023-07-07 The completeness numbers seem to be falling. --Tom Thomas G. Dietterich, Distinguished Professor Voice: 541-737-5559 School of Electrical Engineering FAX: 541-737-1300 and Computer Science URL: eecs.oregonstate.edu/~tgd US Mail: 1148 Kelley Engineering Center Office: 2067 Kelley Engineering Center Oregon State Univ., Corvallis, OR 97331-5501 From: Rainqc-jobman <rainqc-jobman-bounces@engr.oregonstate.edu<mailto:rainqc-jobman-bounces@engr.oregonstate.edu>> On Behalf Of slater@oregonstate.edu<mailto:slater@oregonstate.edu> Sent: Thursday, July 6, 2023 23:23 To: rainqc-jobman@engr.oregonstate.edu<mailto:rainqc-jobman@engr.oregonstate.edu> Subject: [Rainqc-jobman] RainQC Job Manager daily report 2023-07-07 Current UTC date: 2023-07-07 -> scoring models for previous day: 2023-07-06 --------------------------------------------------------------------------------- Daily Model Data Completeness Check: data completeness 50% | complete models: 142 of 273 (52.01%) data completeness 60% | complete models: 141 of 273 (51.65%) data completeness 70% | complete models: 137 of 273 (50.18%) data completeness 75% | complete models: 137 of 273 (50.18%) data completeness 80% | complete models: 137 of 273 (50.18%) data completeness 85% | complete models: 133 of 273 (48.72%) data completeness 90% | complete models: 127 of 273 (46.52%) data completeness 95% | complete models: 127 of 273 (46.52%) data completeness 100% | complete models: 115 of 273 (42.12%) ---------------------------- station status | total: 313, delayed: 120, offline 24h: 64, offline week: 53 | battery, min: 0, max: 100, mean: 65.8, std dev: 24.87 | battery, common values: [(100, 167), (0, 64), (93, 4), (96, 4), (76, 4)] | battery <= mean, common countries: [('KE', 28), ('GH', 21), ('ML', 9), ('UG', 7), ('TG', 5)] ---------------------------- 86 LOW DATA (< 0.9) and 73 NO DATA weather stations impacted 146 RainQC models LOW/NO data station impact on models: [('TA00715', 8), ('TA00568', 8), ('TA00242', 8), ('TA00196', 7), ('TA00414', 6), ('TA00413', 6), ('TA00243', 5), ('TA00190', 4), ('TA00361', 4), ('TA00367', 4), ('TA00368', 4), ('TA00530', 4), ('TA00565', 4), ('TA00543', 4), ('TA00700', 4), ('TA00032', 3), ('TA00222', 3), ('TA00041', 3), ('TA00267', 3), ('TA00178', 3), ('TA00066', 3), ('TA00094', 3), ('TA00115', 3), ('TA00126', 3), ('TA00154', 3), ('TA00482', 3), ('TA00385', 3), ('TA00289', 3), ('TA00436', 3), ('TA00430', 3), ('TA00024', 2), ('TA00050', 2), ('TA00171', 2), ('TA00101', 2), ('TA00118', 2), ('TA00148', 2), ('TA00164', 2), ('TA00487', 2), ('TA00684', 2), ('TA00232', 2), ('TA00271', 2), ('TA00314', 2), ('TA00691', 2), ('TA00339', 2), ('TA00373', 2), ('TA00451', 2), ('TA00462', 2), ('TA00471', 2), ('TA00594', 2), ('TA00001', 1), ('TA00278', 1), ('TA00014', 1), ('TA00031', 1), ('TA00044', 1), ('TA00062', 1), ('TA00077', 1), ('TA00091', 1), ('TA00095', 1), ('TA00123', 1), ('TA00157', 1), ('TA00219', 1), ('TA00650', 1), ('TA00229', 1), ('TA00237', 1), ('TA00260', 1), ('TA00276', 1), ('TA00287', 1), ('TA00286', 1), ('TA00290', 1), ('TA00336', 1), ('TA00350', 1), ('TA00362', 1), ('TA00333', 1), ('TA00369', 1), ('TA00382', 1), ('TA00389', 1), ('TA00422', 1), ('TA00432', 1), ('TA00493', 1), ('TA00529', 1), ('TA00533', 1), ('TA00535', 1), ('TA00652', 1), ('TA00655', 1), ('TA00677', 1), ('TA00702', 1)] ----------------------------------------------------------- Processed daily jobs for UTC date: 2023-07-06 Start time: 2023-07-07T05:33:22+00:00 End time : 2023-07-07T06:22:57+00:00 Elapsed time HH:MM:SS: 0:49:36 --------------------- Before job processing job table stats: Total 'success' count: 127 Total 'failure' count: 137 Total record count: 1123 Job history table record count: 77312 Scoring job record table record count: 301 --------------------- After job processing job table stats: Total 'success' count: 133 (flag=2 count: 8) (flag 2->1 downgrades: 6) | 'success' count for 2023-07-06: 117 (flag=2 count: 8) | 'success' count for 2023-07-05: 14 (flag=2 count: 0) | 'success' count for 2023-07-04: 0 (flag=2 count: 0) | 'success' count for 2023-07-03: 0 (flag=2 count: 0) | 'success' count for 2023-07-02: 0 (flag=2 count: 0) | 'success' count for 2023-07-01: 2 (flag=2 count: 0) | 'success' count for 2023-06-30: 0 (flag=2 count: 0) Anomalies (flag=2): TA00006 2023-07-06 | score: 119.883 (thresh: 73.464) -- 'pr' t: 1.582 mm n: (20.378 mm, 44 km), (7.161 mm, 65 km), (73.073 mm, 75 km), (0.102 mm, 82 km), (6.783 mm, 84 km) TA00013 2023-07-06 | score: 317.347 (thresh: 195.678) -- 'pr' t: 0.000 mm n: (4.237 mm, 25 km), (21.692 mm, 26 km), (0.102 mm, 40 km), (7.161 mm, 44 km) TA00061 2023-07-06 | score: 174.822 (thresh: 80.285) -- 'pr' t: 0.000 mm n: (0.0 mm, 23 km), (12.19 mm, 34 km), (9.138 mm, 35 km) TA00110 2023-07-06 | score: 246.050 (thresh: 190.539) -- 'pr' t: 0.034 mm n: (21.692 mm, 49 km), (4.999 mm, 117 km) TA00140 2023-07-06 | score: 118.721 (thresh: 94.060) -- 'pr' t: 31.055 mm n: (5.92 mm, 35 km), (44.064 mm, 39 km), (0.0 mm, 68 km) TA00141 2023-07-06 | score: 307.212 (thresh: 126.729) -- 'pr' t: 5.920 mm n: (31.055 mm, 35 km), (44.064 mm, 56 km), (2.584 mm, 66 km), (26.76 mm, 70 km) TA00274 2023-07-06 | score: 193.030 (thresh: 107.907) -- 'pr' t: 44.064 mm n: (31.055 mm, 39 km) TA00587 2023-07-06 | score: 264.657 (thresh: 134.554) -- 'pr' t: 0.000 mm n: (9.138 mm, 12 km), (12.19 mm, 20 km), (0.0 mm, 23 km) -------- Total 'failure' count: 143 Total record count: 1132 Job history table record count: 77576 Scoring job record table record count: 302 -----------------------------------------------------------
participants (3)
-
Dietterich, Thomas -
Slater, Michael -
slater＠oregonstate.edu