September 2022 - Rainqc-jobman - Engineering Mailing Lists

RainQC Job Manager daily report 2022-09-22
by slater＠oregonstate.edu 22 Sep '22

22 Sep '22

1 0

RainQC Job Manager daily report 2022-09-21
by slater＠oregonstate.edu 21 Sep '22

21 Sep '22

1 0

RainQC Job Manager daily report 2022-09-20
by slater＠oregonstate.edu 20 Sep '22

20 Sep '22

1 0

RainQC Job Manager daily report 2022-09-19
by slater＠oregonstate.edu 19 Sep '22

19 Sep '22

1 0

RainQC Job Manager daily report 2022-09-18
by slater＠oregonstate.edu 18 Sep '22

18 Sep '22

1 0

RainQC Job Manager daily report 2022-09-17
by slater＠oregonstate.edu 17 Sep '22

17 Sep '22

1 0

RainQC Job Manager daily report 2022-09-16
by slater＠oregonstate.edu 16 Sep '22

16 Sep '22

1 0

RainQC Job Manager daily report 2022-09-15
by slater＠oregonstate.edu 15 Sep '22

15 Sep '22

1 0

Re: [Rainqc-jobman] RainQC Job Manager daily report 2022-09-14 / job manager hand-off update
by Slater, Michael 14 Sep '22

14 Sep '22

This second report was a test run using the new PostgreSQL cloud database. Performance seems a little better than the previous 'compose' cloud database, but it's too early to say anything definitive. There were some small code changes required to get things working. I'll check the code in later this week. FYI, these test run results were not pushed into the TAHMO system -- even though it seems with the additional weather data that had arrived in the system, we were able to find a couple more anomalies for 2022-09-13. I cannot make the meeting today, but should be available for a hand-off meeting arbitrarily next week, or Friday this week if the Kenyan dev team happens to be available. Cheers, Michael From: slater(a)oregonstate.edu <slater(a)oregonstate.edu> Sent: Wednesday, September 14, 2022 3:20 AM To: rainqc-jobman(a)ENGR.ORST.EDU Cc: Slater, Michael <slater(a)oregonstate.edu> Subject: RainQC Job Manager daily report 2022-09-14 Current UTC date: 2022-09-14 -> scoring models for previous day --------------------------------------------------------------------------------- Daily Model Data Completeness Check: data completeness 50% | complete models: 151 of 251 (60.16%) data completeness 60% | complete models: 151 of 251 (60.16%) data completeness 70% | complete models: 151 of 251 (60.16%) data completeness 75% | complete models: 150 of 251 (59.76%) data completeness 80% | complete models: 148 of 251 (58.96%) data completeness 85% | complete models: 143 of 251 (56.97%) data completeness 90% | complete models: 140 of 251 (55.78%) data completeness 95% | complete models: 138 of 251 (54.98%) data completeness 100% | complete models: 131 of 251 (52.19%) ---------------------------- station status | total: 286, delayed: 114, offline 24h: 36, offline week: 27 | battery, min: 0, max: 100, mean: 63.09, std dev: 26.28 | battery, common values: [(100, 188), (0, 37), (91, 3), (88, 3), (44, 3)] | battery <= mean, common countries: [('GH', 21), ('ML', 9), ('KE', 9), ('TG', 6), ('TZ', 3)] ---------------------------- 58 LOW DATA (< 0.9) and 43 NO DATA weather stations impacted 105 RainQC models LOW/NO data station impact on models: [('TA00570', 14), ('TA00157', 6), ('TA00078', 6), ('TA00677', 5), ('TA00126', 5), ('TA00321', 5), ('TA00652', 4), ('TA00586', 4), ('TA00335', 4), ('TA00691', 4), ('TA00124', 3), ('TA00044', 3), ('TA00041', 3), ('TA00043', 3), ('TA00050', 3), ('TA00089', 3), ('TA00422', 3), ('TA00180', 3), ('TA00533', 3), ('TA00655', 3), ('TA00276', 2), ('TA00308', 2), ('TA00101', 2), ('TA00102', 2), ('TA00257', 2), ('TA00487', 2), ('TA00203', 2), ('TA00262', 2), ('TA00265', 2), ('TA00271', 2), ('TA00307', 2), ('TA00339', 2), ('TA00471', 2), ('TA00466', 2), ('TA00664', 2), ('TA00014', 1), ('TA00031', 1), ('TA00039', 1), ('TA00092', 1), ('TA00128', 1), ('TA00254', 1), ('TA00201', 1), ('TA00219', 1), ('TA00236', 1), ('TA00286', 1), ('TA00279', 1), ('TA00392', 1), ('TA00277', 1), ('TA00290', 1), ('TA00336', 1), ('TA00350', 1), ('TA00351', 1), ('TA00362', 1), ('TA00333', 1), ('TA00382', 1), ('TA00434', 1), ('TA00493', 1), ('TA00702', 1)] ----------------------------------------------------------- Processed daily jobs for UTC date: 2022-09-13 Start time: 2022-09-14T10:00:38+00:00 End time : 2022-09-14T10:20:16+00:00 Elapsed time HH:MM:SS: 0:19:39 --------------------- Before job processing job table stats: Total 'success' count: 0 Total 'failure' count: 0 Total record count: 0 Job history table record count: 0 Scoring job record table record count: 0 --------------------- After job processing job table stats: Total 'success' count: 108 (flag=2 count: 5) (flag=1 abstentions: 2) | 'success' count for 2022-09-13: 108 (flag=2 count: 5) | 'success' count for 2022-09-12: 0 (flag=2 count: 0) | 'success' count for 2022-09-11: 0 (flag=2 count: 0) | 'success' count for 2022-09-10: 0 (flag=2 count: 0) | 'success' count for 2022-09-09: 0 (flag=2 count: 0) | 'success' count for 2022-09-08: 0 (flag=2 count: 0) | 'success' count for 2022-09-07: 0 (flag=2 count: 0) Anomalies (flag=2): TA00062 2022-09-13 | score: 771.539 (thresh: 90.000) -- 'pr' mm t: 0.051 [0.255, 43.258] TA00064 2022-09-13 | score: 372.608 (thresh: 140.097) -- 'pr' mm t: 2.025 [3.012, 43.258] TA00127 2022-09-13 | score: 122.218 (thresh: 113.304) -- 'pr' mm t: 0.000 [0.0, 0.0, 0.034, 5.406] TA00281 2022-09-13 | score: 205.156 (thresh: 21.775) -- 'pr' mm t: 0.255 [43.258] TA00567 2022-09-13 | score: 113.622 (thresh: 90.000) -- 'pr' mm t: 0.000 [0.0, 0.0, 0.034, 5.406] -------- Total 'failure' count: 0 Total record count: 251 Job history table record count: 0 Scoring job record table record count: 1 -----------------------------------------------------------

1 0

RainQC Job Manager daily report 2022-09-14
by slater＠oregonstate.edu 14 Sep '22

14 Sep '22

1 0