Re: [Rainqc-jobman] time of day for when we run the daily RainQC scoring
Hi everyone, It seems that data is taking longer to get from the field into the TAHMO system than in the past. I get the sense that there is something of an inflection point around 4-6 hours after UTC midnight. In the past, most data was in by UTC 4am. At this inflection point, there is a 5-8% (absolute) jump in data completeness, say from ~35% to ~40-43%. And, if one waits longer, say until closer to the end of the day, completeness will jump up another 1-3% percentage points. Some time ago, I changed from running the jobs at UTC 4am to UTC 5am to help bias the jobs to be on the good side of the inflection point. I'm guessing changing to running the jobs a little later, say UTC 6am-8am, will offer further improvements in day-to-day data completeness. If we make this change and run the jobs 1-3 hours later to be more consistently on the good side of the inflection point, are there any issues/problems that might arise from this? At the moment, EAT is UTC + 3, e.g., UTC 5am is EAT 8am. Please respond if you know of any issues, have questions, have any preferences on when the reports come out in the day, etc. Thank you, Michael
[This email originated from outside of OSU. Use caution with links and attachments.] Hello Michael, I think this might partially still be because of the west-Africa internet issues after their undersea cables got damaged. At least the field personal I'm talking too still has issues due to this on a daily basis, so I guess our stations too. For now, I guess changing to 8AM UTC is fine. If the models get improved further and we propagate actual error flags we might want to stick with 5AM for the last model runs since that's the start of working day in EAT. Kind regards, Rick ________________________________ From: Rainqc-jobman <rainqc-jobman-bounces@engr.oregonstate.edu> on behalf of Michael Slater via Rainqc-jobman <rainqc-jobman@engr.oregonstate.edu> Sent: Tuesday, April 9, 2024 7:33 AM To: rainqc-jobman <rainqc-jobman@engr.oregonstate.edu> Cc: Michael Slater <slater@baidala.com> Subject: Re: [Rainqc-jobman] time of day for when we run the daily RainQC scoring Hi everyone, It seems that data is taking longer to get from the field into the TAHMO system than in the past. I get the sense that there is something of an inflection point around 4-6 hours after UTC midnight. In the past, most data was in by UTC 4am. At this inflection point, there is a 5-8% (absolute) jump in data completeness, say from ~35% to ~40-43%. And, if one waits longer, say until closer to the end of the day, completeness will jump up another 1-3% percentage points. Some time ago, I changed from running the jobs at UTC 4am to UTC 5am to help bias the jobs to be on the good side of the inflection point. I'm guessing changing to running the jobs a little later, say UTC 6am-8am, will offer further improvements in day-to-day data completeness. If we make this change and run the jobs 1-3 hours later to be more consistently on the good side of the inflection point, are there any issues/problems that might arise from this? At the moment, EAT is UTC + 3, e.g., UTC 5am is EAT 8am. Please respond if you know of any issues, have questions, have any preferences on when the reports come out in the day, etc. Thank you, Michael
participants (2)
-
Michael Slater -
Rick Hagenaars