- Rainqc-jobman - Engineering Mailing Lists

RainQC Job Manager daily report 2022-07-11
by slater＠oregonstate.edu 11 Jul '22

11 Jul '22

1 0

RainQC Job Manager daily report 2022-07-11
by slater＠oregonstate.edu 11 Jul '22

11 Jul '22

1 0

RainQC Job Manager daily report 2022-07-11
by slater＠oregonstate.edu 11 Jul '22

11 Jul '22

1 0

RainQC Job Manager daily report 2022-07-10
by slater＠oregonstate.edu 10 Jul '22

10 Jul '22

1 0

RainQC Job Manager daily report 2022-07-09
by slater＠oregonstate.edu 09 Jul '22

09 Jul '22

2 1

RainQC Job Manager daily report 2022-07-08
by slater＠oregonstate.edu 08 Jul '22

08 Jul '22

1 0

reminder: please fill in the meeting time poll for next week
by Slater, Michael 08 Jul '22

08 Jul '22

Hi everyone, When you have a moment, please fill in your availability for next week's hand-off meetings. Tema and I have put in our availability, so we basically need to know who on the Kenyan team can make MWF 8am (Pacific). Input your availability here: http://whenisgood.net/7gwhy88 View results here: http://whenisgood.net/7gwhy88/results/ye3ecar Thank you kindly, Michael

1 0

Re: [Rainqc-jobman] Tomorrow's headlines (08/07/2022)
by Slater, Michael 07 Jul '22

07 Jul '22

Hi Jason, Asking for the definition of a job is a great question to start with. A job is scoring an anomaly detection model for a single station and a single day and recording the results in the TAHMO system. Our current anomaly detection approach is based on the aggregated precipitation data for a day. Hence, scoring a model works on this same granularity of data, e.g., the aggregated precipitation data of a single day. For the most part, a “day” is defined by the station sensor data that we receive as “midnight-to-midnight-minus-5-minutes”, e.g., 12:00AM to 11:55PM, UTC. For instance, let’s say we have an anomaly detection model for station TA00642. A job might then consist of asking RainQC to compute an anomaly score for this station for a particular day, say 4 July 2022, and then storing these results in the TAHMO system. Note that “storing the results in the TAHMO system” only happens immediately when scoring succeeds (e.g., we get either a data is consistent or data is inconsistent result). If there is insufficient data to score the station, the job manager will retry a number of times on successive runs of the job manager. If scoring doesn’t ultimately succeed, it will then “store the results in the TAHMO system”, but with a different code that indicates insufficient data. We’ll go over this in some detail tomorrow, modulo how much time we spend getting people going with local development. Cheers, Michael From: JASON KABI <jaysonnjue3(a)gmail.com> Sent: Thursday, July 7, 2022 12:17 PM To: Slater, Michael <slater(a)oregonstate.edu> Subject: Tomorrow's headlines (08/07/2022) Hi Michael. We will be attending the session and below is a headline we are requesting you to address. 1. A breakdown of the daily status email with reference to various elements that make up the whole QC architecture. (Main question: What is a job?) Kind regards Jason Kabi

1 0

RainQC Job Manager daily report 2022-07-07
by slater＠oregonstate.edu 07 Jul '22

07 Jul '22

1 0

POLL: job manager hand-off meetings -- week of 11 July 2022
by Slater, Michael 06 Jul '22

06 Jul '22

Hi everyone, We got off to a good start this week with the job manager hand-off, spending almost 2.5 hours on Tuesday morning getting people going on their local machines. We learned some interesting things and hopefully with another session or two, everyone will be up and running on their machines. We'll also be covering how the daily status email gets generated and what it means, and (eventually) cloud deployment. We'll meet again Friday this week, about the same time, 8-10AM Pacific. As I'd like to accelerate the job manager hand-off, if you are available for additional hand-off days beyond Wednesdays next week, please let me know via the below poll. Input your availability here: http://whenisgood.net/7gwhy88 View results here: http://whenisgood.net/7gwhy88/results/ye3ecar There may be some contention between hand-off times for the job manager and RainQC itself, but Tema and I will work out what we can do on which days. The general idea at the moment is for us to get through getting everyone going on their machines with the job manager and then Tema will help everyone get going with RainQC development, so we are not trying to get two development environments going at once. Thanks, Michael

1 0