Hi Jason,

Asking for the definition of a job is a great question to start with.

A job is scoring an anomaly detection model for a single station and a single day and recording the results in the TAHMO system.

Our current anomaly detection approach is based on the aggregated precipitation data for a day. Hence, scoring a model works on this same granularity of data, e.g., the aggregated precipitation data of a single day. For the most part, a “day” is defined by the station sensor data that we receive as “midnight-to-midnight-minus-5-minutes”, e.g., 12:00AM to 11:55PM, UTC.

For instance, let’s say we have an anomaly detection model for station TA00642. A job might then consist of asking RainQC to compute an anomaly score for this station for a particular day, say 4 July 2022, and then storing these results in the TAHMO system.

Note that “storing the results in the TAHMO system” only happens immediately when scoring succeeds (e.g., we get either a data is consistent or data is inconsistent result). If there is insufficient data to score the station, the job manager will retry a number of times on successive runs of the job manager. If scoring doesn’t ultimately succeed, it will then “store the results in the TAHMO system”, but with a different code that indicates insufficient data.

We’ll go over this in some detail tomorrow, modulo how much time we spend getting people going with local development.

Cheers,

Michael

From: JASON KABI <jaysonnjue3@gmail.com>
Sent: Thursday, July 7, 2022 12:17 PM
To: Slater, Michael <slater@oregonstate.edu>
Subject: Tomorrow's headlines (08/07/2022)

Hi Michael.

We will be attending the session and below is a headline we are requesting you to address.

1. A breakdown of the daily status email with reference to various elements that make up the whole QC architecture. (Main question: What is a job?)

Kind regards

Jason Kabi