cluster-users

Download

cluster-users@lists.engr.oregonstate.edu

December 2023

1 participants
2 discussions

CoE HPC News, December 21 2023: Update on Winter maintenance and EL8/EL9 upgrade
by Yelle, Robert Brian 21 Dec '23

21 Dec '23

HPCC users, Update on maintenance activities and EL8/EL9 upgrades The cluster is back online but at reduced capacity while maintenance and upgrades proceed on the rest of the cluster. Additional resources will continue to be restored as maintenance and/or upgrades to EL8 or EL9 are completed. Also, due to a large number of research groups facing deadlines in the coming weeks, the upgrade period will be extended and several systems will remain on EL7 through February 5. If you have any questions or concerns, let me know. For up-to-date status on the cluster, check out the link below: https://it.engineering.oregonstate.edu/hpc/hpc-cluster-status-and-news<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fit.engine…> Happy Holidays! Rob Yelle HPC Manager

1 0

CoE HPC News, December 1 2023: Winter maintenance, EL8/EL9 upgrade, DGX H100 status
by Yelle, Robert Brian 01 Dec '23

01 Dec '23

HPCC users, Winter maintenance/OS upgrades The cluster will operate at reduced capacity throughout the month of December through mid-January while cluster nodes are upgraded to either EL8 or EL9 linux. I will post a tentative upgrade schedule sometime next week, and will coordinate with partition group owners to help minimize the impact of these upgrades. In addition to the OS upgrades, regular maintenance activities will be performed during this time, e.g.: OnDemand HPC portal updates Slurm update and configuration changes Nvidia driver updates Infiniband updates BIOS and firmware updates as needed Miscellaneous hardware maintenance as needed The entire cluster will be offline after Finals week, starting Monday, December 18th at 1pm, and will remain offline until approximately Wednesday the 20th at 3pm. Jobs scheduled to run into that offline period will remain pending with the message “ReqNodeNotAvail, Reserved for maintenance”. If you wish for your Slurm job to start and finish before the offline period begins, you will need to adjust your time limit accordingly, e.g. to change to 2 days do: scontrol update job {jobid} TimeLimit=2-00:00:00 Alternatively, you can cancel your pending job, adjust your walltime (using the --time option) and resubmit. Status of Nvidia DGX H100 systems I want to thank everyone who has given me feedback regarding the new DGX systems. A GPU issue came up which required that I open a ticket with Nvidia to troubleshoot. The nodes were then taken offline, then reinstalled and reconfigured. The GPU issue has now been addressed, so pytorch and other CUDA applications should work. Also, the automount problem appears to be resolved, so users can access directories in /nfs/hpc/*. I anticipate that the HPC portal/vncserver will be available for these hosts sometime next week, so stay tuned. However, ssh access is still not available. The dgxh partition is back online and the testing period resumed while other issues continue to be worked out. Feel free to give them another shot, and please continue to report any further issues to me. If you have any questions or concerns, let me know. For up-to-date status on the cluster, check out the link below: https://it.engineering.oregonstate.edu/hpc/hpc-cluster-status-and-news<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fit.engine…> Have a nice weekend, Rob Yelle HPC Manager

1 0