2025-02-28 - Scheduled Maintenance & Important Changes at Mjolnir from Monday and ahead

Created by Bent Petersen, Modified on Fri, 28 Feb at 12:23 PM by Bent Petersen

Dear all,

This is a reminder about this weekend’s scheduled maintenance, details below, and a heads-up about upcoming changes to how you work at Mjolnir.

More details will be announced over the weekend, but you should anticipate modifying your job submission scripts in order to use Mjolnir.

This weekends maintenance should hopefully fix all the challenges we previously have had with the filesystem.

Friday, February 28

17:00 – All compute jobs will be terminated at Mjolnir.
18:00 - 20:00 – The storage team will restrict access to projects and datasets. If unexpected issues arise, disruptions may extend into the weekend.

To avoid data loss, please save and close any active Tmux, Screen, SFTP, or other sessions before the maintenance window.

Saturday, March 1

All day – Expect disruptions to projects, SSH access, and other services.
I will be implementing a new mandatory Quality of Service (QoS) as part of future improvements. This will require you to include a new argument when submitting jobs.

I will send a separate email about this over the weekend or on Monday, depending on the maintenance progress. The new QoS will only be enforced after maintenance is completed.
Sunday, March 2

13:00 - 22:00 – All nodes will be shut down for upgrades. We aim to bring the cluster back online by 22:00, though delays are possible.

Monday, March 3 & Beyond

The cluster is expected to be available for login.
During the maintenance window, nodes will go up and down intermittently, so I strongly advise against working over the weekend.
User Access & Data Transfer Updates

Starting Monday, I will begin reviewing the list of users who have completed this form:

https://airtable.com/app0YI6HJDQGmtIru/pagOoCi5v6Z8siKml/form

So far, 65 out of 167 users have filled out the form. If you haven’t done so, you will not be able to work on Mjolnir until the form has been completed.

Here’s what to expect based on your submission status:

Data Fully Moved

I will check /projects/mjolnir1 for each user who filled out the form.
If I confirm all your data has been moved, you will receive an email, and compute access will be restored.

File Transfer Queue

If you requested file transfer access, you will immediately be granted access to move your data.
All users should transfer by Monday If I don’t see a reduction in your storage by Thursday 6th March your access to Mjolnirgate will be revoked.

Please note that the Rsync queue will be renamed over the weekend, more info will be sent.

Data Migration by Bent

Once the system is available, I will begin moving user data. You will be notified by email when your data is being transferred. Do not modify your files in the meantime.
Transfers may take several days, depending on data volume and available bandwidth. Good time for you to catch up on papers!

Users Who Have Not Submitted the Form by Wednesday, March 5

Your MjolnirGate access will be revoked Thursday 6 March until your data is moved to your new project location.
After your access has been revoked I will start moving your data.
Once your data has been migrated, your MjolnirGate access will be reinstated.

After a user is reinstated for compute access, their access to /projects/mjolnir1 will be permanently removed.
If you have any questions, please reach out before the maintenance begins.
@CMEC users, I am aware that you do not have a new projects folder yet. This is due to logistical challenges and we will fix that next week. Do not worry. For any concerns please reach out to me directly.
I will be around for the Friday bar at HoloGenomics for a beer or two and you are welcome to reach out if you have any questions.

Best wishes and sorry for the inconvenience I know this may cause.