ITS Service Status Report

Scheduled Maintenance — Resolved

Locker Large File Service File System Upgrade.

Services Affected: Research Storage, Locker Research File Service, Data Den Research Archive.

Start Time: 06/27/2022 8:00 am

Maintenance Completed: 06/30/2022 11:13 am

Issue Symptoms: Outage

Why this is happening 

Support for GPFS version 4 is set to expire soon on the hardware platform we have chosen, and we are required to upgrade.  This upgrade will require each piece of hardware on the Locker File Storage Service to be shutdown and upgrades installed on all of them.    We plan to update GPFS on the Locker service over the course of four days as the number of components that need to be upgraded is extensive across the entire service.   Data Den will be down for the period of the outage because a component of Data Den (the disk cache) is dependent on the Locker File System to be online and sharing volumes. As such, we will also take some time to perform OS updates to Data Den hardware during the update period.  

 

Update 10pm 6/27: 

We have nearly finished the first pass of upgrades.  We will finish that tomorrow and begin the second upgrade.  All firmware updated today.     

Update 7pm 6/28:

The Second pass of upgrades are complete.  We have a number of outstanding minor issues that we are working through at this time, and management servers to upgrade tomorrow.

Update 11pm 6/29: 

We have worked through most of the outstanding issues, and will be cleaning up and running verification stteps tomorrow on 6/30.

Update 11am 6/30:

All of the verification checks have passed, and Locker and Data volumes are available for service. This was a very complex and time-consuming upgrade, and we thank everyone for their patience. 

Who is Impacted? Users of Locker and Data Den

Next Update: June 29, 7pm

Technical Details

Service Type: Production

Comments:

We upgraded GPFS to the current version available and updated the firmware for controllers, enclosures and tape servers to the newest version.  

Report Additional Impacts

Contact the ITS Service Center for more information or to report additional impacts.