ITS Service Status Report

Unscheduled Outage — Resolved

Print Service Outage for Application Upgrade

Services Affected: MiWorkspace, MPrint, Campus Computing Sites

Start Time: 06/16/2017 11:00 pm

Service Restored: 06/18/2017 10:30 pm

Issue Symptoms: Outage

The MiWorkspace and Campus Computing Sites printing services will be down while the PaperCut application is upgraded to version 17.0.7

Card reader authentication, Follow-Me, and printing will be unavailable.

Technical Details

Service Type: Production

Server Name: thedurant.miprint.it.umich.edu

Comments:

10:30 pm - Database import has completed, and the server is back online. We have verified printing and card swipe authentication is working again. We believe all services are functional once again.

3:30 pm - A couple tables still remain to be imported.

10:50 am - A handful of tables still remain to be imported. Current expectation is for the import to take another 3-4 hours, after which we can bring the service back online.

6:00 pm - Due to a problem with print job redirection, Sites users should use the MPrint web application to print until this issue is resolved. MiWorkspace users should not be impacted, and can continue printing using installed queues.

Database import is currently expected to complete on Sunday morning.

5:00 pm - We've re-enabled the MPrint web application, and believe print jobs are processing ok through the application. The database import is continuing, and is currently expected to finish Sunday morning.

3:00 pm - Database import is continuing without issue

1:00 pm - Database import continues to progress. There are 35 tables in total that need to be restored, 10 are complete, and the 11th is well under way. Many of the remaining tables are small and should complete quickly, but the largest couple still remain, and are expected still to take hours to finish.

11:00 am - Changing status to unscheduled outage. Printing via the MPrint website and through Follow Me are currently down for all MPrint customers including Campus Computing Sites and MiWorkspace. Printing directly from a Campus Computing Sites or MiWorkspace computer to an installed printer are not impacted. A problem occurred during a planned upgrade last night that corrupted the MPrint accounting database. Restoration efforts are underway but it is expected the MPrint web and Follow Me services will remain down for several more hours. The next update on service restoration will be posted here around 1pm.

6:30 am: import is still running albeit very slowly. It looks like this will take many more hours still, just to get back to our initial version, 15.3. Upgrade is postponed to a later date.

3:30 am: after an initially failed DB schema upgrade, I spoke to the database team to remedy a tmp space issue and tried again. The schema upgrade was partially successful, which made it impossible to fully complete an upgrade to any version of PaperCut, since it was sitting between versions 39 and 40. Decided to kill the DB to erase the discrepancy and import data from PaperCut's export from last night.

Groups Notified:

  • euc.print

Report Additional Impacts

Contact the ITS Service Center for more information or to report additional impacts.