ITS Service Status Report

Service Degradation — Service Restored

BusinessObjects - Service Patched

Services Affected: Analytics and BI

Start Time: 11/18/2022 2:34 pm

Service Restored: 01/30/2023 5:03 pm

Issue Symptoms: Degradation

Update 1/30/2023: 

We are happy to report that the system is operational and the error messaging issue is resolved. 

Please reach out to 4help@umich.edu if you encounter any issues with Business Objects, even if you have previously reported these. This will help us monitor and manage the system's stability. 

Thank you for your patience as we've worked to resolve this. 

Update 1/29/2023: We have just completed patching the system and testing. 

11/18/2022: Issues are intermittent in some cases and pervasive in others. Users may experience issues while refreshing reports, modifying existing reports, and may see incorrect data exported to excel that doesn’t match the screen. 

In many cases, clearing your browser cache by following the steps mentioned in this document  - in the exact sequence - has resolved the errors. If you are encountering issues and have not yet cleared your cache with the directions, we advise you to do this to see if this fixes your issue. 

Cache-clearing has not resolved every error. ITS has been in on-going conversations with SAP (the vendor) to triage these larger issues. We have taken a number of steps in the past week based on SAP’s recommendations. We are working with SAP to continue to correct errors and have a High priority case open with them. 

If you are encountering errors and have already cleared your browser cache, please do the following:

  1. Screenshot the error message (if applicable)

  2. Create a 4help ticket (either through the TDX portal or by emailing 4help@umich.edu). 

  3. Let us know the Report Name and location (e.g. “PG Balance Report” in my personal folder;  “FN01 RevExp Project Grant to Fund Code” in UM-Maintained/Finance).

  4. Attach any screenshots and send to 4help.

We appreciate your patience while we continue to troubleshoot and adjust the environment to make it more operational.

 

 

List of Errors and Issues

Intermittent issues include:

  • Error messages when refreshing or editing a document like 

    • “Document not found”, 

    • “You do not have access to one or more data providers”, 

    • “The resource of type with identifier does not exist”, etc. 

    • “An internal error occurred while calling ‘processDPcommandsEx API” (or similar internal error type messages)

Larger scale-issues include:

  • When running the same report multiple times, later iterations will export data into excel incorrectly (data in Excel won’t match what you see on the screen)

  • When running the same report multiple times, you will get error messages and be unable to run the report again

  • Queries you create, run, and save to BO disappear from the file repository (you can’t access the report again and it’s missing in folders and searches)

  • Editing variables and saving the document, then reopening the report and the variables are in their original state

Who is Impacted? BusinessObjects Users

Technical Details

Service Type: Production

Comments:

1/29/23 - Patching completed and testing completed. No errors or issues were encountered. We will monitor production servers and ticket volume during the upcoming week. 

1/9/23 - We have confirmed the decision to upgrade BO Production to Patch8 on 1/28/2023 at 7 PM and anticipate completing the maintenance by 1/29/2023      7 AM. 

1/3/23 - We are working with a specific report that is not working as expected in Patch8. This new patch resolved all issues found in the current production version Patch6.

12/19/22 - Patch 8 introduced an issue that could not be resolved\work around created.  DISC holding on decision of how to proceed until after the season break.

12/6/2022 - We received a Patch update from Vendor and are installing it in our Testing system Sandbox today. We will test the Patch and update Production soon.

12/02/2022: We have rolled out a series of changes that will eliminate some issues. We continue to work with the vendor to determine root cause. We are working after hours to gather log files and test specific error cases.

11/29/2022: We continue to work with the vendor to determine root cause. We are working after hours to gather log files and test specific error cases. 

11/22/2022: We continue to work with the vendor to determine root cause.

Report Additional Impacts

Contact the ITS Service Center for more information or to report additional impacts.