UniSuper private cloud, secondary systems removed due to “rare” Google Cloud “problems” – Cloud

The week-long outage of UniSuper’s systems is due to a “combination of rare issues in Google Cloud” that together caused misconfiguration during resource provisioning and triggered a secondary software error.

The industry retirement fund said last week that an “isolated” issue in Google Cloud was the reason its online services were offline.

The cloud provider has now given its own explanation for the outage: that a series of cascading “issues” led to problems with UniSuper’s primary and secondary systems.

“The outage of UniSuper services was caused by a combination of rare issues in Google Cloud that resulted in an inadvertent misconfiguration during provisioning of UniSuper’s private cloud, triggering a previously unknown software bug that affected users. UniSuper secondary systems,” a Google Cloud spokesperson said. news.

“This was an unprecedented event and steps have been taken to ensure this issue does not happen again.”

The exact combination of issues that led to the misconfiguration and its flow effects is still unclear, nor is the architecture of the UniSuper environment.

The fund has only moved many of its workloads to Google Cloud Platform over the past year, having previously split them between Azure and two of its own data centers.

It is known to use the Google VMware Engine (GCVE) managed service on GCP; Since I was already running workloads on VMware virtual machines, this made the cloud migration effectively a rehosting exercise.

UniSuper managing director Peter Chun said yesterday that restoration work would likely begin on Thursday this week.

“The progressive restoration of services to members will begin on Thursday, May 9,” he said.

“Please note that some services will remain limited as we continue restoration.

“Services we hope to have online in some capacity will include the ability to log into online services, access mobile apps and view balances, initially starting Monday, April 29, 2024.

“As trading and investment has continued as normal, this will be reflected in (customers’) balances once our systems have been restored.”

In an updated FAQ, UniSuper suggested that it could rely on other unspecified cloud systems to operate at some capacity, as well as for data logging.

“Google Cloud is not the only cloud service provider that uses UniSuper, and this planning has ensured our ability to restore services and minimize data loss,” the fund said.

The Google Cloud spokesperson said its teams continued to “work around the clock with UniSuper to fully remedy the situation, with the goal of progressively restoring services as soon as possible.”