2+2000+3000 = 1 big challenge

thousands-of-app-in-a-bank

You read it right.

2 DCs + 2,000 servers with 3,000 applications are going into a new Data Center in three years time, and the man/woman to do it is yet to be found.

I have had the unfortunately experience of network interruption that caused slow and unacceptable access to hundreds (my then estimate) of online application in a large Enterprise data center, but then again we were an internal shared co-location data center so we only count the projects and departments but never the number of application systems.

 

In the above picture is an advertisement that calls for a data center migration project manager, and it has been repeatedly put up for more than 6 months and I have seen the indicated salary range gone up (from 10k SGD per month to now 15k SGD). The scale and complexity is also fully spelled out, as previously their advert did not indicate two data centers moving into a new one, and 2,000 servers with 2,000 application systems, and to move all in (I presume by phases, it will be nearly mission impossible if all at once) by 2020. Don’t forget, system change and new system and services will not stay frozen during this period, while new system may still be added to the current data centers given the inter-dependencies of system and data required to introduce these new systems and services. Hopefully, no IP address change for any of the system is required, oh that is only one out of many possible things to consider for such a move.

I had one data center move done in the mid 1990s with the enterprise having one mini-computer sytem as the one and only mission critical system, we went beyond the planned downtime window of 24 hours by additional 30 hours because the new site’s telecommunication cables are digital compared to the old site which was analog, and our analog modem was unable to work and we had to bring in new ones while the migration took place on a Sunday when the vendor’s warehouse are closed.

I had on occasions when talking to data center facility owners, sales people and fellow consultants about the mission critical nature of IT for most of today’s enterprises and mentioned hundreds of applications are in use on average by medium to large enterprises.

098733b

3,000 applications is one big number. I hope only 10% of them are mission critical and the entire application system portfolio are prioritized and inter-dependencies already mapped out.

When a bank data center ran into problem (see reference section below), what we see externally is ATM being down, and counter staff may switch over to back-up system and service became slower. What really happen will involve lots more effort to bring the critical applications back to service.

What makes me wonder though, shouldn’t the bank have identified such a role and bring him in earlier in the process of decision making to have a new data center, and this person or better a team of data center migration experts be better in all sorts of way rather than bring someone from outside and who is to manage any knowledge and therefore mitigate any migration risks? I am pretty sure that the local financial regulator will dive in to audit and assess the bank’s migration plan.

Anyway, I have learnt a number (1,000s of servers, applications) for a Singapore bank is probably typical.

Best of luck to their data center migration.

Reference:

  1. http://www.datacenterknowledge.com/archives/2013/12/16/year-downtime-top-10-outages-2013/
  2. https://www.theregister.co.uk/2017/01/13/lloyds_bank_in_talks_to_outsource_bit_barns_to_ibm/
  3. https://www.linkedin.com/pulse/20140616192008-655694-best-practices-for-data-center-migration
Advertisements
2+2000+3000 = 1 big challenge

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s