Loading Data as Fast as Possible

Loading Data as Fast as Possible

One of the issues you will face with implementing MDM solutions on premises is whether you go with the vendor’s recommendation for hardware specs and the number of servers required. Obviously due to budget constraints, you may not be able to afford the gold plated solution, especially if this is a new install and you are just starting your mass deployment. Being able to grow incrementally is not usually possible where you need a base set of hardware.

Typical Hardware Required

So let’s say you provide two application servers, two database servers, two web servers and two communication servers for redundancy on all levels (not to mention the switches, power supplies, firewalls and the numerous other components necessary). You may find that most of the time the application, web and communication servers are doing practically nothing, except for when you have a scheduled retrieving and loading of data from your deployment of meters.

What Happens if Data Arrives Late?

So you’ve now virtualised your servers to share resources so that for the other times of the day when you’re not doing a scheduled load, the underlying hardware could be doing something more useful. However what happens if the scheduled load of data is late? Which virtual server then gets priority – the one for your MDM software or for some other application?

What about if a scenario happens where your meter vendor has a major issue with their firmware upgrade and means that you stop being able to retrieve interval data on a daily basis, but three months later you manage to obtain the backlog of data and now want to load all of it (i.e. it is really late)? How do you manage to load all that data as quickly as possible so it doesn’t impact your normal operations with only a couple of each type of virtual servers?

Big Data = Big Problem

When numbers start to rapidly increase and you have 100,000+ meters deployed, you need to be able to load that mass of data quickly, while filtering out potentially any unwanted events (e.g. momentary outages) and have standard import processes to ensure all data loaded is correctly validated and stored.

With our Fluid MDM platform, you control the size of the service you receive from us which enables you to grow incrementally as your deployment and business expands.