Fiona Hamilton, Vice President EMEA Operations Volante Technologies
The Path to a Successful Data Integration Project Part 1
In the first article in this three-part series we looked at the business and technical drivers behind the business case for implementing a common data integration architecture. In this second part, we examine the process of deciding how best to achieve that end-goal by selecting the most appropriate architecture.
Define the purpose of your project
It is essential that from the outset there is a clear understanding of reasons for embarking on the project, which will ordinarily affect how the business case is presented for budgetary approval. There are many drivers to the initiation of any integration project whether it is strategic across the organization or tactical within a line of business. For example:
- to meet a new regulatory requirement or infrastructure such as Dodd-Frank, EMIR, T2S or SEPA
- to be able to use a new service such as T2S or DTCC ISO 20022 corporate actions
- to be able to process a new financial instrument such as an OTC derivative product
- to replace out-of-date technology that is difficult and expensive to maintain, for example when message standards change as a result of maintenance cycles or regulatory change, thereby cutting operating costs
- to increase customer satisfaction or improve the time and cost involved in adopting new channels or customers, which also allows potential revenue streams to come online sooner
- to improve overall straight-through-processing (STP) efficiency, which in turn improves customer satisfaction and reduces risks such as the cost of failed settlement and the cost of manual exception processing
- to facilitate geographical or application/data consolidation thereby reducing operating overheads and facilitating better access to operational/customer information
- to implement cutting edge “big data” technology such as Apache Hadoop that greatly increases customer focused decision making and sales and marketing
The bottom line is that if you don’t understand why you are embarking on such a major undertaking and are unable to at least quantify the benefits in terms of cost savings, operational efficiency, increased revenues or customer satisfaction, then budget sign off is unlikely to be forthcoming.
Define what is considered a successful outcome and how you will measure success
It is essential that not only is there a clear vision of what the project is going to achieve, it is also vital to define from the outset, what the criteria is for measuring the success of the project not only at the end-point of the delivery phase, but also on an on-going basis. The success criteria can usefully be broken into:
- Milestones – e.g. addition of a new clearing and settlement channel or central counterparties or consolidation of multiple payment channels either centrally or into regional hubs.
- Process improvements – e.g. increased STP rates and the elimination of manual processes to reduce exceptions.
- Cost and revenue – including:
- reduction of staff and overall costs from increased STP rates, reduced exceptions and reduction in IT costs (not forgetting to factor in how the released resources and cost savings can potentially be redeployed).
- ease of on-boarding new systems/banking channels: There is often a major challenge in both the costs and the timeframes to implement new applications. Savings and revenue opportunities exist when the implementation time is reduced.
- Technical benefits – from replacing legacy (old) technology, including moving to real-time mode of operation, removing the risk of maintaining out-of-date applications. This should include looking at the cost savings over time in ease of maintenance and access to appropriate skilled resources. Legacy systems often require skills that are no longer readily accessible in the market and are therefore increasing in cost.
- Improving latency – the time taken to execute a transaction or instruction, e.g. time taken to accept an equity quotation thereby enabling best execution.
- Future proofing – this phrase is often misleading as in the world of IT, there is no such thing as protecting yourself from all possible technological change. However, it is possible to mediate some of the risk by adopting a component approach whereas one element is superseded it can be replaced without affecting the other pieces, thereby reducing risk and testing costs.
- Compliance to regulation – non-compliance can result in loss of business as customers go elsewhere to trade with other compliant parties. In other cases it can lead directly to fines from regulatory bodies.
There may be other measurement criteria that apply to an individual organization but the best practice is not only to try to quantify them as part of the business justification for the project, but to also embed them in the success metrics for the entire project lifecycle and beyond. Too often these criteria are forgotten when the project gets signed off and not surprisingly, often after the fact, C-level executives, when looking at the projecting costs, may lose sight of why sign off on the project was approved. Put simply, how can you prove that a programme of work was worth doing if you can’t measure the benefits on an on-going basis?
Assess the nature and types of gaps in current infrastructure
As stated previously you need a good reason for embarking on an integration project and therefore, bearing in mind what you’re trying to achieve, you need to drill down to the next level of detail to guide the design of the target architecture. Unless the organization is a completely green field site there is almost certainly much to be learned from past architectural mistakes and technology choices. It is therefore essential to conduct an honest and objective audit of how the current infrastructure would deal with the target objectives of the project. Specifically, attention should be paid to where the bottlenecks are in supporting new markets, messaging, internal process changes, on-boarding external customers, utilities or clearing venues.
These functional gaps could be a lack of real-time capabilities as current operations are batch based. It could be related to connectivity security or lack of guaranteed delivery and therefore this drives a move from file transfer technology to perhaps secure messaging. These gaps often relate to an ability to support, despite the on-going drive to standardise message formats and multiple variant message formats that are specific to individual endpoints. The maintenance of these formats, and the transformation between the myriad formats and the internal application representations of those transactions, often leads to huge on-going costs and significant inertia and indeed cost in implementing change to external communications with customers and trading parties.
Knowing what the current architecture does well and what it does badly at a detailed level, and taking into account what your target business and operational objectives are, you are then in a position to move to the next stage.
Design a candidate architecture and functionality
The key word here is “candidate”. At this stage nothing should be cast in stone. During the investigation and potential vendor selection processes it may be that improvements can be made, so it is important to keep an open mind. Whilst it is worth taking note of what your existing infrastructure excels at, it is important not to fall into the trap of simply reinventing the same thing using more modern technology.
One pitfall that is especially common at this point is in supporting the user-based workflow in terms of manual message initiation, monitoring, repair and general exception management. Many integration projects have made the mistake of trying to follow the same process flow and provide the business user with the same user interface functionality. Whilst it is important to bear their overall requirements in mind, the aim should be to try to eliminate manual user interaction throughout the transaction lifecycle to avoid introducing significant cost, latency and or risk associated with human intervention.
The architecture itself should cover the user functional units such as exception management reporting and interactive querying and also technical functional units such as management of message standards and their variants, ease of validating, transforming, enriching, routing and persisting of data or messages.
The technology approach should then be considered with respect to preference of programming language support, connectivity, service oriented architecture (SOA) security, cloud and big data support to mention just a few.
This phase is crucial and it is worth taking time to assess in detail, and previous experience is essential. Not all organizations will have internal resources with previous and up-to-date experience of integration projects, making it essential to bring in a consultant. However, make sure that they really are a subject matter expert with at least 10 years of integration project experience. It may often be the case that you require two; one on the technology side and another on functionality, such as message standards and customer or trading partner on-boarding. It cannot be stressed how important it is to invest in this phase.
This consideration of the components of a typical integration architecture is too large to do justice within the confines of this article and, given its importance, will be addressed in detail in a separate and dedicated article.
Choose the solution required
After assessing all the requirements and arriving at your candidate architecture you will come to one of four decisions:
- Do nothing – the numbers do not justify the investment
- Build your own – although in many cases this may have a good initial return on investment (ROI), beware of the significant hidden costs and risks with respect to ongoing total cost of ownership (TCO) as resources move away, technology changes, regulatory changes occur, message standards change and back-end systems are upgraded or replaced.
- Select a supplier to work with – look for a one-stop-shop to either buy off-the-shelf or build a solution for you. Remember that off-the-shelf packages may have good initial ROI but can mean flexibility is lost as the off-the-shelf product will have to accommodate many different companies’ requirements and so may not be a 100% fit to your way of doing business. However, outsourcing the custom build may still have similar issues to building your own, as the issues of continuity of resourcing are transferred to a different place.
- A combination of 2 and 3 – by acquiring appropriate component technology such as model-driven code generation development tools, enterprise service bus middleware, databases, etc., that allow your organization to build a solution that fits your exact needs, you benefit from a tailored solution that is under your control. The providers of the individual components will usually have more focus on smaller parts of the functionality and therefore the on-going resourcing, whilst never going away, is usually less of an issue.
Choose supplier(s)
In most cases the project will need to acquire technology from external suppliers and it is therefore important to assess them in many ways:
a) Geographic coverage – for example, do you need a global supplier with 24x7 support that understands message formats in many different countries, or is your business purely domestic?
b) Appropriateness of technology – as compared to the assessment of requirements from previous points
c) Type of supplier – are they a software vendor, a re-seller, a systems integrator?
d) Level of control / level of outsourcing – do you want to have total ownership of the implementation or do you want the supplier to manage the project or an independent third party expert?
e) Financial stability – of prospective supplier(s)
f) Proven methodology and track record – integration and connectivity projects are complex
g) Functional match – how close does their solution match the requirements “out-of-the-box”?
h) Ease-of-use and flexibility with regard to change – bear in mind that implementing a solution from scratch may be the initial goal, but anticipate the need to continually support internally and externally driven change
Patently the list above is not meant to be exhaustive, but the detailed requirements and architecture should result in the generation of either a Request For Proposal (RFP) which is then sent to a number of selected parties or acts as a working evaluation checklist. The former is the most traditional approach where a high level extract of requirements is sent to a larger list of potential suppliers as a Request For Information (RFI) and the responses then whittled down to a shortlist of suppliers that will be investigated further by means of sending out full RFPs. This process can be very time consuming for both the organization and the suppliers and can add costs on both sides, significantly delaying the project.
In the author’s opinion, a more fruitful approach is to court a reasonable number of potential suppliers after doing research via the internet, personal contacts in other organizations, visiting conferences and reading trade journals. Share a high level RFI that describes your project’s goals and high-level functional and technical requirements, requesting that they present back to you either in person or via a web session their proposed solution architecture, case studies of where they have implemented similar solutions and demonstrate the key areas of functional and technical support.
This approach will then allow the selection of a shortlist without having to wade through written responses. The shortlisted suppliers should then be invited to carry out on-site workshops that should be driven by the evaluation checklist, which would have all the points that an RFP would cover but can be much less verbose. Depending on the scope of the requirements, these workshops may last from 1 to 5 days. This may seem like a large investment in time, but as with any project; the more investment up front, the more chance of success.
The benefits of this approach include seeing first hand the support for the functionality, and how easily it is achieved. You will also have time to get to know not just their product, but also get a sense for the type of organization and how responsive they have been to making the workshop as applicable as possible to the stated requirements. Always bear in mind that you are potentially entering into a long-term relationship with them. If you don’t get on with them well in the sales phase it might be telling you that the marriage is unlikely to be a happy one!
The decision
As with most product buying decisions it comes down to cost vs functionality. However, it is equally true that choosing the cheapest product is often a false economy. In some cases the experience of the supplier will be far greater than your organizations’, and if they say it is going to take twice as long as you have budgeted for then that may well be the case. After having those assertions substantiated you may have to revisit the budgeting process, reassess the minimum acceptable functionality or adjust the ROI timescales. The fairly intractable problem being that accurate costs can really only be arrived at toward the end of the process. If you can share as much detail as possible with the vendors around functional requirements, volumes, etc., early in the process, then they may be able to give some indicative costs that need to be added to internal ones. However, the devil really is in the detail so whatever budget you request at the start should always include that caveat.
The Path to a Successful Data Integration Project Part 1
To know more about financial message and data integration visit www.volantetech.com
