Additional ways for reducing the BI TCO (Total Cost of Ownership)

Last Updated on Monday, 07 February 2011 00:21

• Reuse your SOA (Service Oriented Architecture) and ESB (Enterprise Service Bus) to simplify your ETL. The traditional BI requires the ETL developer to understand the database tables (and rules) for each source that is being used to populate the EDW. When you have to understand the underlying of a database to create the ETL process you are tightly coupling your DW to the internal of a database schema that you do not control! It is much better to receive the creation of a Purchase Order (for example) via the ESB than having to read the PO System or even buy connectors that can read the ERP database. This is something that is now technically available and used by some companies
• Understand the pricing models and the product offerings. When negotiating with a BI vendor, you may get additional tools as part of their enterprise edition version for a small increase in costs. If you are in the Best of Breed model and you need these additional tools, then you still have to put a certain weight on cost (for example 20%) and this may be sufficient to tip toward the enterprise edition. Also, be extremely careful on the yearly maintenance costs sometimes the “deal” is good for 2-3 years only. Also, look around you, if another project (even outside BI) is talking with your BI vendor for another tool (for example an ESB!) you may be able to reduce your maintenance costs as part of the ESB deal! As you can see the BI Architect as to be closely involved with the procurement to get the most of a transaction. Once we even built a sophisticated what-if analysis spreadsheet on a proposal to really understand what was our real price… procurement could not figure that one and the supplier was also confused since they asked for our spreadsheet!
• Open Source BI (OSBI). Open Source is not limited to Apache (web server), Linux (operating server), JBOSS rules (rules engine), MySQL (dbms), Joomla (content management system) and many others, which are all used to varying degree by the biggest companies in the world. OSBI have improved greatly and multiple companies have replaced proprietary BI tools by OSBI products. Now, we are not saying to ditch your BI investments on the contrary. Keep the proprietary tools that serve you well being an ETL tool, an OLAP tool, a Data mining tool or others. For the BI tools that do not deliver sufficient value, include in your replacement study OSBI tools and you may be surprised by the results. So, a combination of proprietary and open sources is quite acceptable for an organization and you must provide BI governance on which tools to use for which task and situation. It is also possible to use a proprietary ETL tool and an OS ETL tool under some conditions to minimize your TCO, an example will be to use the proprietary ETL tool for very complex processing or for very high volume ETL task and use the OS ETL for the rest (this position is based on mitigating the ETL risks and will become obsolete as more companies implement OS ETL to drive very large volume ETL processes and/or very complex ETL processes) . OSBI tools include but are not limited to: Pentaho and Jaspersoft!
• Use transactions files produced by applications. If your company did not invest in SOA and ESB then you may still be able to simplify your ETL. Some systems create files as part of their processing, in a Telecommunications Company, the rating engine was producing files containing mediated and rated voice calls constantly, so we took a look at the file content and asked the Billing team to add few columns on the file that was required for the DW. They did it in a matter of days and the ETL team never had to understand or go thru the hundred tables of the billing system. We even added the file (with time hierarchy) as a dimension so that we were able to simulate the state of the rating-billing system at any point in time and it was also used as part of a control and audit process