As part of an ongoing project for one of our clients, we were recently asked to incorporate a Business Intelligence (BI) connector to facilitate regular data analysis.
Let me give you some context: the client’s current database is built in SQL, while the new platform that the team is currently working on is built with the MEAN stack, hence MongoDB for data.
Without getting into a debate about the pros and cons of NoSQL vs SQL, for producing data snapshots in tabular format that can be easily analysed in an Excel spreadsheet, SQL is definitely more appropriate than NoSQL. Fortunately, Mongo have developed their own BI connector that allows for visualising data “using existing relational business intelligence tools” such as Tableau or PGAdminIII.
As a long-time Windows user, the first issue I ran into was that Mongo BI is only supported for Linux platforms (boo). This lead me to my first encounter with VirtualBox (unfortunately, getting a decent 64-bit virtual Ubuntu machine running on my laptop threw in a number of hurdles, but I will not go into those here).
Once I had this up and running (after numerous attempts), I had to identify the base requirements to get a working BI connector, and since I wouldn’t be copying the virtual machine over to the client’s server I also had to document the most straightforward process of getting one ready on any machine with minimal prerequisites. This is when I discovered Ansible could solve all my problems (and then some!).
There has been a push recently at Outlandish to incorporate a DevOps approach into more of our projects, and one of the things that was discussed at the last DevOps meeting was the advantages of automation scripts. Ansible provides this automation with easy human-readable syntax and a very shallow learning curve.
Although there were a few bumps along the way, I found that approaching the project setup via an automated script helped me to carve out the simplest route to getting a Mongo DB up and running on any Debian-based machine, and forced me to take the time to consider how jobs that I initially assumed would have to be manually performed could be incorporated into the script, thus relieving the client of as much work as possible. (As an added bonus all of my documentation was being created along the way with no extra effort, thanks to that nice human-readable syntax.)
But of course, as well as the setup script I also created a regularly reusable script that would retrieve the database dump from a repository, clear down the data on the existing DB, create a new schema file, restore the Mongo DB using this schema, and then create the required custom views from a SQL script so that all the client would need to do to get the necessary tables into Excel would be to open their chosen BI tool and export the views from there – simple.
This project has made it obvious to me the benefits of automation. I have seen the light.
I hope that we can use this approach more at Outlandish in the future, as in my experience it is often the setting up of a project that can take up the most amount of time – if we can reduce that time as much as possible then us developers can spend more of our time doing what we would actually like to be doing: developing!
For information on getting started with Ansible, click here for installation instructions and click here for getting started on writing playbooks.