Preparing To Switch

Big data has been a persistent buzzword in business publications. Run a web search, and big names like Accenture and Oracle will come in at the top. However, unless you are a Fortune 50 company, you do not have terabytes of data on you customers. This overly generic buzzword has little meaning or relevance to a small to mid-sized company. The average company has, at best, gigabytes of information regarding their customers, in their customer relationship management suite.

This does not mean there are no lessons to be learned from the big companies. Mining your databases for patterns that can be translated into business cases can be very useful. The resulting information can corroborate gut-feeling trends in your business or it can help define a new product or service your customers would be interested in. More importantly, customer trends can dispel long-held and entrenched beliefs about what is profitable to the business.

Why don’t more businesses engage in such internal research? The highest hurdle to overcome when businesses are interested in database mining is the additional payroll required for such a position. Data analysts are in high demand and these positions are currently commanding top dollar. The potential payoff for companies is enormous, and many large companies are justifying the initial investment required in order to create this position.

Hiring a data analyst is not enough to ensure project success. Prior to bringing a data analyst on board, the company must unify all of its data sets. If the company has a separate financial software package, a separate CRM tool, and separate operations management software, then all of the databases for these software suites are likely not unified. This is the selling point for the large enterprise resource planning (ERP) suites. However, implementing an ERP is often a very risky project with high failure rates and thus, it is not a project a company should approach lightly.

In cases where companies have disparate data sets that are not synchronized via a single ERP, data analysts often have to do the work of marrying the various datasets together. The resulting reports are based off of non-realtime data, but in many cases this does not impact the relevance of the reports. Performing the data transformations necessary to draw out the patterns the data analysts look for requires beefy hardware. This is where recent advances in solid state technology have been extremely helpful to simplifying the analyst’s job.

Instead of performing the heavy data lifting on the servers, analysts now prefer to export the data to a workstation that runs a sufficiently large capacity PCIe-based solid state card. Once the data set has been transferred to the workstation that runs such a storage setup, the data analyst can run queries against the data very quickly. The queries require a much shorter time to run, not only because the data is not running on a remote system with multiple users connected concurrently, but also because the solid state IO cards are significantly faster than even the latest solid state hard drives.

The benefits from data mining have enormous revenue potential as the recent developments from Sears Canada and Bank of America have shown. Smaller companies are working to begin analyzing their internal customer data to tease out trends. Is your company one of them? Let us know via twitter @AventisSystems.