Digital: Disrupted: The Power of Data Lineage

Paul Muller: Alright, first thought that comes to mind when you think about data and lineage.

Chris Wey: I would say turn by turn directions. I think when we use that term at Rocket Software, we think how does data get from point A to its destination, which is usually a report or a business intelligence system, and there's a long path to get there and it can take many paths through your organization, through databases and applications, and the lineage is the turn-by-turn directions on how it gets from its origination, its creation point to its destination.

PM: I have to say that it sounds so hard. It makes me want to give up. So, we're going to get to how you've managed to fix or at least address part of that solution to the problem in a minute. But let's talk about the consequences of not getting it right because obviously one of them is that you don't trust the data. I've been in those meetings at a board level where we argue about even what the revenue for the company was. I mean, it sounds absurd, but you're like people can't even agree on how much money you made that quarter. And if you can, it takes a really long time to get to an agreement. So, it's like months after you've actually closed rather than minutes. So, there's obviously some of the operational downside, but as I alluded to, I understand there's also a lot of legal and regulatory implications to not getting your data. Is that correct?

CW: That's right. I mean, many of our customers are in regulated industries, highly regulated industries where there are banking regulations, insurance regulations, and those things are important. They're important for consumer protection and they're important for the stability of the banking system, for example, which is a hot topic these days. And so, there are certain regulations that need to be complied with. The reserves within a bank, for example, and other regulations, and those are very, very complicated and they get updated, and they are customized by jurisdiction. And so, if you're in Europe or you're in the US and then the states have their own regulations and that gets very complex, and companies need to comply. And if they don't comply, there are fines and certain reports need to be generated within a certain amount of time and they need to be delivered in a timely manner to the regulators.

And if that's not done, again, there are fines and other adverse repercussions. And so I think that that's the baseline goal of many of our customers is we need to deliver these compliance reports and we need to do it on a regular basis and they have to be efficient and we need the right data and we need to be sure that the right data is in the report because many times the chief executive officer and the chief financial officer have to sign on the report saying, I approve this message. And so that's important for our customers.

PM: So, let's talk then a bit about how automation can solve the problem because obviously we're dealing with too much data, too many sources, too many conflicts, and certainly too much speed and volume to be able to do this just using manual labor. Talk to me a bit about the solution that you've been working on some of these problems.

CW: Well, I think that that's where software can really help drive a lot of scalability and automation in solving some of these problems. And one of the things that we've had in the market for decades is our data intelligence solution and we call them scanners, but they scan databases or the transformation software or the data warehousing where a lot of data is stored or the business intelligence system that have a lot of reports. And so, we're scanning all of these databases, both the source systems and the destination systems and everything in between. And that's been a specialty of what we've done for more than 30 years. The problem that has happened is that that aggregation and the connection and the ability to connect those dots has been presented in our customers environments to the data engineers on the IT side.

And you have to be the really technical users to understand how all of those connections work. And it's not in the business user language. And the opportunity that we saw based on the conversations we've been having is how do we open that up to the business user community? Because if I'm in a business analyst in marketing or if I'm in a sales operations role or if I'm in an operations team leading customer service agents and I want to get data as a consumer of the data that's in my very large enterprise and I want to know where it come from and why doesn't it match my colleagues' data in a different department and what's the difference? I can't do that in today's environment. It's too complicated because then you have to go talk to it and they don't speak business and that's complicated. And so that's what we're trying to solve with our data intelligence solution. And we've released a new version of the software that makes that available for business users. And we launched that a few weeks ago.