How to know if your company's digital initiatives are on the path to success
- Companies are investing heavily in digital overhauls, and it's critical that tech chiefs are able to properly calculate the return on investment to ensure funding continues.
- The outcomes, however, can go beyond just financial returns. That's why Morgan Stanley's Jeff McMillan also examines whether digital initiatives result in behavioral changes among clients.
- As chief data and analytics officer for the bank's $17 billion wealth management division, McMillan has to determine ROI on the 50-60 initiatives his team has ongoing at any time, as well as the projects that other departments are running.
- Click here for more BI Prime stories.
Companies like Walmart, JPMorgan Chase, and PepsiCo are investing heavily in overhauling their digital operations. But a key challenge for organizations is increasingly how to fund those projects and, more importantly, the return on investment.
It's becoming more common, for example, for top management to give tech chiefs unallocated pools of money to pursue projects independently without having to return to the enterprise each time for funding. But that creates the mandate for chief data officers and other tech leaders to figure out the impact of those initiatives to ensure the investments remain ongoing.
It's one of the main jobs for Morgan Stanley's Jeff McMillan. As chief data and analytics officer for the bank's $17 billion wealth management division, his team is tasked with calculating the returns on the 50-60 initiatives it has ongoing at any time - along with the impact of the slew of data projects that other departments are running on their own.
"Not everything that we've done over the last five years works great," he told Business Insider. "We tested something. It didn't work. It didn't prove to be effective. Now, let's do that better next time. And that's the process that we're focused on and the outcomes will take care of themselves over time."
For McMillan, the key is thinking about analyzing results before an initiative goes live. That, along with the realization that investment return on digital projects can go beyond just financial gain, is another example of how the role of tech chiefs is changing.
"You have to start the process with the measurement, you can't end the process with it," he said. If you come to the tech chief after "and say 'how did it go?' The cat is already out of the bag. I can't really help you because you've probably tainted your approach."
At Morgan Stanley, McMillan views his job as a decision support function to help empower the 25,000 employees to use data to improve how the legal and compliance department pinpoints potential fraud, or how financial advisors make recommendations to clients.
That's why it's so critical for McMillan and his team to adequately determine the impact of projects, so they can advise others on the most effective approaches. McMillan explained how the wealth management division laid the groundwork to properly quantify return on investment - and the steps he takes to figure it out.
Understanding the touch points
One area where McMillan spent ample energy was ensuring that his team and others has access to the data for all projects.
Say financial advisors are trying to market clients on a new service. It's not enough to compare the final enrollment figures against another campaign, McMillan said.
To properly analyze return, McMillan and his team need to look at who the service was advertised to, whether the customers actually received the marketing materials, and which clients called their advisors about it. "We spent an enormous amount of time architecting our system so all those touch points are understood. And that took years," he said.
It's also key for McMillan that project leads come to his department early in the process. If Morgan Stanley is hosting an event with clients in San Diego, for example, he urges advisors to think about whether the goal is to raise awareness, sell a product, or another outcome.
Primary vs. secondary results
McMillan thinks about outcomes in two buckets: primary and secondary results.
Primary results are often easier to calculate, like did the client apply for the advertised service. But secondary impact - or whether there was a positive behavioral change - is more difficult to determine.
For every client project, McMillan creates three "look-a-like clients" that act as a control group. Those test cases make it easier for McMillan and the team to figure out whether there was any adjustment in how the client interacts with their Morgan Stanley advisor or other behavioral changes.
"Sometimes it's absurdly clear, sometimes it's clear that it works. Sometimes it's clear that it didn't. And sometimes it's ambiguous in the middle," he said.
That's why it's important for McMillan to get involved early, so those test groups can be established and his team can work with the other departments to figure out the best metrics to weigh success or failure.
To help with those relationships, McMillan also created a "business analytics manager" position that effectively acts as a conduit between his team and other units. The business analytics manager engages with other teams early-on in the process to inform McMillan's team of upcoming projects.
"We're no longer in a position where someone does something and then, three months later, we're like, 'how did that happen?'" he said.
While companies will ultimately find their own ways to determine the return on digital investments, it's a necessary step that tech chiefs must take in order to justify continued funding - and McMillan's approach can serve as one route to achieving that.