The difference between AI projects that deliver ROI and those that stall isn’t the technology. It’s asking the right questions before you start building.

Whether you’ve heard the buzz about artificial intelligence in healthcare, or you’re already piloting something, it’s clear: not all AI efforts pay off. Some advance to enterprise-scale, others stall. The difference? Surprisingly, it’s all in the preparation.

Consider what’s possible when that preparation is done right: Nebraska Medicine increased its discharge lounge usage by 2,500% with AI. Ballad Health reduced clinician burnout and improved documentation speed. Lee Health enabled doctors to see more patients per day while improving satisfaction.

Meanwhile, other healthcare organizations invested in similar pilots, only to watch them stall, fail to scale, or deliver no measurable ROI.

After years of building digital solutions for  healthcare organizations, from AI chatbots,to patient engagement platforms, and operational automation tools, we’ve seen what makes these projects succeed and what makes them fail. And increasingly, the difference comes down to three critical questions before writing a single line of code.

If you’re considering AI to improve patient experiences, reduce operational costs, or support clinical workflows, start here.

 

Question 1: What problem are we actually solving, and for whom?

The core issue in healthcare AI is not a lack of problems. It is that teams often pick the wrong problem, or the wrong group to solve it for.

You feel pressure to “do something with AI” from the board, vendors, or internal champions. IT talks about data pipelines. Operations talks about throughput. Clinicians ask for time back. Patients want better access and clarity. If you do not align these goals early, you end up with competing definitions of success and projects that appear impressive on a deck but have minimal impact on daily work.

The main risk is not a bad model. It is a good model pointed at the wrong target.

A chatbot that handles FAQs does not guarantee higher patient satisfaction. A documentation tool that speeds up note taking does not fix burnout if the workflow itself is broken.

AI works when it is tied to a clear, specific problem that a defined group of people cares about, and when you can measure what “better” means for them.

Lee Health got this right. They started with a very specific pain point physicians were losing hours every day on documentation, which meant seeing fewer patients and burning out faster. They had a specific user group: providers. Their ambient dictation tool directly addressed that pain point. The result? Measurable revenue impact as doctors could see more patients per day, plus improved provider satisfaction. The AI wasn’t implemented because it was cool or innovative. It was implemented because it solved a documented, expensive problem.

This kind of problem-first thinking matters even more as healthcare faces broader workforce challenges. The nursing shortage, physician burnout, and staffing constraints aren’t going away. AI can help address some of these operational pain points, but only when it’s deployed to solve specific workflow problems, not just because the technology exists.

Before we start building digital solutions for healthcare clients, we run discovery workshops to identify the specific workflows causing friction, the users experiencing that friction, and what success actually looks like for them. A chatbot that helps patients schedule appointments solves a different problem than one that answers clinical questions. Those require different design, different integration, different success metrics. Getting this clarity upfront prevents expensive pivots six months into development.

Here’s the bottom line: If you cannot name the user, describe the problem in one sentence, and say how you will measure progress, you are not ready to build AI. Without that shared, tested problem statement, AI becomes expensive technology searching for a purpose.

 

Question 2: Do we have the data foundation to support this?

This one trips up more projects than you’d think. AI is only as good as the data it’s trained on and integrated with. We’ve seen too many healthcare organizations discover halfway through a project that their data is siloed across multiple systems, inconsistent, inaccessible, or noncompliant.

Think about what your AI actually needs to function. Chatbots need access to real-time appointment data, patient records, or clinical information. Predictive tools need clean, historical data for training. Patient engagement platforms need to be integrated with EHRs, scheduling systems, and billing systems. And HIPAA compliance adds complexity to every single one of those data flows. You can’t retrofit this stuff later.

The warning signs are clear. If anyone on your team says, “We’ll figure out the data integration later,” that’s a red flag. If your data lives across multiple systems with no unified way to access it, that’s a problem. If you don’t have a governance framework for who can access what data, or if your current processes involve manual data exports, you’re not ready.

Nebraska Medicine’s 2,500% increase wasn’t just about the algorithm. Byron Jobe, CEO of Vizient, pointed out that their success came from having the data infrastructure and integration strategy ready to deploy the solution effectively. They treated AI as part of a larger ecosystem, not a standalone tool. The technical foundation had to support the ambition, or nothing else mattered.

When we work with clients, we assess data availability, quality, and accessibility during discovery. We help them understand what data they need for their AI to actually work and what integration points are required. Sometimes we discover the foundation isn’t ready, and honestly? We tell them to build that first. It’s better to invest a month or two in data infrastructure than to build an AI tool that can’t access the information it needs to function.

When we built an enterprise platform for a health data science company, integration with their existing systems and HIPAA compliance weren’t things we bolted on at the end. They were core architectural decisions from day one. Data encryption, audit trails, access controls—all of that has to be part of the plan from the beginning. The organizations that succeed with healthcare AI involve their compliance and security teams early, not after the code is written.

The reality is simple: the best AI in the world fails without clean, accessible, compliant data.

 

Question 3: How will we measure success, and do we have the baseline?

Most AI projects define success as “it works” or “users like it.” That’s not enough. Healthcare CFOs need ROI proof. Clinicians need workflow improvement proof. Administrators need operational efficiency proof. Without clear metrics and baseline measurements, you can’t prove value. And if you can’t prove value, you won’t get funding to scale.

Here’s what happens all the time: a pilot shows promise, everyone gets excited, and then when it’s time to roll it out across the organization, leadership asks, “Okay, but what’s the actual business impact?” And nobody has a good answer because nobody measured the baseline. The project dies right there.

Different stakeholders care about different metrics. Cost savings versus patient satisfaction versus clinical outcomes. If you don’t have baseline data on your current state (call center volume, documentation time, patient wait times), you can’t demonstrate improvement. And the “soft” benefits like reduced burnout and improved satisfaction? Those are actually harder to quantify but often more valuable in the long term than direct cost savings.

Watch out for these traps: defining success only as “we launched the tool,” having different departments with conflicting success criteria and no way to prioritize them, or measuring AI performance without connecting it to business impact.

Ballad Health did this well. When they implemented DAX Copilot, CIO Pam Austin didn’t just check whether the generative scribe tool worked. She measured reductions in documentation time, improvements in clinician satisfaction, and quality indicators in patient care. They had clear before-and-after metrics that told a story about real impact.

And here’s something traditional spreadsheets miss: in a market where replacing a physician represents a significant cost to the organization, keeping doctors from burning out is measurable ROI. It just doesn’t show up as a line item in cost savings. Provider retention, reduced recruitment costs, sustained patient relationships—all of that has financial value that justifies AI investment.

We help clients define success metrics during discovery and build measurement into the solution from the start. Analytics dashboards, integration with existing reporting tools, automated data collection for before-and-after comparison—whatever makes sense for that specific project.

But we also push clients to think beyond technical metrics. A chatbot that successfully answers 90% of queries sounds great, but if it doesn’t reduce call center volume, improve patient satisfaction, or free up staff to focus on higher-value work, what was the point? We connect AI performance to business outcomes that matter to the organization, the patients, and the bottom line.

If you can’t measure it, you can’t prove it worked. And if you can’t prove it worked, you won’t get funding to scale it.

 

Starting with the right questions

Healthcare organizations that answer these three questions before starting development see dramatically better outcomes. Their AI gets adopted by end users. It scales past the pilot phase. It delivers measurable ROI that justifies continued investment.

Organizations that skip these questions often build impressive technology that nobody uses, that can’t integrate with existing systems, or that leadership can’t justify continuing to fund.

The good news? These questions aren’t hard to answer. They just require intentional thought and honest assessment before you commit to building.

We’ve built our process around these questions. We start with discovery and strategy, not code. We assess data readiness, not just what’s technically possible. We define success metrics upfront, not after launch.

Because the goal isn’t just to build AI, it’s to build AI that actually works in the complex reality of healthcare delivery.

Ready to move from hype to impact? We’ve helped healthcare and wellness companies build digital platforms that deliver measurable results. Explore our healthcare work or contact our team to discuss your AI initiative.