Booster Analytics

In this article, I discuss how lean production principles can be applied to building data products. As a CTO, I often see organizations pouring resources into data product development with little to show for it beyond bloated infrastructure and dashboards nobody uses.

The problem? They fail to apply lean principles to their data initiatives. Lean productivity, rooted in the philosophy of maximizing value while minimizing waste, is crucial for building effective and efficient data products. Here’s how I would do it.

1. Start with a Clear Business Value Hypothesis

Too many data products exist because someone thought they might be useful. That’s the wrong approach. Lean productivity starts with defining a clear business value hypothesis:

• What specific problem does this data product solve?
• Who are the end users, and how will they benefit?
• What decisions or actions will this data product drive?

If you can’t answer these questions, your data product isn’t a product – it’s an expensive science experiment. Take a walk, and rethink.

2. Eliminate Waste in Data Pipelines

Traditional data product development often results in bloated ETL processes, unnecessary data duplication, and overcomplicated architectures. Lean principles emphasize reducing waste, including:

• Overprocessing: Collecting and transforming data that no one uses.
• Waiting time: Slow data pipelines delaying insights.
• Defects: Poor data quality requiring excessive cleaning.

A lean approach promotes building minimal viable data pipelines – only ingest the data you need and refine incrementally.

3. Build Minimum Viable Data Products (MVDPs)

Instead of spending months (or years) building a perfect data platform, focus on Minimum Viable Data Products (MVDPs):

• Launch small, focused data solutions that address a single, high-impact problem.
• Get feedback early from real users.
• Iterate rapidly based on actual business needs.

This approach avoids the all-too-common scenario where a company spends millions building a “data platform” that nobody uses. Unless, you are a vendor.

4. Empower Cross-Functional Collaboration

Data product success depends on breaking silos. Lean productivity thrives in an environment where data engineers, analysts, domain experts, and end users collaborate. This means:

• Embedding data engineers and scientists into business teams.
• Using agile methodologies to ensure continuous feedback.
• Prioritizing outcome-driven metrics over technical perfection.

If your data team is working in isolation, you’re setting yourself up for failure.

5. Measure Value, Not Just Activity

Traditional data teams measure success in terms of how much data they process or how many dashboards they generate. That’s meaningless. Lean productivity requires tracking real business impact, such as:

• Reduction in decision-making time.
• Increase in revenue or efficiency due to data-driven actions.
• Adoption rates of data-driven insights by business users.

If your data product isn’t driving measurable value, you should be questioning why it exists.

6. Automate What Matters

Automation is often misapplied in data product development – teams automate tasks before proving their necessity. Lean productivity calls for just-in-time automation, meaning:

• Start with manual, simple solutions.
• Automate when it becomes a bottleneck.
• Use low-code/no-code tools for rapid deployment when possible.

Overengineering is the enemy of lean data teams.

7. Continuously Improve with Feedback Loops

Lean principles emphasize Kaizen (continuous improvement). Data products should evolve based on real-world usage, not just internal roadmaps. Implement:

• Usage analytics to see which features are actually being used.
• Regular user feedback sessions to refine based on real needs.
• A/B testing to validate improvements before rolling them out widely.

Applying lean production to data product development means cutting waste, focusing on user value, and iterating fast. The key is to think like a product manager, not just a data engineer. If a data product isn’t delivering measurable business impact, it doesn’t deserve to exist. The best data teams aren’t the ones with the biggest infrastructure budgets – they’re the ones that deliver results with minimal waste. Lean thinking is how you get there.