GenAI is (not always) all you need — A decision making guide

As the founding engineer at my company, I often find myself in a maze of decisions — both business and technical. The decisions we make today will shape the foundation of our technology stack, our product, and how we tackle problems in the years to come.

The technical decision space has grown dramatically in recent years, especially with the emergence of Generative AI (GenAI). This new technology offers incredible promise but also comes with a unique set of challenges. GenAI has brought a paradigm shift, expanding what’s possible and enabling us to approach problems in novel ways. However, this shiny new tool can sometimes feel like a hammer searching for nails. For example, asking GenAI to add two numbers is not only overkill but also grossly inefficient — both in cost and complexity.

We are in the early days of fully understanding where GenAI fits into our workflows. It’s exciting, but also daunting. The risk of misusing it — or applying it where it doesn’t belong — is real. To navigate these challenges, I built this decision framework for myself, and I’m sharing it here in the hope that it might help others grappling with similar decisions.

Too Many Options, Too Little Clarity

When faced with a technical problem, choosing between traditional programming (Python Code), Classical Machine Learning, and GenAI can feel overwhelming. Each option comes with trade-offs: explainability, cost, scalability, and team expertise. Without a structured approach, it’s easy to fall into analysis paralysis or make suboptimal choices.

A Map Through the Decision Maze

This framework simplifies decision-making by focusing on key factors: determinism, explainability, flexibility, cost, and scalability. It also includes practical tools like flowcharts and decision matrices to guide you through the process.

Decision framework — simplified

Key Decision Factors

1. Business Requirements

2. Data Readiness

3. Team and Tech Considerations

Expanded Recommendations

Option 1: Python Code

Key Use Cases:

• Data transformation pipelines.
• Web scraping.
• Rule-based workflows.

When to Choose:

• High Determinism Needed: Output must be predictable and repeatable.
• Compliance/Explainability Critical: Regulatory requirements demand clear logic.
• Team Expertise: Team is skilled in Python development.

Option 2: Classical Machine Learning

Key Use Cases:

• Spam detection.
• Sentiment analysis.
• Regression analysis for pricing.

When to Choose:

• Data-Driven Problems: Structured problems with measurable outputs.
• Moderate Explainability Needs: Success metrics can balance explainability with performance.
• Data Available: Clean, structured data is ready for training.

Option 3: Generative AI (GenAI)

Key Use Cases:

• Summarizing large texts.
• Generating creative content.
• Conversational agents.

When to Choose:

• Open-Ended Needs: No clear deterministic solution.
• Quick Prototyping: Need results quickly to test feasibility.
• Budget Available: High costs are acceptable for experimentation.

Trade-Off Table

Quantitative Decision Matrix

Example Weighted Scores:

• Python Code: 59
• Classical ML: 56
• GenAI: 54

Ethical and Practical Considerations

1. Hidden Costs

• GenAI: Ongoing API and observability costs.
• Classical ML: Retraining and data drift management.
• Python Code: Debugging and refactoring for scale.

2. Risks

• GenAI: Hallucinations and bias.
• Classical ML: Poor generalization with low-quality data.
• Python Code: Inflexibility for evolving requirements.

Phase-Specific Application

1. Exploration Phase:

• Use GenAI for rapid prototyping.
• Test problem feasibility and gather initial feedback.

2. Definition Phase:

• Transition to Classical ML for structured, measurable solutions.
• Refine data and success metrics.

3. Optimization Phase:

• Implement Python Code for deterministic, scalable, and cost-efficient deployment.