Building Recommendation Systems That Actually Convert (2026)

We built our first recommendation system in 2019. It suggested products that were technically similar to what users viewed. Users ignored the recommendations. Management wanted to kill the project.

Then we changed approach. Instead of "similar products," we asked: "What will this user actually buy?" The conversion rate tripled overnight.

That's when I learned the difference between a good recommendation system and a profitable one. At Viprasol, we've built systems for e-commerce, streaming platforms, marketplaces, and financial services. What I'm sharing are the patterns that actually work.

Why Most Recommendation Systems Fail

The technical problem is solved. We have enough algorithms. The business problem is harder.

Most systems fail for one of these reasons:

Wrong optimization target: Recommending the most popular items (they'd find those themselves) or the most similar items (technically correct, commercially irrelevant).

Cold start problem: New users with no history. New items with no interactions. You can't recommend based on data you don't have.

Lack of diversity: Recommending the same category repeatedly. Users get bored.

Ignoring business constraints: Recommending products that don't make money. Items that are out of stock. Competitors you're losing margin to.

No real-time updates: Recommendations lag behind user preferences. A user browsed cameras all week but the system still recommends laptops.

Poor ranking at the end: Even with good predictions, bad ranking destroys conversion. Showing items in the wrong order kills performance.

The teams that win don't use more sophisticated algorithms. They optimize for what matters: actual user behavior and business metrics.

Understanding Your Users and Items

Before building anything, we map out three things:

User dimensions: How we categorize users

• Demographic: Age, location, customer segment
• Behavioral: Purchase history, browsing patterns, engagement level
• Temporal: New vs. established customers, seasonality
• Value: Lifetime value, profit margin, churn risk

Item dimensions: What we're recommending

• Category and subcategory
• Price and margin
• Inventory level (recommend in-stock items higher)
• Seasonality
• Popularity

Interaction types: What signals we use

• Explicit: Ratings, reviews, purchases (strong signal, sparse data)
• Implicit: Views, clicks, adds to cart, time spent (weak signal, abundant data)
• Contextual: Device, time of day, location

For many businesses, we weight implicit feedback more heavily than ratings. A user browsing a product page for 2 minutes signals more intent than a 3-star review.

Cold Start Problem: Your First Challenge

You have two users. You have 10,000 items. How do you recommend anything?

Content-based approach: If you know item features (category, price, brand), recommend items similar to what the user viewed or purchased. This works but tends toward bland recommendations.

Knowledge-based approach: Ask questions ("What's your budget? What category?") and recommend based on answers. Works for complex purchases (cars, mortgages, insurance).

Hybrid approach: Combine user input with item features. Ask a few questions, use those answers to bootstrap item-based recommendations.

For new users specifically, we often use:

• Popular items in their inferred category
• Items purchased by similar users (if we can infer similarity)
• Trending items in their region
• Trending items overall

When new users first arrive, you know almost nothing. Best practice: gather some preference information explicitly or implicitly before using complex algorithms.

Collaborative Filtering: The Workhorse

This is the most common approach. The core insight: similar users like similar items.

User-user collaborative filtering: Find users similar to your target user, recommend what those similar users liked. Problem: doesn't scale well and new items have no signal.

Item-item collaborative filtering: Find items similar to what the user liked, recommend those. Works better because items change slower than user preferences.

Matrix factorization: Decompose the user-item interaction matrix into latent factors. Users and items are represented as vectors in latent space. Recommendations are based on dot products.

We use matrix factorization (and its neural network variants) for most problems. It handles new items better and scales more efficiently.

The classical approach is SVD. Modern approaches add neural networks:

In practice, we often combine multiple approaches. Fast matrix factorization as primary, with content-based filtering as fallback for cold items.

Content-Based Recommendations

If you know what items are, you can recommend similar ones without user history.

Features we use:

• Textual: Product name, description, category, tags
• Visual: Images (using computer vision embeddings)
• Categorical: Brand, manufacturer, style
• Numerical: Price, rating, popularity
• Temporal: Newness, trend velocity

We embed these into vectors using:

• Traditional: TF-IDF for text, one-hot for categories
• Modern: BERT embeddings for text, vision models for images
• Hybrid: Learning embeddings jointly on multiple feature types

Content-based is our safety net. When user data is scarce or unreliable, content-based recommendations provide something reasonable.

Ranking Beyond Prediction

Here's where theory and practice diverge. A good prediction model doesn't make a good recommendation system if you rank poorly.

We optimize for:

Relevance: Predicted rating or purchase probability (what most models optimize)

Diversity: Users don't want the same item recommended 5 times. We penalize showing multiple items from the same category.

Coverage: Recommend items that need it (deep catalog item that wouldn't be discovered otherwise)

Freshness: Bias toward new items slightly to maintain discovery

Business metrics: Margin, inventory, strategic priorities

Ranking strategies:

1. Pure relevance ranking: Rank by model score. Simple, but risks monotonous recommendations.

2. Diversity-aware ranking: Start with relevant items, then select diverse items from remainder. Balances relevance and diversity.

3. Contextual bandit approach: Treat recommendations as exploration-exploitation. Mostly recommend high-scoring items, occasionally recommend exploratory items.

4. Learning-to-rank: Train a second model that takes predicted scores plus diversity metrics and outputs final ranking.

In production, we usually use approach 3 or 4. Contextual bandits work well because they naturally balance exploration and exploitation.

Real-Time vs. Batch Recommendations

Some recommendations must be real-time (product pages, search results, checkout). Others can be batch (weekly email, homepage takeover).

Batch recommendations (email, push notifications):

• Computed offline, can be complex
• We run daily or weekly
• Can afford expensive models
• No latency constraints
• Often higher variance in recommendations

Real-time recommendations (product page, search):

• Must return in <100ms
• Can't use expensive models
• Must handle session data in memory
• Need fallback for edge cases

Our approach: Pre-compute heavy recommendations in batch (send via email, display on homepage). Light models serve real-time.

For real-time, we pre-compute embeddings and store them in fast stores (Redis, Memcached). Computing recommendations is then a simple similarity lookup.

Feature Engineering for Recommendations

Just like ML pipelines, garbage features produce garbage recommendations.

Key features we compute:

User features:

• Total items purchased
• Average spend
• Purchase frequency
• Favorite categories
• Last purchase date (recency)
• Price sensitivity (median price purchased)
• Return rate (if available)

Item features:

• Purchase count
• Average rating
• Review sentiment
• Visual properties (color, style)
• Price and margin
• Stock availability
• Seasonality factor

Interaction features:

• User-item similarities
• Co-purchase patterns
• View-to-purchase rate
• Time between view and purchase
• User-item price similarity

Temporal features:

• Day of week
• Hour of day
• Day since last interaction
• Seasonal factor

The feature engineering phase often takes longest. Getting it right determines whether the system works or doesn't.

Handling Popularity Bias

If you train purely on user interactions, the system learns to recommend popular items. This is easy but wrong.

Problems:

• Popular items are already discoverable
• Niche items never get recommended
• New items have no chance
• You miss inventory optimization opportunities

Solutions:

Position bias correction: Popular items appear in top positions by default. When computing probabilities, down-weight popular items in top positions (they'd appear there anyway).

IPS weighting: Inverse propensity scoring adjusts for the probability an item appeared in training data.

Causal inference: Model the true preference instead of apparent preference, accounting for what was shown.

Strategic recommendations: Recommend items with good margin, adequate inventory, or strategic importance, not just highest predicted score.

Most production systems use a combination: adjust training to reduce bias, then apply business constraints to ranking.

A/B Testing Recommendations

How do you know if your new model is better?

You can't trust offline metrics. A model with higher predicted rating might perform worse in production because users don't actually like the recommendations.

We always A/B test:

• Hold out 10% of traffic for control (old model)
• 45% sees new model, 45% sees old model
• Track: click-through rate, conversion rate, average order value, repeat purchase rate

Timeline: Run for 1-2 weeks to cover different traffic patterns.

Key metrics we track:

We implement infrastructure to run dozens of A/B tests in parallel. Each test compares to the baseline, and winners gradually become the new baseline.

Handling Fraud and Manipulation

Once you have a recommendation system, people will try to game it.

Sellers might:

• Buy their own items to boost interaction
• Write fake reviews
• Collude to boost recommendations
• Use bots to generate interactions

We protect through:

• Interaction filtering: Ignore interactions from suspicious accounts
• Review moderation: Detect and remove fake reviews
• Bot detection: Flag suspicious user behavior
• Collaboration detection: Find sellers coordinating

For sensitive applications (financial services, marketplace with high fraud incentive), we employ dedicated fraud teams.

Technical Implementation at Scale

When you serve recommendations at scale, infrastructure matters.

Batch layer (offline):

• Compute recommendations for all users (or high-value users) daily or weekly
• Store in database
• Can afford expensive models

Serving layer (online):

• Fetch pre-computed recommendations from database
• If user is new, compute on-the-fly using fast model
• Support A/B testing infrastructure

Learning layer (feedback):

• Log all recommendations and outcomes
• Periodically retrain models
• Update model in batch layer

We typically use:

• Spark for offline computation
• Python for model training (scikit-learn, PyTorch)
• Java or C++ for low-latency serving
• Kafka for event streaming

For details on scaling this kind of system, see our Cloud Solutions page.

Personalization Beyond Recommendations

Once you understand user preferences, applications expand:

Search ranking: Rank search results by predicted user preference

Email subject lines: Personalize based on user preferences and behavior

Push notifications: Time and content based on user activity patterns

Pricing: (carefully) adjust pricing for users with different willingness-to-pay

Content curation: Adjust feed/home page to show preferred categories

The same features and models power these applications. We build a platform for preferences then apply it across the product.

Common Pitfalls and How We Avoid Them

Over-optimizing for metrics without business sense: We've seen systems that maximize click-through rate by recommending cheap, popular items. Nobody buys them. Metrics are tools. Business outcome is the goal.

Ignoring context: A user browsing for gifts has different needs than one shopping for themselves. Time of day matters. Device matters. Ignoring context produces generic recommendations.

Fighting with inventory and logistics: Recommend items that are expensive to ship. Customers stop using recommendations. We integrate with inventory and logistics systems from day one.

Not addressing cold start adequately: Systems fail on new users because cold start was an afterthought. Design for cold start as a core requirement.

Assuming recommendations work uniformly: Different user segments respond differently. New users, power users, churned users, price-sensitive users—they all need different strategies.

Getting Started

For a typical e-commerce business:

1. Weeks 1-2: Collect interaction data, define metrics
2. Weeks 3-4: Implement simple collaborative filtering
3. Weeks 5-6: Add content-based fallback
4. Weeks 7-8: Implement ranking and diversity
5. Weeks 9-10: A/B test, iterate on features
6. Weeks 11-12: Integrate with production systems

A simple system working on real data beats a complex system in theory.

For complex recommendation problems (multi-sided markets, complex constraints, heavy personalization), our team at Viprasol helps design and implement systems. See AI Agent Systems and SaaS Development for system integration details.

FAQ: Recommendation Systems

Q: Collaborative filtering or content-based recommendations?

A: Both. Use collaborative filtering as primary because it learns from collective user behavior. Use content-based as fallback for new items and cold start. Combine them for best results. The weighted blend usually performs better than either alone.

Q: How much data do we need?

A: Surprisingly little to start. We've built working systems with 10,000 interactions and 1,000 items. Quality matters more than quantity. A month of real interactions beats a year of bot data.

Q: Should we use off-the-shelf recommendation platforms?

A: For small-to-medium businesses, yes. Platforms like Algolia, Coveo, or Kenshoo handle most cases well. For very large scale or unusual requirements (two-sided market dynamics, complex constraints), custom systems become necessary. We usually start with platforms and graduate to custom if needed.

Q: How often should we retrain?

A: For most systems, weekly retraining works. Some systems benefit from daily retraining or even continuous learning. Always monitor performance; if it degrades, retrain more frequently.

Q: How do we handle returning users who changed preferences?

A: Weight recent interactions more heavily. A purchase from last week matters more than one from last year. You can explicitly model preference decay: older interactions contribute less to the recommendation.

Q: What's a realistic lift from recommendations?

A: 10-30% increase in click-through rate is common. 5-15% increase in conversion is realistic. Revenue lift depends on order value. A well-built system can improve revenue 20-50%. These vary widely by industry and baseline.

Q: How do we prevent filter bubbles?

A: Actively inject diversity. Occasionally recommend items from categories the user hasn't explored. Monitor that recommendations span reasonable category diversity. Have editorial override to manually inject serendipitous recommendations. The business value of discovery sometimes exceeds pure personalization.

Wrapping Up

Recommendation systems are powerful but nuanced. The teams building the best systems:

• Optimize for business outcomes, not metrics
• Combine multiple algorithmic approaches
• Test changes on real users
• Handle cold start as a first-class problem
• Regularly audit performance across user segments
• Integrate with business constraints

The biggest wins don't come from better algorithms. They come from better understanding user needs, better feature engineering, and better integration with business systems.

Start simple. Measure what matters. Iterate quickly on real data.

The most profitable recommendation systems aren't the most sophisticated. They're the ones that understand their users and their business deeply.