Build a Recommendation Engine or Buy One

You've got user data, a product catalogue, and a backlog full of personalisation tickets. The question isn't whether a recommendation engine would help — it's whether building one from scratch is worth the engineering investment, or whether an off-the-shelf solution gets you to the same place faster and cheaper.

This post lays out how to think through that decision honestly, including the cases where buying is clearly the right call.

What Does "Good Enough" Actually Look Like for Your Use Case?

Before you evaluate any vendor or write any code, pin down what the system needs to do.

Recommendation engines are not a single thing. A collaborative filtering model that powers "users who bought X also bought Y" is architecturally different from a content-based system that ranks articles by similarity to a user's reading history. A session-based model using transformers (think BERT4Rec or SASRec) that predicts the next item in a cold-start session is different again.

The questions that matter:

• How many items are in your catalogue? A few thousand SKUs behave very differently from 10 million.
• How fast does your catalogue change? Real-time indexing has a cost.
• Do you have user history, or are most sessions anonymous?
• What latency can you tolerate at inference? 50ms is achievable; 5ms with complex models requires hardware optimisation.
• Do you need explainability, for example for regulatory reasons or for merchandising teams who need to understand why an item was surfaced?

Write these down as hard constraints before you look at any product demo.

What Do Managed Recommendation Services Actually Give You?

The major cloud providers all offer managed recommendation services. AWS Personalize uses AutoML to train and serve models without you managing infrastructure. Google Cloud Recommendations AI is targeted at retail and uses the same models Google uses internally. Azure Personalizer uses reinforcement learning and is a reasonable fit for content ranking.

Third-party options like Recombee, Algolia Recommend, and Constructor.io sit further up the abstraction stack and include merchandising controls, A/B testing dashboards, and no-code rule overrides.

What you get with any of these:

• A working system in days, not months
• Managed infrastructure, including retraining pipelines
• Reasonable default models that are well-tested at scale

What you give up:

Concern	Managed service reality
Model transparency	You cannot inspect the model weights or architecture
Feature engineering	You're limited to the feature types the vendor supports
Latency control	You're on shared infrastructure with SLA floors, not ceilings
Cost at scale	AWS Personalize pricing is per event and per recommendation; at tens of millions of events per month, bills climb steeply
Data sovereignty	Training data leaves your environment unless you use a VPC deployment option

AWS Personalize, for instance, charges around $0.05 per 1,000 recommendations after the first 100,000 per month, plus data processing and training costs. That sounds small until you're running 50 million recommendations a month across a high-traffic platform.

When Does Building In-House Actually Make Sense?

Building makes sense when at least two of the following are true:

1. Your recommendation logic is a core differentiator, not a commodity feature
2. You have catalogue or interaction patterns unusual enough that general-purpose models underperform significantly (this is worth testing, not assuming)
3. You need full control over the training pipeline for compliance, auditability, or experimentation speed
4. You have the ML engineering capacity to maintain it, including retraining schedules, feature store management, and drift monitoring

The minimum viable internal stack for a production recommendation system is not trivial. You're looking at a feature store (Feast, Tecton, or a homegrown solution on Redis), an offline training pipeline (typically on Spark or Ray), a model registry (MLflow is the standard choice), an online serving layer (Triton Inference Server or TorchServe for deep learning models), and an A/B testing framework that's properly integrated with your analytics.

If you don't have at least two senior ML engineers who've shipped recommendation systems before, the timeline to production will surprise you. Six months is optimistic for a system that's actually reliable.

Collaborative Filtering vs. Content-Based vs. Hybrid

Most production systems are hybrid. Pure collaborative filtering breaks on cold-start. Pure content-based filtering misses serendipity and tends to over-recommend within narrow similarity bands. Hybrid approaches, where you train a two-tower model or use a stacking ensemble, handle both problems reasonably well.

The two-tower architecture (one tower for the user, one for the item, trained with contrastive loss) is currently the dominant approach for large-scale retrieval. Companies like Pinterest (PinSage) and YouTube have published their architectures. TensorFlow Recommenders (TFRS) and PyTorch's torchrec library both provide reasonable starting points.

For ranking after retrieval, gradient boosted trees (XGBoost, LightGBM) still outperform neural approaches on tabular interaction data in many benchmarks, despite the hype around deep ranking models.

The Retraining Problem Nobody Talks About

Building the model is 30% of the work. The other 70% is keeping it accurate over time.

User behaviour shifts. Catalogues change. New items have no interaction history. If you don't have automated retraining triggered by data drift detection (using something like Evidently AI or a custom KL-divergence monitor on your score distributions), your model's performance degrades silently. Managed services handle this for you. In-house builds require you to own it.

How Do You Evaluate Whether Your Chosen Approach Is Working?

Offline metrics like precision@K, recall@K, and NDCG are useful for iterating on models, but they are poor proxies for business outcomes. A model that scores well offline can underperform a simpler baseline in an A/B test because it's optimising for the wrong thing.

The metrics that matter in production:

• Click-through rate on recommendations (segment by new vs. returning users separately)
• Conversion rate attributed to recommendation clicks
• Coverage: the percentage of catalogue items that actually get recommended
• Serendipity: a harder metric, but important for avoiding filter bubbles

Run proper A/B tests, not holdout evaluations. Statistical significance at 95% confidence requires enough traffic to detect real effect sizes. If your platform doesn't have that traffic yet, managed services with built-in experimentation tools (Constructor.io has reasonable tooling here) will get you results faster than a homegrown system.

Conclusion

If personalisation is a commodity feature for your product and you need it working in weeks, buy. AWS Personalize or a dedicated vendor will get you 80% of the way there for a fraction of the engineering cost.

If your recommendation logic is core to your product's value, if your data is unusual, or if vendor pricing becomes punitive at your scale, build. Go in with clear eyes about what "build" actually requires: a feature store, retraining pipelines, a serving layer, and ML engineers who've done it before.

The next step is a data audit. Before any vendor POC or architecture design, pull three months of interaction logs and characterise them: event volume, unique users, catalogue size, repeat-visit rate, and cold-start ratio. That data will tell you more about which path to take than any vendor demo will.

---

FAQ

Is collaborative filtering still worth using in 2024? Yes, but rarely in isolation. Collaborative filtering works well for retrieval in large catalogues because it's fast and scales well with approximate nearest neighbour search (FAISS, ScaNN). Most teams combine it with content-based signals and a neural re-ranking layer. Pure collaborative filtering breaks badly on cold-start and sparse interaction data.

How much data do you need before a custom recommendation model outperforms a managed service? There's no universal threshold, but most practitioners find that below roughly 1 million monthly interaction events, managed services with their pre-trained priors outperform custom models trained from scratch. Above 10 million events, custom models tuned on your specific data distribution typically close the gap and often exceed managed baselines.

What's the biggest mistake teams make when building recommendation systems in-house? Underestimating the infrastructure work. The model is the visible part. The real cost is in the feature store, the retraining pipeline, data quality monitoring, and the serving layer that needs to hit latency SLAs under load. Teams that budget only for model development regularly find themselves six months behind schedule.

Can you switch from a managed service to a custom system later without restarting from scratch? Partially. Interaction event logs are portable if you own them. But if your training pipeline was tightly coupled to the vendor's data format or feature schema, migration is painful. Design your event logging layer to be vendor-agnostic from day one, even if you start with a managed service.

When does a recommendation engine genuinely hurt user experience? When it optimises for the wrong signal. Recommending items with the highest click rate, rather than the highest purchase or return-visit rate, can increase short-term engagement while decreasing satisfaction. This is especially common when the model is trained on implicit feedback without accounting for negative signals like returns, short dwell times, or repeated skips.

Have a project in mind? Contact Sodio Technologies to discuss your requirements and explore the right technology solution for your business.