/recommendation-system

name: Recommendation System
description: Build collaborative and content-based recommendation engines for product recommendations, personalization, and improving user engagement

Recommendation System

Overview

This skill implements collaborative and content-based recommendation systems with matrix factorization techniques to predict user preferences, increase engagement, and drive conversions through personalized item suggestions.

When to Use

• Developing recommendation features to improve user engagement and retention
• Implementing personalized product suggestions to increase sales and conversion rates
• Building hybrid recommendation systems that combine collaborative and content-based approaches
• Analyzing and optimizing recommendation coverage, diversity, and accuracy
• Handling sparse user-item interaction matrices and cold start scenarios
• Running A/B tests to measure the impact of recommendation algorithms on business metrics

Approaches

• **Collaborative Filtering**: Users similar to you liked X
• **Content-based**: Items similar to what you liked
• **Hybrid**: Combining multiple approaches
• **Matrix Factorization**: Latent factor models
• **Deep Learning**: Neural networks for embeddings

Key Metrics

• **Precision@K**: % recommendations relevant
• **Recall@K**: % relevant items found
• **NDCG**: Ranking quality metric
• **Coverage**: % items recommended
• **Diversity**: Variety in recommendations

Implementation with Python

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import NMF
import seaborn as sns

# Create sample user-item interaction data
np.random.seed(42)
users = [f'user_{i}' for i in range(100)]
items = [f'item_{i}' for i in range(50)]

# Generate ratings (sparse matrix)
ratings_list = []
for user in users:
    n_items_rated = np.random.randint(5, 20)
    rated_items = np.random.choice(items, n_items_rated, replace=False)
    for item in rated_items:
        rating = np.random.randint(1, 6)
        ratings_list.append({'user': user, 'item': item, 'rating': rating})

ratings_df = pd.DataFrame(ratings_list)
print("Sample Ratings:")
print(ratings_df.head(10))

# Create user-item matrix
user_item_matrix = ratings_df.pivot_table(
    index='user', columns='item', values='rating', fill_value=0
)

print(f"\nUser-Item Matrix Shape: {user_item_matrix.shape}")
print(f"Sparsity: {1 - (user_item_matrix != 0).sum().sum() / (user_item_matrix.shape[0] * user_item_matrix.shape[1]):.2%}")

# 1. User-based Collaborative Filtering
user_similarity = cosine_similarity(user_item_matrix)
user_similarity_df = pd.DataFrame(
    user_similarity, index=user_item_matrix.index, columns=user_item_matrix.index
)

print("\n1. User Similarity Matrix (Sample):")
print(user_similarity_df.iloc[:5, :5])

# Get recommendations for a user
def get_user_based_recommendations(user_id, user_sim_matrix, user_item_mat, n=5):
    similar_users = user_sim_matrix[user_id].sort_values(ascending=False)[1:11]

    recommendations = {}
    for item in user_item_mat.columns:
        if user_item_mat.loc[user_id, item] == 0:  # Not yet rated
            score = (similar_users * user_item_mat.loc[similar_users.index, item]).sum()
            recommendations[item] = score

    top_recs = sorted(recommendations.items(), key=lambda x: x[1], reverse=True)[:n]
    return [rec[0] for rec in top_recs]

# Example: Get recommendations for user_0
user_recommendations = get_user_based_recommendations('user_0', user_similarity_df, user_item_matrix)
print(f"\nRecommendations for user_0: {user_recommendations}")

# 2. Item-based Collaborative Filtering
item_similarity = cosine_similarity(user_item_matrix.T)
item_similarity_df = pd.DataFrame(
    item_similarity, index=user_item_matrix.columns, columns=user_item_matrix.columns
)

print("\n2. Item Similarity Matrix (Sample):")
print(item_similarity_df.iloc[:5, :5])

# 3. Content-based Filtering
item_features = np.random.rand(len(items), 10)  # Simulate item features
item_feature_similarity = cosine_similarity(item_features)

fig, axes = plt.subplots(2, 2, figsize=(14, 10))

# User similarity heatmap
sns.heatmap(user_similarity_df.iloc[:10, :10], annot=True, fmt='.2f', cmap='coolwarm',
            ax=axes[0, 0], cbar_kws={'label': 'Similarity'})
axes[0, 0].set_title('User Similarity Matrix (Sample)')

# Item similarity heatmap
sns.heatmap(item_similarity_df.iloc[:10, :10], annot=True, fmt='.2f', cmap='coolwarm',
            ax=axes[0, 1], cbar_kws={'label': 'Similarity'})
axes[0, 1].set_title('Item Similarity Matrix (Sample)')

# Rating distribution
axes[1, 0].hist(ratings_df['rating'], bins=5, color='steelblue', edgecolor='black', alpha=0.7)
axes[1, 0].set_xlabel('Rating')
axes[1, 0].set_ylabel('Count')
axes[1, 0].set_title('Rating Distribution')
axes[1, 0].grid(True, alpha=0.3, axis='y')

# Sparsity by user
user_rating_counts = user_item_matrix.astype(bool).sum(axis=1)
axes[1, 1].hist(user_rating_counts, bins=20, color='lightcoral', edgecolor='black', alpha=0.7)
axes[1, 1].set_xlabel('Number of Rated Items')
axes[1, 1].set_ylabel('Number of Users')
axes[1, 1].set_title('User Activity Distribution')
axes[1, 1].grid(True, alpha=0.3, axis='y')

plt.tight_layout()
plt.show()

# 4. Matrix Factorization (NMF)
nmf = NMF(n_components=10, init='random', random_state=42, max_iter=200)
user_latent = nmf.fit_transform(user_item_matrix)
item_latent = nmf.components_.T

print(f"\n4. Matrix Factorization:")
print(f"User latent factors shape: {user_latent.shape}")
print(f"Item latent factors shape: {item_latent.shape}")

# Reconstruct ratings
reconstructed_ratings = user_latent @ item_latent.T
reconstructed_df = pd.DataFrame(
    reconstructed_ratings, index=user_item_matrix.index, columns=user_item_matrix.columns
)

# Calculate RMSE
original_ratings = user_item_matrix[user_item_matrix > 0]
pred