AI Engine for generating personalized content and recommendations
Our client, who owns a commercial social media platform needed to implement an AI-driven recommendation engine that generated better or more relevant content for its users.
Menerva implemented an AI engine that automated the generation of recommendation periodically using a combination of deterministic and collaborative filtering algorithms, and taking into account factors such as content recency, the frequency of access and other factors.
Apache Spark and the pytorch libraries were leveraged to implement this AI engine.
As the data scale was very big, there was significant performance scaling and tuning needs specific to Apache Spark and the custom client data sets that were addressed.
The resulting engine proved to be both effective and scalable, given the large data sizes that are constantly growing. The system is successfully running in production.