Building low latency, user-responsive real-time features

by Yongliang Zhao and Zhi Chai

Everyone in game development knows users are dynamic: user characteristics may change at any time. For example, after a user makes a payment, if the user’s characteristics are still set for non-paying user status, it will affect how effective recommendation engines are in pushing messages, alerts, incentives, etc. So it is very important to build what we call low latency, real-time features for the best, most responsive user experience.

At Happy Elements, we have built a user-features-pipeline. In this pipeline, the user behavior logs (client-side) are sent to the log collector. The log collector inserts data into Kafka as the producer.

Kafka starts a service to consume these data. It then filters out the effect of features change information from the consumed data and then stores them in redis. To speed up redis updates, we use the command “hincrby“ to send updates to redis through the pipe mode.

At present, the lag from user actions to updates of user characteristics is generally within 2s-10s. We typically process 5.2 billion data points per day.