SYSTEM Cited by 1 source

Hotstar Knol (internal Kafka data platform)

Knol is Hotstar's internal managed-Kafka data platform. From the 2024 Capturing a Billion Emo(j)i-ons post: "Hotstar has an amazing data platform called Knol built on top of Kafka which is flexible for all our use cases." Knol is cited as the reason product teams (e.g. the Emoji-swarm team) picked Kafka without having to own Kafka cluster operations — "managing Kafka on our own takes significant effort."

Knol is the canonical wiki instance of the "managed Kafka as an internal platform product" shape: a single central team owns Kafka clusters, schema management, partitioning, and operational health; many product teams consume it as a service.

See the Knol announcement post (referenced from the emoji-swarm article) for the original product framing.

Known consumers (from the 2024 post)

• systems/hotstar-emojis — emoji-swarm real-time ingest → aggregate → fanout pipeline. Raw emoji topic + aggregate-counts topic both live on Knol/Kafka.
• By extension: Voting, Polls, and Trivia share the same platform.

Seen in

• sources/2024-03-26-highscalability-capturing-a-billion-emojions — the emoji-swarm system uses Knol as the Kafka substrate for both the raw-ingest topic (producer-side async-buffered Go writers) and the aggregate topic (Spark Streaming micro-batch outputs read by a Python PubSub pusher).

Related

• systems/kafka — the underlying broker Knol is built on.
• systems/hotstar-emojis — the canonical consumer.
• systems/hotstar-pubsub — the sibling Hotstar real-time delivery platform.