Open source data collection systems for monitoring large distributed systems.
OS Solutions: Apache Flume built by Cloudera Apache Chukwa built by Yahoo
Scribe (Log server)
Scribe is a server for aggregating log data streamed in real time from a large number of servers. Built on top of Apache Thrift framework (RPC library). Programming language: mostly C/C++
Logging Messages to Scribe from PHP
Scribe works beutiful with Zookeeper (special branches on github), and with Thrift. This guy did most of the stuff: traviscrawford/scribe Add zookeeper-based network store discovery. · 2e71b14 · traviscrawford/scribe
On top of Zookeeper
Logging system: bookkeeper is a ZooKeeper contribution, that receives your logs and assures they’ll be kept save.
Built by LinkedIn. Distributed publish-subscribe messaging system. Part of it is Zookeeper - is essential for the correct operation of Kafka.
The use for activity stream processing makes Kafka comparable to Facebook’s Scribe or Apache Flume (incubating), though the architecture and primitives are very different for these systems and make Kafka more comparable to a traditional messaging system.
kafka-php allows you to produce messages to the Kafka distributed publish/subscribe messaging service kafka/clients/php at master · quipo/kafka Lorenzo Alberton - Articles - Updated Kafka PHP client library