Every application has a narrator commenting its execution, be it a humble println or a
more structured log. But this narrator
is unreliable: It decides what's important and what's not, forgets to mention the
parts of the plot, and usually rambles for gigabytes.
Obviously, though, there are more things coming out of your application: Metrics, tracing
events, all the system chatter
that surrounds a running process. Every single sub-system supporting your application, be
databases, message queues, git dags, which deep down are logs.
Meanwhile, architects have been promoting Event Sourcing as a full and faithful
representation of the application's business.
From the "domain" emerges a well-defined language, exhaustively describing all event
the state of the application and focusing on its dynamics.
But what if there was
One Log? What if we used well structured messages, integrating in a
dtrace application logs, iostat metrics, prometheus signals, and domain events. What if
relinquished up-front filtering and throttling and let serendipity do its job? What if
separate realms of information (business events, kibana views, grafana boards) we work
were just views of a big stream of log events?
This session will be a live-running experiment pushing knobs to 11 and explore what
information we can harvest from this
hoard of data. Building upon a simple event source application, we'll aggregate more
implement traffic replay as a reverse event log, embrace system logs, and see what
when we don't silence the narrator but let it loose.
Arnaud Bailly & Yann Schwartz