Mozilla Monolith¶
Note
monolith is still under heavy development
Mozilla Monolith is a project to provide statistic gathering, aggregation, a web-service API and a dashboard.
The first consumer of Monolith is the Firefox marketplace. Statistics include amongst others public global page views / hits, application specific downloads and even payment related information.
This aggregator part deals with gathering data from multiple sources, bringing them into a common format and storing them.
Currently data is stored in MySQL for durable archival and ElasticSearch is used provide the actual data access for the web-service and dashboard.
The web-service and dashboard are implemented in monolith.
There’s also a Python client library: monolith-client.
Here’s the high-level overview of the whole system:
monolith-aggregator adds data to a MySQL database, using a command-line script called monolith-extract.
This script that grabs data from various sources and adds them in the SQL Database.
The script also indexes all the content in Elastic Search, which can be used to do time-series queries on the data.
The SQL database is the single source of truth of Monolith. The Elastic Search can be recreated at anytime for any date range, using the SQL database.
A typical setup is to run monolith-extract every day as a cron.
Detailed Documentation¶
Source Code¶
All source code is available on github under monolith-aggregator.
Further information¶
You an read up on some background and motivation.
There is also a detailed product specification available to the public.
We also keep some progress information in an etherpad.
License¶
monolith-aggregator is offered under the MPL 2.0.