.. bops documentation master file, created by
   sphinx-quickstart on Wed Oct 26 10:02:22 2011.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Full Examples
=============

This page contains three full script examples for using the bops module as well as a script to generate the test data.

 * A word count example
 * Data generation script
 * MapReduce script using sample data
 * A comparison of the 'mapreduce' and 'mapreducebatch' functions


**Mailing List**

A mailing list has been created to support the use of this module. You can join 
and follow the discussion on `Google groups <http://groups.google.com/group/python-bops>`_. Any errors, issues and enhancements can be discussed here.

Bops aims to be a top-notch data analysis module, but only with your help can this module actually be great. Please chime into the discussion. Your inputs are welcome as well as any suggested features, patches or fixes.


Word Count
----------

Word counting is the *Hello World!* for MapReduce. I've included an example using bops. The example finds the top 5 most frequent words in the paragraph, as well as a count of the words' starting letter in numeric order of occurrence.

There are basically four steps taking place.

 * Initialize data in a 'bop' instance
 * Write 'mapper' function
 * Run map reduce job using 'mapper' function and Python's built-in 'sum' function
 * Sort results by frequency and grab Top 5

.. literalinclude:: ./examples/word_count.py


Sample Data
-----------

This script generates the sample data for the example scripts. The generated 
data produces a CSV (comma-delimited file) which contains a list of people with 
names, ages, gender, years in college and number of friends. 


.. literalinclude:: ./examples/test_data_gen.py


Comparison of the two map reduce functions: 'mapreduce' and 'mapreducebatch'.

.. literalinclude:: ./examples/simple.py


Ever want to do a popularity analysis... OK, probably not, but here's how you could. 
This also shows performance testing to compare the two map reduce functions: 'mapreduce' and 'mapreducebatch'.

.. literalinclude:: ./examples/mr_perf_testing.py