Home       Trees       Indices       Help   
Package prest :: Module minr :: Class FacebookMiner
[hide private]
[frames] | no frames]

Class FacebookMiner

source code

object --+        
         |        
     Miner --+    
             |    
     TextMiner --+
                 |
                FacebookMiner

The Facebook user profile miner.

This miner use an access token to access a user facebook wall and retrieve
its text message. While the messages's text is retrieve, the miner write
them into a txt file. Once every messages have been write to the file the
miner generate n-grams from the file and insert them in the database, using
the L{minr.TextMiner} methods.

@note: The miner do not retrieve every facebook wall messages each time.
    When mining a facebook wall he saves the published date of the latest
    message and on next mining it will only retrieved the messages that have
    been published AFTER the saved date.
    See: L{minr.FacebookMiner.update_fb}

@todo 0.2.0:
    Create web app in order to log the user to facebook and twitter,
    authenticate them and ask for permissions (if needed) and finaly get
    access token.

G{classtree FacebookMiner}
Instance Methods [hide private]
 
__init__(self, config, minerName, callback=None)
Constructor of the FacebookMiner class.		 source code
 
mine(self)
Perform the mining operation.		 source code
 
write_to_file(self, post, fo)
Write a facebook post message to a file.		 source code
 
scrap_fb(self, posts)
Fully mine every posts of a facebook profile.		 source code
 
update_fb(self, posts)
Mine posts of a facebook profile since the last mining operation.		 source code
dict
get_user_details(self)
Use the facebook access token to get details about the user.		 source code
 
rm_db(self)
Override the parent method.		 source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

    Inherited from TextMiner
 
add_to_db(self, ngramMap, n, append=False)
Add n-grams of an n-gram dictionary to the database.		 source code
 
crt_new_db(self, textPath)
Mine a text file.		 source code
dict
crt_ngram_map(self, textPath, n)
Create a n-gram dictionary from a file.		 source code
 
update_db(self, textPath)
Mine a text file, updating the database.		 source code
Class Variables [hide private]
    Inherited from Miner
  __metaclass__ = abc.ABCMeta
Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, config, minerName, callback=None)
(Constructor)

source code 

Constructor of the FacebookMiner class.

Parameters:
  • config (drvr.Configuration) - The configuration file. It is used to retrieve the miner parameters.
  • minerName (str) - The name of the miner.
  • callback (fun(float, ...)) - The callback is used to show the progress percentage. In the gui a callback method is implemented to update a progress bar showing the n-grams insertion progress (cf. gui.py).
Overrides: object.__init__

mine(self)

source code 

Perform the mining operation.

Overrides: Miner.mine

write_to_file(self, post, fo)

source code 

Write a facebook post message to a file.

Parameters:
  • post (dict) - A facebook post is a dictionary. If the post contains a textual message then it is associated to the 'message' key.
  • fo (TextIOWrapper) - The file to write in.

scrap_fb(self, posts)

source code 

Fully mine every posts of a facebook profile.

Mine the posts contained in the "posts" parameter (which should contains the latest posts) and keep requesting older posts until we reach the last post. If a post contains a textual message it is automatically written in the file at self.fbFile. The latest post unix time is computed and written in the config.

Parameters:
  • posts (dict) - "posts" is returned by GraphAPI.get_connections(), it contains posts of a facebook profile.

Note: It is not possible to get every posts of a facebook profile in a single request (except if the profile contains very few posts).So The method must scan the "posts" dictionary and request the older posts until the request return a posts dictionary.

update_fb(self, posts)

source code 

Mine posts of a facebook profile since the last mining operation.

Mine the posts contained in the "posts" parameter (which should contains the latest posts) and keep requesting older posts until:

  • We reach a post that has already be mined (comparison is carried out using unix time and the 'last_update' config option)
  • We reach the last post of the facebook profile.

If a post contains a textual message it is automatically written in the file at self.fbFile. The latest post unix time is computed and written in the config so that we know which posts have been published after this one the next time the method is called.

Parameters:
  • posts (dict) - "posts" is returned by GraphAPI.get_connections(), it contains posts of a facebook profile.

Note: It is not possible to get every posts of a facebook profile in a single request (except if the profile contains very few posts). So The method must scan the "posts" dictionary and request the older posts until the request return a posts dictionary.

get_user_details(self)

source code 

Use the facebook access token to get details about the user.

Returns: dict
The user details or an empty dictionary if the request fail wich probably means that the access token is invalid or outdated.

rm_db(self)

source code 

Override the parent method.

This method delete the database file and also set the last_update option of the facebook miner to the oldest value possible so that the facebook account will be fully scraped on next mining operation.

Overrides: Miner.rm_db

   Home       Trees       Indices       Help   
Generated by Epydoc 3.0.1 on Tue Jun 16 23:30:30 2015 http://epydoc.sourceforge.net