InputStream

Tweet

shellstreaming.istream.tweet

synopsis:Generates public tweets sequence.

class shellstreaming.istream.tweet.Tweet(public_tweets_url, consumer_key, consumer_secret, access_token, access_token_secret, records_in_batch=100, **kw)

Bases: shellstreaming.istream.base.Base

Infinite input stream which generates public tweets sequence.

Note

Not every attribute is fetched from API. You need to fix Tweet to add other attributes, or develop another istream. See Tweets for further attributes.

__init__(public_tweets_url, consumer_key, consumer_secret, access_token, access_token_secret, records_in_batch=100, **kw)

Constructor

Parameters:

• public_tweets_url – Public tweet streaming API’s URL
• consumer_key – Twitter app consumer key (got from twitter)
• consumer_secret – Twitter app consumer secret (got from twitter)
• access_token – Twitter app access token (got from twitter)
• access_token_secret – Twitter app access token secret (got from twitter)

Raises :

requests.HTTPError if twitter API returns error response status

run()

Fetches tweets from Twitter public stream

TextFile

shellstreaming.istream.textfile

synopsis:FiniteStream for text files.

A line is used as record with single column.

class shellstreaming.istream.textfile.TextFile(path, records_in_batch=100, **kw)

Bases: shellstreaming.istream.base.Base

FiniteStream for text files

__init__(path, records_in_batch=100, **kw)

Constructor

Parameters:path – path to text file

run()

Reads a text file line-by-line until EOF