Extract Data from Twitter and Format of Twitter Tweet

After describing the steps to analyse data in Twitter, here goes the code to extract data from Twitter. Before you proceed with the code below. Perform following steps
  1. Download and Install Python
  2. Download and Install Tweepy
  3. Download and Install Requests (if required)
  4. Register application at https://dev.twitter.com/appshttps://dev.twitter.com/apps

Data extraction Code

#-------------------------------------------------------------------------------
# Name:        module2
# Purpose:
#
# Author:      Muskan
#
# Created:     10-06-2015
# Copyright:   (c) Muskan 2015
# Licence:     <your licence>
#-------------------------------------------------------------------------------
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

def main():
    pass

if __name__ == '__main__':
    main()

ckey = 'your consumer key'
csecret = 'your consumer secret'
atoken = 'your access token'
asecret ='your consumer secret'

class listener(StreamListener):
    def on_data(self, data):
        print data
        return True
    def on_error(self, status):
        print status

auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken,asecret)
twitterStream = Stream(auth,listener())
twitterStream.filter(track=["\"location\"=\"Chandigarh\""])

Format of Twitter Data

You will get data in the following format:

{
    "created_at": "Thu Jun 11 09:31:03 +0000 2015",
    "id": 608929425472626700,
    "id_str": "608929425472626688",
    "text": "XXXXXXXXXXXXXXXXXXTEXTXXXXXXXXXXXXXXXXXXXX",
    "source": "<a href=\"http://dlvr.it\" rel=\"nofollow\">dlvr.it</a>",
    "truncated": false,
    "in_reply_to_status_id": null,
    "in_reply_to_status_id_str": null,
    "in_reply_to_user_id": null,
    "in_reply_to_user_id_str": null,
    "in_reply_to_screen_name": null,
    "user": {
        "id": 1205763398,
        "id_str": "1205763398",
        "name": "WalterBrianna",
        "screen_name": "WalterBrianna1",
        "location": "",
        "url": null,
        "description": null,
        "protected": false,
        "verified": false,
        "followers_count": 131,
        "friends_count": 2,
        "listed_count": 14,
        "favourites_count": 0,
        "statuses_count": 116804,
        "created_at": "Thu Feb 21 19:09:36 +0000 2013",
        "utc_offset": 7200,
        "time_zone": "Pretoria",
        "geo_enabled": false,
        "lang": "en",
        "contributors_enabled": false,
        "is_translator": false,
        "profile_background_color": "C0DEED",
        "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png",
        "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png",
        "profile_background_tile": false,
        "profile_link_color": "0084B4",
        "profile_sidebar_border_color": "C0DEED",
        "profile_sidebar_fill_color": "DDEEF6",
        "profile_text_color": "333333",
        "profile_use_background_image": true,
        "profile_image_url": "http://pbs.twimg.com/profile_images/3304904615/f057106eb323b476e9ef1f56b1790d0a_normal.jpeg",
        "profile_image_url_https": "https://pbs.twimg.com/profile_images/3304904615/f057106eb323b476e9ef1f56b1790d0a_normal.jpeg",
        "default_profile": true,
        "default_profile_image": false,
        "following": null,
        "follow_request_sent": null,
        "notifications": null
    },
    "geo": null,
    "coordinates": null,
    "place": null,
    "contributors": null,
    "retweet_count": 0,
    "favorite_count": 0,
    "entities": {
        "hashtags": [],
        "trends": [],
        "urls": [],
        "user_mentions": [],
        "symbols": []
    },
    "favorited": false,
    "retweeted": false,
    "possibly_sensitive": false,
    "filter_level": "low",
    "lang": "en",
    "timestamp_ms": "1434015063051"

}

Share your views regarding the same.
Meet Me at Twitter or Facebook

Comments

Post a Comment

Popular posts from this blog

Different Fields for Data Science

Image
Data science , also known as   data-driven science , is an interdisciplinary field about scientific methods, processes and systems to extract   knowledge   or insights from   data   in various forms, either structured or unstructured,similar to   Knowledge Discovery in Databases   (KDD). Data science is a " concept to unify statistics, data analysis and their related methods " in order to "understand and analyze actual phenomena" with data.   It employs techniques and theories drawn from many fields within the broad areas of   mathematics,   statistics,   information science, and   computer science, in particular from the subdomains of   machine learning,   classification,   cluster analysis,   data mining,   databases, and   visualization. Turing award   winner   Jim Gray   imagined data science as a " fourth paradigm " of science (empirical,   theoretical, computationa...
Read more

Different types of clustering for textual documents

Clustering Model 1: Nearest Neighbour Algorithms: KD-Trees and Locality Sensitive Hashing Core Machine Learning Concepts: Distance Metrics and Approximation Algorithms Problem Domain: Finding similar documents Nearest neighbour: Find distance between all other documents and query document. Retrieve the document which is closest Critical Component Document Representation Computing Distances Scaling to large dataset: KD Trees - Not appropriate for very high dimensional documents Thus, Locality sensitive hashing is used for approximate nearest neighbour search. Clustering Model 2: Capturing uncertainty in clustering: Mixture Models Any document can be related to t i topic by x%. Can learn users topic preferences Making soft assignments to clusters -Expectation Maximization Clustering Model 3: Latent dirichlet allocation Mixed membership Probability of words in vocabulary Unsupervised learning task
Read more

Significance of Woman in Data Science

Image
Research scientists, regardless of gender, are doing great in this field. This has been evidenced by a great conference " Global Women in Data Science " hosted by Stanford University (livestream) for 75+ locations all over the world on 3rd February 2017. Many talented academic researchers and practitioners have successful connected to make their dreams and goals burst into reality. It has been sponsored by Microsoft, Google, WalmartLabs, SAP and many more A Grade firms.  Fig 1: Global Women in Data Science Another great initiative, Women in Machine Learning   has been hosted on Facebook. Many female potential researchers join hands for research in machine learning. Many of them have achieved great milestones and many are still on the way.  The world is full of talented, successful and fantastic female speakers Women in Machine Learning and Data science are often un-noticable by traditional recruiters and we can turn it down. Another initiative at Meet...
Read more