Posts

Showing posts with the label Data Science

Schedule for Machine Learning

Books to be followed hands on machine learning with scikit learn and tensorflow 1.4.2018 Introduction to machine learning 2.4.2018 Introduction to statistics and probability 3.4.2018 Understanding Basics of Python and modules required 4.4.2018 Regression (Linear and Logistic) 5.4.2018 Implementing regression techniques 6.4.2018 Other regression techniques and differences 7.4.2018 Implementing other techniques 8.4.2018 Classification (Decision Tree, Naïve Bayes and SVM) 9.4.2018 Implementing 10.4.2018 Other classification techniques 11.4.2018 Implementing 12.4.2018 Discussions with doubts 13.4.2018 Clustering techniques (k means and hierarchiacal) 14.4.2018 Implementing 15.4.2018 Other cl...

Different Fields for Data Science

Image
Data science , also known as   data-driven science , is an interdisciplinary field about scientific methods, processes and systems to extract   knowledge   or insights from   data   in various forms, either structured or unstructured,similar to   Knowledge Discovery in Databases   (KDD). Data science is a " concept to unify statistics, data analysis and their related methods " in order to "understand and analyze actual phenomena" with data.   It employs techniques and theories drawn from many fields within the broad areas of   mathematics,   statistics,   information science, and   computer science, in particular from the subdomains of   machine learning,   classification,   cluster analysis,   data mining,   databases, and   visualization. Turing award   winner   Jim Gray   imagined data science as a " fourth paradigm " of science (empirical,   theoretical, computationa...

Identification of Topics from tweets

Using tweets, we identified the topics of today as: aninews indian pakistan army successful counter terrorism operations terror groups aninews indian surgicalstrike army the boys played really well jai hind rajnathsingh congratulations army surgicalstrike indian successful counter terrorism operations terror groups india indian surgicalstrike army counter terrorism operations terror groups killed army surgicalstrike indian successful counter terrorism operations terror groups virendersehwag salute army indian successful terrorism operations terror groups manoharparrikar congratulations indian successful terrorism operations terror groups manoharparrikar congratulations indian surgicalstrike army successful counter terrorism operations terror groups Automatically identification of topics from large dataset is another challenge in industry. The information has been identified using recent tweets of the day. Many people are congratulating brave Indian Army. This is indeed ...

Data Extraction from Social Media Websites

There are ample of ways to extract data from Twitter because it is available in public domain. The implementation code and procedure to extract data from twitter has been explained in detail here . Moreover, data can be extracted from Meetup.com as explained . These implementations have been performed in Python language.  As per analysis, popular profiles have been obtained using the same code of Python. Such analysis can be carried out for both commercial marketing purpose and ordinary audience surfing purpose. This can help to serve community in better ways. Other source of standard dataset for testing Twitter analysis are as follows: http://socialcomputing.asu.edu/uploads/1296759055/Twitter-dataset.zip https://ia800501.us.archive.org/32/items/twitter_cikm_2010/twitter_cikm_2010.zip http://socialsensor.eu/results/datasets/72-twitter-tdt-dataset http://snap.stanford.edu/data http://ipv6.nlsde.buaa.edu.cn/zhaojichang/paper/wntm.rar http://mklab.iti.gr/files/eve...

Are Hadoop and Spark Important for Data Science?

Data science is very new concept. Many institutes around the world found data science as source of their income. Apart from managing data, data science deals with new research ideas from which data can be converted into actionable format. Although technical training for data science deals with database, data mining, machine learning techniques, data warehousing, SAP and much more. However, a new idea for development gives life to data science project. Data science project includes Automatic car Internet of Things Google Glass Gesture recognition based brain games Virtual reality self running cycle and much more. There are ample of local problems of society which can be solved using data science subject. Understanding of existing projects and research areas gives wide scope of development and improvement in this field.  It is not good idea to go for Hadoop and Spark type projects initially. One can always start with existing machine learning techniques an...

Social Media Marketing Strategies for Business

India is booming with startups in 2015 era. It has been rightly said that today, a cool dude is not the one who knows guitar, but the one who is crazy for his passions and own a startup. Startup can be of any kind. With this increase in number, there exist need to retain audience. Customer relations are very important for any and every business.  As people tend to lose interest in advertisements on radio and television, crowd becomes more smart and discuss about brands. This intellectual behavior of people is ever changing. To study this customer behavior is need of the hour. This can be done by understanding what people tweet, post or share on social media websites. From twitter tweets to youtube videos, there is lot of information available on web regarding human behavior.  There is need to study social media data and its analysis is required. Different strategies are prepared for customer retention, awareness of local/ global brands, marketing etc. Thus, differe...

Networks and Games

"Networks and Games" workshop held at IIT Ropar was a great platform to share knowledge regarding network sciences and game theory. This was held from 7th of December 2015 to 9th of December 2015. There is considerable amount of work which is being carried on nodes homophily, cascading information, coalition etc. These concepts are widely being practices by academic researchers and practitioners. Moreover, nash equilibrium, 80:20 rule, utility models are few other major areas of research. Making analysis by extracting information from network among nodes, predicting information about what is going to get happen in future are good field of related research. This work is related to Network theory and game theory. However, there are many other workshops and conferences related to similar areas which are happening around the world. Data science is the emerging area of research.

The basics of Social Network Analysis

Social Media Analysis is known better by Linton C. Freeman who developed fundamentals of social media analysis in 2004. It is multi-disciplinary. Basic Vocabulary: Node (Vertex) Link (Edge) Node is a person's profile id/ wikipedia article/ email address/ twitter name/ facebook name etc. Link is from one node to another Link can be directed or undirected. In social media analysis, we use only directed link. Two-way links are said to be reciprocated links One-way link are called unreciprocated links Degree is the number of ties to other nodes/ actors in the network. Indegree: Count of number of nodes connected to the nodes (Popularity). Outdegree: Count of number of ties that a node directs to others. (Gregariousness). Density: How close the edges are towards each other. More the number of links, more is the density.  Graph with few edges is called sparse graph Bridges is the edge which if removed, makes the graph disconnected. Centrality: ...

Industry is Looking for Way to Smarter Business

All of the data that generated till 2003, we build in 2 days today. But much of it is unstructured and senseless. To make some analysis on that data would have been very very difficult about five years ago. Business managers realize that, in order to get competitive in market, you should be able to get lot of data which is available and realize your potential towards betterment. It is all about discovering interesting ways to position yourself in public sector. Predict data from millions of tweets, facebook posts and lot of data available on web. Every other client/ customer is asking for analysis of their data in IT industry and even in others. Fifteen years download the line, Analytics will be so regular and common skills set, that there will be no need to discuss about it anymore. One of the most promising factor is to predict the behavior of the existing clients/ customers. All the companies are trying to reach human being for which they need to look the behavior of their ...

Brief Introduction to Data Science and Hadoop

Getting data and performing some analysis to give visualization of data. This should be accompanied by results comparable to some ground truth. Data Science refers to an emergence area of work concerned with collection, preparation, analysis, visualization, management and preservation of large collections of information [1]. Although we studied that Data scientist is introduced with question and big data, but Data Science is about building data products and not just answering the questions Data Science in relation with ·          Business intelligence ·          Statistics ·          Database management ·          Visualization ·          Machine learning Future Scope for Data Science: 1.       Databases: Unstructured Data 2.     ...

Difference between Hadoop, HDFS, Big Data, MapReduce, Data Science | Who is data scientist

Data Science Data Science is the science to study data and convert it into actionable form. . For this, there are number of frameworks available. Data Science inherently deals with huge amount of data which is analyzed by framework.               Terminologies of Data Science Data Science is an art for converting the big data to actionable form. Big Data, huge amount of data, is the input for this processing. Hadoop and Spark are frameworks to work with Big Data and perform Data Science over it. HDFS (Hadoop Distributed File System) is the concept of Hadoop storage through which it distributes data over multiple nodes and maintains it thereby. Map Reduce is the concept of machine learning when data is mapped (grouped) on basis of one criteria and Reduce in another form (on the basis of other criteria).            Data Scientist Data scientist is someone who finds new discoveries with data. They...