Understanding of Overview of Data Analytic

Machine Learning is a primary focus. Cover concepts of probability, statistics and regression analysis.

 Related topics

  • Data mining
  • Statistical Learning
  • Pattern Recognition
More focus on algorithms which are going to be used to channelize data in useful patterns

Broad Classification
  • Supervised Learning
  • Unsupervised learning
These have already been defined in Introduction to Machine Learning.

Studying Further details:

Supervised Learning: Essentially a task of creating a function or a relationship on training data (historic data) - labeled data which has some explicit output variable. 

Functional and algorithmic mapping between these input and output variable.

Further Classified into
  • Classification Problem: 
    • Output variable is discrete categorical variable and not some continuous variable.
    • Predict as to which class the variable should belong to.
  • Regression Problem
    • Output variable is continuous variable. 
    • Marginally different from Regression Analysis
Unsupervised Learning: Task of creating patterns from data which have no explicit measure or signal guiding us - Data is not labeled. We have x variables.

Tools and Techniques
  • Advanced step towards Regression 
  • Logistic Regression (Categorical)
  • k-NN
  • Classification and Regression Trees - CART
  • Support Vector machine (SVM)
  • LDA/ QDA
  • ANN
  • Ensemble Methods
Major Topics in Unsupervised Learning:
  • Clustering: 
    • Task of grouping a set of objects into groups based on their similarities (common set of attributes/ features that these objects possess) 
      • k-NN
      • Hierarchical
      • Graph Based
      • Density based
  • Association Rule Mining (Market Basket Rule Mining)
    • Task of identifying relationship between features across the set of objects. 
    • Any variable can be part of rule mining
    • Not specific output.
    • Judge how good the rule is
Major Topics for Data Driven Decision Making

Design of Experiments
  • No Data
  • Need data
  • Conduct an Experiment
  • explicitly change the input variable and record data
Active Learning: 
  • Already have some data. 
  • Fairly expensive to gather this data. 
  • Partial knowledge. 
  • Sequentially query the system. 
  • Check which data we need to gather. 
  • Gather the critical data only in order to process it for insights
Area of reinforcement learning
  • No/ Partial data
  • No experiments to create data.
  • Possible because we cannot do experiments because it affects the end users (going offline)
  • Called Banded Problems
  • Form of experimentation or performance on data as well as you can online (live)
References:
Introduction to Data Analytic by www.nptel.iitm.ac.in 

Comments

Post a Comment

Popular posts from this blog

Statistics and Graphical Models in Data Science

Image
Statistics: For any aspiring data scientist, I would highly recommend   learning statistics with a heavy focus on coding up examples, preferably in Python or R. Mostly favorite series is the   Statistical Learning series. The SL Series is a   great primer on statistical modeling / machine learning with applications in R.(Reference by qoura.com) ·         Crucial component: Statistics is a crucial component of data science. At Twitch, The Professional data science team brings together three things: the first is statistics, second is  programming, and  third last is product knowledge. And we would never hire someone who wasn’t strong in stats. You can be a great programmer, but if you don’t know what Byes Rule is, then we have an engineering department I can point you to.” The origin in statistics is mostly undeniable. ·         Programming: Python:   Python is a mostly...
Read more

Significance of Woman in Data Science

Image
Research scientists, regardless of gender, are doing great in this field. This has been evidenced by a great conference " Global Women in Data Science " hosted by Stanford University (livestream) for 75+ locations all over the world on 3rd February 2017. Many talented academic researchers and practitioners have successful connected to make their dreams and goals burst into reality. It has been sponsored by Microsoft, Google, WalmartLabs, SAP and many more A Grade firms.  Fig 1: Global Women in Data Science Another great initiative, Women in Machine Learning   has been hosted on Facebook. Many female potential researchers join hands for research in machine learning. Many of them have achieved great milestones and many are still on the way.  The world is full of talented, successful and fantastic female speakers Women in Machine Learning and Data science are often un-noticable by traditional recruiters and we can turn it down. Another initiative at Meet...
Read more

Different types of clustering for textual documents

Clustering Model 1: Nearest Neighbour Algorithms: KD-Trees and Locality Sensitive Hashing Core Machine Learning Concepts: Distance Metrics and Approximation Algorithms Problem Domain: Finding similar documents Nearest neighbour: Find distance between all other documents and query document. Retrieve the document which is closest Critical Component Document Representation Computing Distances Scaling to large dataset: KD Trees - Not appropriate for very high dimensional documents Thus, Locality sensitive hashing is used for approximate nearest neighbour search. Clustering Model 2: Capturing uncertainty in clustering: Mixture Models Any document can be related to t i topic by x%. Can learn users topic preferences Making soft assignments to clusters -Expectation Maximization Clustering Model 3: Latent dirichlet allocation Mixed membership Probability of words in vocabulary Unsupervised learning task
Read more