Text Processing in R Talk With the TM Package

I gave a talk at my local Cleveland R User Group about text processing and document vectorization. You can view the talk here:

Note that I’m using the tm package, which is the traditional way to work with a document collection in R. There are new ways like tidytext that are gaining popularity. I may do a follow up talk on that.

Feedback, and More Videos

Enjoy, and feedback is welcome! And if you are interested in more video content on machine learning in R, check out this post.

Simulating the Monty Hall Problem in R.

The Monty Hall Problem is famous in the world of statistics and probability. For those struggling with the intuition, simulating the problem is a great way to get at the answer. Randomly choose a door for the prize, randomly choose a door for the user to pick first, play out Monty’s role as host, and then show the results of both strategies.

Simulating Monty Hall in R
Simulating the strategies of Monty Hall

The numeric output will vary, but look something like:

> print(summary(games$strategy) / nrow(games))
stay switch
0.342 0.658

The following code does this in a rather short R example:

The Math of Machine Learning

Matrix multiplication diagram
(hover for CC attribution)

One of the challenges of data science in general is that it is a multi-disciplinary field. For any given problem, you may need skills in data extraction, data transformation, data cleaning, math, statistics, software engineering, data visualization, and the domain. And that list likely isn’t inclusive.

One of the first questions when it comes to machine learning in specific, is “how much math do I need to know?”

This is where I would recommend you start, to get the most value for your time:

  • Matrix Multiplication (Subject: Linear Algebra)
  • Probability (Subject: Statistics)
  • Normal Distributions (Subject: Statistics)
  • Bayes Theorem (Subject: Statistics)
  • Linear Regression (Subject: Statistics)

Of course you will run across other math needs, but I think the above list represents the foundation.

If you need places to get started with those topics, check out Kahn Academy, Coursera, or your location library.

For more on machine learning, check out other posts such as ML in R, Linear Algebra in R, and ML w/XGBoost.

Presentation on Linear Algebra in R

At our January meeting, I presented on Linear Algebra basics in R. I have been taking the Andrew Ng’s Stanford Machine Learning course. That course primarily uses Matlab (or Octave, and open source equivalent), and machine learning involves manipulating and calculating with matrices. Naturally, being an R person, I have been working with some of the techniques in R.

In order to limit the scope of the talk, I focused on matrices, vectors and basic operations with them. There is a practical example that uses a machine learning algorithm, but it’s just to show how R handles a more involved equation with matrices. The talk is not an attempt to teach machine learning.

The slides are available here, and comments or suggestions are welcome.

Mapping Functions in R

I have been buffing up on some areas of Math that I felt rusty in. One of the tools I was using was my old TI-82 calculator I used in high school.

TI-82 Graphing Calculator

I even got the TI Connect software working where I could download screenshots, etc.

TI-82 Screen Cap of Sin, Cos, Tan

You can put in data sets (see screen cap below), and do some graphing and some statistical analysis.

Data Entry on TI-82

However, mapping functions is a more straight forward use of the calculator. Which got me thinking… How does one do that in R?

I did a little digging. With the traditional R graphics and plotting functions, you would use curve() to draw a function. It works fine, but I like to use ggplot2 when I can.

Turns out ggplot2 supports this well. The following code sample maps the same functions I was mapping on the calculator (sin, cos, and tan from 0 to 2pi radians).

And the resulting graph is…

ggplot2 Trig Function Graphing