Files and Pipes in R Video Demo

I’ve worked with various alternate file handlers in python before and wanted to explore the options in R. I was pleasantly surprised to find handlers prebuilt for tasks like compressing data. In addition, a pipe function is available to allow you to use less common commands on your file, like gpg for encryption.

I put together a quick video demo of how to use these functions, and it’s available on youtube:

If you are having a hard time reading the text, click here to view the video directly on youtube.

Comment here or on the video with any feedback or questions.

Installing pymc on OS X using homebrew

I’ve been working through the following book on Bayesian methods with an emphasis on the pymc library:

However, pymc installation on OS X can be a bit of a pain. The issues comes down to fortran… I know. The version of gfortran in newer gcc implementations doesn’t work well with the pymc build, you need gfortran 4.2, as provided orignally by apple. Homebrew has a package for this.

I dealt with this before, but had problems again after upgrading to Sierra. So this time, I thought I’d document the steps so I don’t have this problem again. Let me know if there are any steps that you feel need added as you try this.

Installing TensorFlow on CentOS

Google released TensorFlow as open source for community use and improvement. From the site: “TensorFlowâ„¢ is an open source software library for numerical computation using data flow graphs.”

The instructions on tensorflow.org are aimed at Ubuntu and OS X. I had a need to install it on CentOS so I documented the steps in a github gist. Feel free to comment if you find something I missed:

* Updated 8/18/2016 for TensorFlow 0.10
* Updated gist 10/18/2016 to correct typo in epel-release

Running a Local ElasticSearch Cluster for Development

ElasticSearch is a document database built on Lucene, a full text-search engine. It clusters and is useful in a variety of scenarios. If you want to run it locally and test some of the clustering feature, here are some things I learned from my experience.

Install with your preferred package manager, or from source. In my case I use homebrew, so install is as easy as:

brew install elasticsearch

You can run multiple nodes from one install. First, you will want to tweak the config. For me, the elasticsearch.yml config file was located in /usr/local/Cellar/elasticsearch/1.4.0/config/, but that may vary based on your OS, package manager, or version.

Mainly, I set a value for cluster. name:

cluster.name: some_cluster_name_of_your_choosing
marvel.agent.enabled: false

By default, elasticsearch joins any cluster with the same name, so you do not want to run on the default or you will be syncing with other local developers on your network.

Then you want to configure your startup mechanism. For some users, that would mean configuring the elasticsearch file in /etc/init.d, but for OS X homebrew users, my startup file is ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist. First, setup extra nodes. In OS X, that means making copies of the LaunchAgent file in the same directory with unique names. It’s worth noting if you’re using homebrew, that the original LaunchAgent plist file is a symlink and you’ll want to copy the contents to a new file. I numbered my extra nodes, so the filenames were homebrew.mxcl.elasticsearch2.plist, and so on. I altered the plist file to have a couple of extra arguments, specified in the ProgramArguments node. I removed the xml nodes that specify keeping the worker alive. The results were:

I set custom node names and ports via the plist file. Note that on a Linux based machine, you would later the service to run multiple ElasticSearch nodes with those custom parameters.

Finally, since I didn’t want to run those scripts individually each time, I create a script to launch all nodes at once. Note that you’ll want to add execute permissions and put it somewhere in your path.

Finally, I recommend installing Marvel, particulary for Sense, a nice tool for running commands on ElasticSearch. You can install Marvel with this command in the ElasticSearch root directory (for me, /usr/local/Cellar/elasticsearch/1.4.0/):

bin/plugin -i elasticsearch/marvel/latest

Now you can bring the cluster up with es_cluster

Enjoy!

VicinityBuzz Update: Windows Phone 8 & More

VicinityBuzz on Win Phone 8

While attending Codemash a few weeks ago, I ended up in a Windows Phone development precompiler (Codemash’s name for a training session). It was my plan to hit mostly mobile and analytics sessions, but I was not originally planning on attending this session. With Windows Phone still struggling for market share, I wasn’t in a rush to work with it. However, other sessions were cancelled because weather had delayed some presenters, so I ended up in this session. Microsoft’s Jeff Blankenburg was teaching the session, and I have enjoyed some of his presentations and a Silverlight fire-starter event in the past. It’s one of my rules of conferences to attend sessions based more on good speakers, rather than based solely on topic.

With regards to marketshare, Jeff made the point during the session that with a less crowded app store, you do have a bit more discoverability. Even if that doesn’t hold up, the platform shares enough similarity with Windows 8 that a port to the Windows Store will be trivial. The Windows App Store isn’t exactly setting the world on fire either, but I’d like to see my app on all of these platforms, and as Windows 8 adoption rises with new machine sales, that marketplace should see constant upticks.

Having worked with Silverlight in the past, I found it pretty easy to get going on Win Phone 8 development. There was some definite rust on my XAML skills, but it came back to me fairly quickly. One thing to keep in mind is that you want to keep things relatively simple on a mobile platform. I have worked on some WPF projects in enterprise settings with MVVM frameworks, dependency injection frameworks, and more. While I followed an MVVM pattern, I just rolled my own with a simple base class.

My project was to do a version of an app I already have in the iOS App Store, VicinityBuzz. It does location based searches of twitter. You can search around you, or by entering an address. The radius is a configurable setting. I like using the app at conferences like Codemash to catch all the chatter that may not have a hashtag. One catch is that obviously only tweets that included location will be found. If folks have that feature turned off in their twitter app, then it won’t show up.

Since I had written the app before (in phonegap for iOS), I knew the feature set and domain cold. The challenge was just getting up to speed with the latest API’s for search and geolocation, and then implementing within a new platform. One of the biggest benefits of this project was getting up to date with the latest Twitter API. I still need to update the version for iOS, as it’s currently non-functional because of api changes over the last several years. I plan on doing that very soon now that I know the latest version.

Anyway, I won’t go into the development details here too much, but I finished a version 1 of VicinityBuzz, and it is now in the windows phone store here, and it’s free. So go check it out. If you like it, I’d love to have some more reviews.

Also, if you are inspired to do any Windows Phone development yourself, you may be interested in a device to do some real testing. I recently found there are some prepaid phones new on Amazon that are dirt cheap for that purpose. Check out the Nokia Lumia 520 and Nokia Lumia 521 on Amazon.

Watch this blog for upcoming posts about working with the Twitter API, and some of the things I learned working with Windows Phone 8. And more mobile in general. I have the bug again…