ElasticSearch is a document database built on Lucene, a full text-search engine. It clusters and is useful in a variety of scenarios. If you want to run it locally and test some of the clustering feature, here are some things I learned from my experience.
Install with your preferred package manager, or from source. In my case I use homebrew, so install is as easy as:
brew install elasticsearch
You can run multiple nodes from one install. First, you will want to tweak the config. For me, the elasticsearch.yml config file was located in /usr/local/Cellar/elasticsearch/1.4.0/config/, but that may vary based on your OS, package manager, or version.
By default, elasticsearch joins any cluster with the same name, so you do not want to run on the default or you will be syncing with other local developers on your network.
Then you want to configure your startup mechanism. For some users, that would mean configuring the elasticsearch file in /etc/init.d, but for OS X homebrew users, my startup file is ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist. First, setup extra nodes. In OS X, that means making copies of the LaunchAgent file in the same directory with unique names. It’s worth noting if you’re using homebrew, that the original LaunchAgent plist file is a symlink and you’ll want to copy the contents to a new file. I numbered my extra nodes, so the filenames were homebrew.mxcl.elasticsearch2.plist, and so on. I altered the plist file to have a couple of extra arguments, specified in the ProgramArguments node. I removed the xml nodes that specify keeping the worker alive. The results were:
I set custom node names and ports via the plist file. Note that on a Linux based machine, you would later the service to run multiple ElasticSearch nodes with those custom parameters.
Finally, since I didn’t want to run those scripts individually each time, I create a script to launch all nodes at once. Note that you’ll want to add execute permissions and put it somewhere in your path.
Finally, I recommend installing Marvel, particulary for Sense, a nice tool for running commands on ElasticSearch. You can install Marvel with this command in the ElasticSearch root directory (for me, /usr/local/Cellar/elasticsearch/1.4.0/):
Building software teams is hard. Fostering culture, improvement, learning and community in a group of individuals that have other options is a difficult thing to do. Fortunately, it’s not all that different than building many other kind of groups. Yet too often, we fail to look around at other successful groups and learn from their example. There are two challenges in particular that hold back leaders in their quest to build a highly functioning group.
First, creating success is not about the composition of the group alone. Don’t get me wrong, you should look for A-Players, but in any environment where there are other groups people will be more or less evenly distributed by the draw of money, opportunity, ego-stroking, etc. And what will differentiate your group is what it can do with it’s B and C-players. Your A-Players are already good, and mostly know how to handle themselves. Leadership might provide them some marginal returns. But compare that to the return of turning a C player into a B player with proper support. Yet how many of us see leaders trying to run off everyone but A-Players in the naive belief that they can build a team of only A-Players. Those teams don’t exist, and if they do, they don’t need leadership.
This is why some really successful leaders are viewed as simplistic, or unintelligent. Let me cite some examples. Jim Tressel had great regular season success with the OSU Football team, but struggled at times to win the big game. This was often chalked up to inferior coaching strategy. Critics said he was too conservative with play-calling, and that he harped on the basics instead of explosive plays. He was playing the numbers. He made some flawed teams better by reducing mistakes. There may be some truth that he could have opened up the playbook more and found some new strategies for the big games, but ultimately I think the key differences in the conferences that have plagued most outside of the SEC showed up those games. Urban Meyer never had that reputation at Florida, but is now turning out mostly the same results.
Another example is the financial advisor Dave Ramsey. He is trying to lead financial change across huge groups of readers and people attending classes in their community. His system is very simple and is critiqued for that lack of sophistication. But he has succeeded in helping millions of people (who span across a wide range of intelligence and financial knowledge) out of debt. I challenge the best financial Wall Street consultant to do the same. He tuned the message to the B & C players. The folks who were the most capable were on there way to success when they picked up a book and started thinking about fixing their finances and doing some basic tracking and planning. That was the nudge they needed. And the folks who wouldn’t understand a financially complicated plan got something they could digest that was better than what they are doing today.
Second, creating success is not a set of linear steps that reaches a done phase. Leaders lay out short-term plans that are focused on fixing all the problems, and then thriving in some utopian state. Like creating a group is like a construction problem, when in fact, it’s more like owning the property. You have to maintain the building. You have to weed the garden. You have to pay the bills…
Leadership is a repetitive job, where you will fix the same problem more than once, and you can’t get impatient about that. You will tread over the same ground, sometimes with the same people. If this doesn’t make sense, go ask a minister when his or her church will be “done”. Ask them when the last time they will need to do a baptism is, or when the last time is they will need to comfort a grieving family. Ask a coach when his or her team will be finished and perfect.
Who Should Lead
Now we come to some of the biggest sources of confusion. The mistaken belief that the best welder should lead the welders. The best burger flipper should manage the restaurant. The best programmer should lead the team. How many times have you seen that tried and failed.
Leadership is a service job. It is taking responsibility and solving problems. It is building community and motivating growth. Leading is about understanding people, and more importantly group interaction. Understanding the domain is usually the easy part.
While attending Codemash a few weeks ago, I ended up in a Windows Phone development precompiler (Codemash’s name for a training session). It was my plan to hit mostly mobile and analytics sessions, but I was not originally planning on attending this session. With Windows Phone still struggling for market share, I wasn’t in a rush to work with it. However, other sessions were cancelled because weather had delayed some presenters, so I ended up in this session. Microsoft’s Jeff Blankenburg was teaching the session, and I have enjoyed some of his presentations and a Silverlight fire-starter event in the past. It’s one of my rules of conferences to attend sessions based more on good speakers, rather than based solely on topic.
With regards to marketshare, Jeff made the point during the session that with a less crowded app store, you do have a bit more discoverability. Even if that doesn’t hold up, the platform shares enough similarity with Windows 8 that a port to the Windows Store will be trivial. The Windows App Store isn’t exactly setting the world on fire either, but I’d like to see my app on all of these platforms, and as Windows 8 adoption rises with new machine sales, that marketplace should see constant upticks.
Having worked with Silverlight in the past, I found it pretty easy to get going on Win Phone 8 development. There was some definite rust on my XAML skills, but it came back to me fairly quickly. One thing to keep in mind is that you want to keep things relatively simple on a mobile platform. I have worked on some WPF projects in enterprise settings with MVVM frameworks, dependency injection frameworks, and more. While I followed an MVVM pattern, I just rolled my own with a simple base class.
My project was to do a version of an app I already have in the iOS App Store, VicinityBuzz. It does location based searches of twitter. You can search around you, or by entering an address. The radius is a configurable setting. I like using the app at conferences like Codemash to catch all the chatter that may not have a hashtag. One catch is that obviously only tweets that included location will be found. If folks have that feature turned off in their twitter app, then it won’t show up.
Since I had written the app before (in phonegap for iOS), I knew the feature set and domain cold. The challenge was just getting up to speed with the latest API’s for search and geolocation, and then implementing within a new platform. One of the biggest benefits of this project was getting up to date with the latest Twitter API. I still need to update the version for iOS, as it’s currently non-functional because of api changes over the last several years. I plan on doing that very soon now that I know the latest version.
Anyway, I won’t go into the development details here too much, but I finished a version 1 of VicinityBuzz, and it is now in the windows phone store here, and it’s free. So go check it out. If you like it, I’d love to have some more reviews.
Also, if you are inspired to do any Windows Phone development yourself, you may be interested in a device to do some real testing. I recently found there are some prepaid phones new on Amazon that are dirt cheap for that purpose. Check out the Nokia Lumia 520 and Nokia Lumia 521 on Amazon.
Watch this blog for upcoming posts about working with the Twitter API, and some of the things I learned working with Windows Phone 8. And more mobile in general. I have the bug again…
There are varying ways that a client / service relationship can work, and this view can be the cause of harmony or discord in projects. It’s something I’ve understood for a while, but had trouble explaining at times. What follows is an attempt to explain my view of something that knowledge workers need to understand to have a successful and fulfilling career.
When an artist does work, we typically think of them as having complete control of the work. This is certainly true in the case where the eventual owner is not yet determined. Meaning an artist creates, and then sells directly or via a gallery to the public. Even commissioned artwork has only varying levels of control. A commissioned portrait certainly falls into the realm of work that would come with constraints. After everyone involved is dead and gone, people tend to remember the artist, not the owner.
When work is not sponsored, but contracted, it begins to fall into a completely different classification. Though you may refer to it as “artwork”, a visual painting done for advertising is really more “creative” work for hire. Certainly work that is more functional (like a bridge) comes with rigid requirements. This is the attitude of the engineer. That work is asked to meet certain goals, and ultimately subject to the approval of the buyer. While the engineer is an expert who expects to provide guidance, but does so at the behest of the client. In terms of credit, this world is muddled. Sometimes it is the architect, other times it is the owner or visionary that is remembered.
What does any of this have to do with consulting, software development, functional work, or any other type of service industry? Don’t get caught up in the work medium (paint, steel, code), or left brain / right brain aspect of the work, but just consider the metaphor in terms of relationship to the client and ownership of responsibility. In my world (consulting in the areas of software development, creative, marketing, etc) this distinction makes a big difference. All of these types of work have a right-brained aspect to them. People think of software development as a very regimented thing, but there is a lot of freedom to work in your own styles and patterns. Languages are fairly abstract these days, and they are high level enough to provide nearly infinite ways to solve complex problems. Certainly I don’t need to discuss how easy it is for a graphic designer to relate to an abstract artist.
Here’s where the risk is of confusing yourself with having the control an artist does, when you do not. The consultant works in a world where the client owns the final product. And the project would never even exist but for the idea and financial commitment of the client. Deadlines, requirements and other constraints are all part of the context. “Ownership” of the direction is a privilege that goes to the person that takes the risk. While an artist that is producing works for sale is bearing the risk that the piece will not sell, the bridge builder is not. It was sold before the project started. Sure, many contracts include shared risk clauses, but for all intents and purposes, the risk is on the client.
It is arrogant and unproductive for a paid consultant to believe they have the final say in work. They have the right to turn down work. And some clients may choose to give control over to the consultant, but that is their choice, not a right of the consultant.
When a consultant is doing work for hire and thinks they have the privilege that come with the artist mentality, it easy to develop a disdain for the client. Whereas in the engineering mindset, it is understood that one can recommend to the client, but ultimately the idea must sell itself. There is no expectation that the consultant wins because they are the subject matter expert. They are expected to be able to convey their idea and it’s merits in plain English and at a level understood by someone outside the field. In short, they must sell the idea.
Am I saying that workers should not have principals? Not at all. Take the story of Howard Roark, the principal’ed architect of Ayn Rand’s The Fountainhead. Roark suffered while turning down business as he would only work on buildings where his style of architecture was called for and he had the free hand to build modern and innovative buildings. He turned down others with work that would not satisfy him, but he did so respectfully. In this fictional work Roark has many enemies, but not the clients he turns down. A read sympathizing with Roark may agree with those clients, but it’s hard to see them as malicious.
The problem is that so many give away the right to have those principles for the sake of expediency without realizing they gave up that control. So they still expect the control. As an example, coming to work for a consulting company, you give up that right. Why? I’ve already made the point that the client has control, so you can only control work by selecting projects that meet your criteria, or working with clients who are ceding control. But as an employee of the company, you are giving up the client selection privilege in order to minimize your risk and investment. The consulting company provides you with a salary, book of work, etc, in return for your work. You do not get to control what work the company takes or not. It is your choice to leave if their client base does not suit you.
Too easily do I see people in the business blame owners and clients for work they do not like. But owners and clients anted up. They paid for the right to call the shots. If you want them to call the shots differently, it is your obligation to sell them on those ideas. Not your right to complain if the idea is not sold.
So what should someone who believes in the artist mentality do? There are people that believe so strongly in the control of their output. Folks like this tend to idolize people they view as uncompromising, like a Steve Jobs. (It’s worth noting that there are a lot of signs that he compromised more than the public perceived).
There are some simple solutions. Being an independent consultant means that you can turn down work that does not fit preferred principles. Owning / building your own consulting firm allows you the control of selecting your projects and clients. The third option is to return to the artists root of non-commissioned work. In other words, start a product company. While your clients have say in the form of sales, you are free to put whatever product you see fit on the market that can be held to only your standards. For all the knowledge workers that idolize Steve Jobs and he uncompromising reputation, it is worth remembering that he started and returned to a product company.
Media is a great analogy. Being an independent musician, filmmaker, or game developer is hard. You have to market for yourself, sell each product, secure distribution, etc. But for that price, you have control of the product. Signing with a publisher makes those tasks easier, but you have to be aware of the conditions you agreed to with the publisher. Like any contract or relationship, you should consider what those terms will mean in good times and in bad.
This is not to say that the rest of us don’t have some control and determination of our own destiny. It’s just that you have to remember you only have one lever to pull, leaving the company. You can negotiate terms based on a company or client’s desire to have you be a part of the team. But to get emotional about it, or think you have a right to control without taking the primary risk is folly.
Blue Ocean Strategy is a powerful thing. The Wii took great advantage of this, as has Apple and others. Some people try to cite this as an example of the direction for Microsoft’s upcoming XBoxOne. But that ocean isn’t blue anymore. Older parents and grandparents that bought Wii’s aren’t looking for another box. Certainly not one with 3 Operating Systems and a camera.
The Wii was an innovative and mass-market product that hit big before tablets, and during the early days of touch phones. They have taken over the “casual” gaming market.
So what does that leave? The Smart TV / Living-Room audience. That was the stated goal of the XBox project when it launched… to “invade the living room.” IGN in particular has been jumping all over this point when defending the strategy. Saying the mass market is so much bigger than the gaming market. So what of that argument? Sony and Nintendo aren’t making serious runs as the all-in-one living room box. Is that the win for Microsoft?
That argument forgets Roku players, Apple TVs, Google TV, tivo, smart TVs and all of the other sub-$100 products in this market space. Sure, they don’t do as much. And that innovation could come at a much lower cost.
How hard would it be for Apple or Roku to make an ipad app with Siri features that controls an Apple TV or Roku box? Now I have a touch screen remote and voice commands for my entertainment.
And all of those companies are good at consumer devices. Much more consistent than the Microsoft track record with Zune, Surface, etc.
Add to it things that don’t work in a new digital world. Take today’s Penny Arcade discussion on backwards compatibility for digital assets. Microsoft didn’t think that one through.
I’m not trying to pile on. I hope there are multiple good next-gen consoles. But if you’re going to play in the consumer device model, you have to remember who’s already there, and what they have conditioned consumers to expect.
This is a quick thought. But a pattern has been on my mind lately. A lot of leaders want their employees to listen to them and learn from them.
However, when that same leader is questioned by their employees, they often don’t want to have to justify decisions. Do this “because I told you to.”
Why not let your ideas stand on their own merit? If a subordinate wants to engage in discussing the value of an idea, that sounds like an opportunity to teach them (or learn from them if they are right).
Recently, my job has changed a bit. I’ll probably write about that in a post coming up, but for now it suffices to say that I’m definitely spending more time than usual thinking about entire projects and all aspects of the project. Naturally that has me thinking about process more. I have always tended to prefer Agile methodologies, and the Lean flavors in particular.
However, I didn’t feel that I really had a solid background in the lean parts. While I’ve understood the processes and motions and studied Lean manufacturing in college, some of the motivations and subtleties are tied to a deeper understanding. I feel like I needed to spend some time reading the best works on the subject. Practical experience on lean teams is great, but I wanted more.
With that in mind, I reached out to Rick Simmons (@simmons3k). I used to intern for Rick way back in my college days and we keep up and discuss software, Agile from time to time. Rick is an Agile Coach for Rally, and at one time helped implement and improve Agile processes at Constant Contact. Needless to say, he’s an expert in the area. Rick was kind of enough to give me a list of Lean books that he recommended, and I got the idea to publish the list as a post with his permission.
PhoneGap (now becoming known as the Apache Cordova project) is commonly thought of as a cross-platform mobile development solution. While true, I think it’s best to think of PhoneGap as modernzr for the mobile web.
The W3 is working on standards for web sites to be able to access specific hardware features. These common hardware features were first prominent in mobile, but are coming back to laptops via tablets and hybrids. Feature examples are GPS, Camera, Motion Sensors, Network Sensors. Also, mobile has popularized the idea of system maintained contact lists and other information that is now available (with proper permissions) to all apps.
When those standards are in place, mobile web sites will be able to fulfill most of the promise of native applications. This is why Steve Jobs didn’t want to build an app store in the beginning, and both RIM and Microsoft have been pushing html/css as 1st class ways for building apps.
In this new world with device access standards, only special circumstances like intense 3d graphics should require native code in a c-like language.
But this isn’t the world. Those standards aren’t in place. So PhoneGap fills that gap (any guesses about the name?). And it has the added benefit of supporting the app store distribution model, where mobile web apps require the user to bookmark the page to their homescreen.
Also, cross platform frameworks are usually criticized for their look and feel. In PhoneGap, you’re using html and css, a platform that we’ve been working to style for a very long time (in tech terms). I think this fact explains some of the reasons that companies like facebook and linkedin use html heavily in their frameworks.
If you’re interested in getting started in PhoneGap, I recently reviewed a book I recommend to get started. It’s a couple versions back now, but it covers things conceptually well, and it’s a nice companion to the docs.