Life is mostly just yak shaving

Yesterday I mentioned the concept of “yak shaving” to a friend. When he expressed bafflement, I suggested he look it up. When I tried to find the original story myself, I had a little trouble.   It’s here:

So there I am, shaving a yak




Architectural Bestiaries

There’s an old expression I used to hear, saying that most programmers only write one program in their careers; they just keep rewriting it over and over.  Developers would learn one useful pattern and then reuse it.  A lot.  Possibly improving it, but definitely milking it.  The gang-of-four patterns book addressed the situation substantially, since it set the expectation that professionals should learn many different techniques.

Software architecture deserves the same treatment.  Architectures are bigger than single programs.  They can develop a lot of momentum, so it’s common that most of us will only see a couple of distinct application architectures.  Conservatism plays a role too.  It’s risky (maybe foolish) to bet your company’s revenue on a novel architecture.

A remedy may be for us to catalog the different high-level designs.  Even if we don’t get to use these new architectures, we benefit by surveying the field and analyzing the possibilities.  The best survey I’ve seen recently is a report by Srinath Perera titled A List of Known Scalable Architecture Templates.   The article is just a single page, but it covers a lot of ground.  Scalability isn’t the only quality that an architecture must address, but it’s often the most constraining.

Netbook with Snow Leopard

For almost a year, I’ve been trying different Linux distributions on my Dell Mini 10v.  Lately, I decided to try something in the BSD family.


Dell Mini 10v

I’d tried a similar project a couple years ago.  At that time, I’d set up a Dell Inspiron 1525, a cheap laptop I’d chosen because the hardware neatly mirrored a MacBook of that time.  My intention had been to load Linux, but a friend had clued me in on the Hackintosh concept.  My experience was that it was amazingly easy to get the OS first installed, but then incredibly difficult to get it running perfectly.   The webcam always worked, but the wi-fi was hit-or-miss. First the sound wouldn’t work, and then just the microphone didn’t work.    Eventually everything worked properly, but when I installed software to support my Blackberry, the wireless network cut out.  Ultimately, I’d spent several full weekends getting it running, plus quite a few week-nights.

At this point, my Inspiron is working fine.  But, I’m careful not to upgrade anything.  It’s an attention-getter at work.  When people ask, I tell them that it’s a fine hobbyist project.  However, if you just want to have a Mac … go out and buy one from Apple.

So here I am trying the same stunt with my tiny netbook.  Fortunately, there’s an excellent tutorial over on Gizmodo.  Folks are hacking all kinds of netbooks, but it sounds like the Mini 10v is ideally suited for this adventure.

The result was … easy.  I purchased a new copy of Snow Leopard and then walked through the tutorial.  It took a couple hours but most of that time was spent waiting:  waiting for the download of the netbook bootmaker, waiting while reformatting the hard drive, waiting for Snow Leopard to install. And the result .. worked.    After the reboot, I had a working copy of 10.6.3.

I immediately opened “Software Update” to bring in all the latest code.   This was all too good to be true, and sure enough, the result wouldn’t boot.

I reinstalled and then updated everything except the 10.6.6 update.  Now I’ve got everything except the App Store.  The microphone, sound, and camera works.  The SD reader slot works.  Wi-fi works just fine.

How to manage your technical education

I recently noticed an article on the InfoWorld site offering up 7 programming languages on the rise. I’ve been seeing similar articles for many years. They address a common insecurity in our field, which is how to keep your skill set relevant and valuable. A little hindsight shows most of these reports to be ridiculous. The best are just rehashed stats from the Tiobe Index or Google trends. The worst are cooked numbers advocating some company’s products or some enthusiest’s opinions.

I’ve probably made a career decision or two based on such poor journalism. To combat this misinformation, I’m going to put down in writing the most useful bit of career advice I’ve ever received. It is this: choosing which technologies to study is very similar to choosing financial investments.

Suppose you’ve come into a big chunk of money. You need to grow it for use in your later years. I know of three principals to guide you in this task.

First, don’t invest it stupidly. It’s often difficult to find the smartest, best investments, but it’s quite easy to find terrible and stupid ones. The first principal is to avoid the losers. This ought to be obvious, but an amazing number of people invest based on a random tip. Or, they invest based on some personal relationship. The money goes into their roomate’s movie project or their brother-in-law’s startup. Dumb. You’d be better off taking it all to the casino. Or, give it to charity; at least a charitable gift pays off in karma.

Pick investments that have a proven track record, a good product, and a definite market.

The second principal is to diversify your investments to manage the risks. You could put all your money in one stock that may pay off big. But, the higher the potential reward, the greater the chance you have of losing everything. You could dump it all in boring, low-yield stocks that are unlikely to go wrong, but that’s also missing opportunities. The trick is to figure out how much risk you can afford, and then buy a combination of high and low risk investments that average out to your desired risk level. Spreading the money around to multiple similar investments lowers your risk, but don’t pick so many stocks that you can’t watch them all intelligently.

The third principal is to hedge your bets. This is really just a corollary of the first two principals, but hedging is a good principal to understand. Suppose one stock goes up whenever the price of oil rises. You can lower your risk by also buying a different stock that goes up when oil prices fall. This probably sounds like a win-win situation, but it’s hard to accomplish. Money spent on the hedge takes away capital that you could spend on the primary investment. You need to understand the market and pick combinations that make sense.

Now, having laid out the metaphor, I hope the similarity to education planning should be clear. You have a fixed amount of training time available to you, both on-the-job and on-your-own. You should study things you enjoy, but you should also be smart about the choices.

First, don’t spend your time developing skills that have no future. It’s hard to know what will really be hot in the next five years, but it’s usually easy to spot technologies that are going away. Don’t focus your energies on a technology just because it’s what you know. Maybe you had a good job last year with some obscure bit of tech. Ask yourself seriously if there are other markets for these skills. We should all focus on areas that we consider fun, but ask yourself occasionally how the time spent will pay off in the long run.

Second, diversify your skill set. Build up a portfolio of skills that minimize your risk of going broke, and optimize the chances of having a fulfilling and well-compensated career. Try to have a few highly valuable and noteworthy skills. Balance these with some basic, bread-and-butter skills that are always in demand.

I used to know guys who were experts in the compilers of supercomputers. Those are pretty valuable skills … up until the supercomputer market deflated. After that crash, those guys had to either move across the country to find work, or settle back into less exotic development. All of them actually had a good collection of skills, so they’re doing fine. We can all develop a good skills portfolio.

Build up a few unusual but valuable skills (keeping the first principal in mind). Learn some challenging software frameworks. Study a new (but viable) programming language. Hack your linux box. Delve into networking or security or embedded software. At the same time, invest in the common skills necessary in your field. Understand the web and the database and the essential programming languages and the software development life cycle.

The third principal, hedging, also applies here. As with finances, it’s a tricky thing to do. Learn similar technologies from different sources, in case one falls out of favor. If you know Tomcat, spend some time learning JBoss and GlassFish. If you’re a web developer, you should also understand Flash and Flex, but also REST and general XML manipulation.

In technology, hedging your skills doesn’t just lower your risks of unemployment. Frequently, employers are looking for specific combinations. Maybe they’re looking for an Oracle admin, but they’ve also got a MySQL and an SQL-Server system. Or, the company that delivers everything over the web might be considering deployment to iPhone; having both skills could give you the edge. Avoid combinations that rarely occur in real companies.

I can’t say this advice will make choices easy, but it has provided me a framework on which to make decisions. Invest in yourself. Optimize your value for the long run.

NFJS – day 2

It’s the second day of the No Fluff, Just Stuff symposium, and my focus is on a couple of software architecture sessions put on by Ted Neward.  This is my primary area of interest and I’ve been particularly looking forward to these.

Architecture in this context is the application of software engineering to large systems, particularly  to the creation, care, and feeding of distributed software systems.  Too often we hear the word “architecture” used casually as a synonym for “design” or else pretentiously to mean a philosophy for software development. It is more useful to think in terms of engineering.  Large systems have architectural requirements which can be addressed by architectural patterns.

All software starts with requirements.  If you aren’t addressing requirements, you’re just goofing around.  Large systems have architectural requirements, often grouped as the “*ilities”: scalability, availability, reliability, manageability, extensibility.  Also performance, security, etc.  All professional software should create business value, and architects should be directly involved in the overall business models supported by their systems.

Ted is a good speaker and very entertaining.   Me, I’d have been content with less stand-up comedy and more architecture.  But that’s just me, and everyone else seemed pretty happy.  Ted plainly knows the subject and covered it well, puncturing some of the pretension that usually surrounds the field.  He went into the “*ilities”, the types of architecture, and the concepts high level design.  His particular contribution is a catalog of architectural concepts, decomposing how we can think about large systems.  The discussion focused on pragmatism and choosing solutions that are appropriate rather than philosophically pure.

The second session was the most useful.  In the “Architectural Kata”, we broke up into small teams and each group was given a software project.   We had 45 minutes to throw together an architectural design and then presented our approach for discussion.

There was only time for five presentations.  Three teams proposed using Google App Engine.  One proposed using Grails.   My group hadn’t picked a presentation framework or app server, but we had broken out basic components and distributed data centers.  We’d worked out requirements for data replication, security, availability, and usability, and had discussed basic use cases.  Definitely not a whole system, but good work for 45 minutes.

Overall, a good and informative conference.

NFJS – day 1

I’m attending the No Fluff Just Stuff Symposium this year.  This is a (relatively) low cost traveling conference with some excellent speakers.  Since it’s held here in the Twin Cities, there’s no travel costs and my employer sent a half dozen of us.

Friday morning I saw two excellent talks by Neal Ford, one covering REST and another on automated software testing.  Based on the first talk, REST seems to be growing up finally.   Up to now, my every exposure to REST digressed into low levels of abstraction.  Every REST proponent has seemed fixated on implemented an entire REST framework from scratch and crafting every problem to fit the HTTP protocol.  Neal’s talk, on the other hand, seemed perfectly happy with the JAX-RS stack and even talked about WADL definitions of REST services.  This is healthy.  A very good coverage of REST and the latest architectural thinking.

The testing lecture was even better.    As developers, we usually view unit tests as the only checks applied before we throw our code over the wall to QA.  This talk certainly covered unit tests well, but also covered many more tools and techniques available for us to improve the quality of our code.   It’s perfectly OK for developers to work on functional testing, integration testing, and user acceptance testing.  New techniques for mock testing and behavior driven testing were introduced.

In the afternoon I saw three sessions given by Venkat Subramaniam about Scala and about JVM concurrency.

Learning Scala has been on my to-do list for a while.  Venkat gave a good presentation, but the talk focused on the tricky, arcane things that can be done.   The presentation made the language look doubly intimidating, but did give me a better understanding of the Scala landscape.

Venkat has written books on Groovy and Scala and seemed pretty knowledgeable about Clojure, so he really knows his concurrency.   Most of his points were illustrated by writing code on the fly and executing them immediately.

The final presentation was about unit testing concurrent code.  I didn’t find his conclusion very satisfying, but he definitely showed that it is possible to do test driven development against multithreaded code.

That’s Life

I’ve been playing with the <canvas/> element in HTML5 lately.  There’s a lot of little tutorials and cheat sheets out on the web, but nothing yet in the way of a comprehensive manual or cookbook.  Mostly I’d like to create little games and apps that run on the web, but also work well on smart phones and tablets.

My latest toy runs Conway’s game of life.  It runs fine on FireFox / Safari / Opera, and also on my iPod touch.

One puzzle that came up in this project was to translate the location of a mouse click into a position within the canvas.  This should be a pretty common problem, and you’ld think that all mouse events would have some canvasX / canvasY properties attached to tell the position relative to the canvas.  No such luck.  The closest thing are clientX and clientY which tell the absolute mouse location within the whole browser window.

You could calculate canvasX based on clientX if you know the absolute position of the canvas within the window, but again it’s not there.  The closest thing available is a positionLeft property telling the canvas’s relative position within it’s parent element.

Here’s my solution:

function calculateElementPosition(canvas) {
 if (canvas.positionLeft || canvas.positionTop) {
 canvas.positionLeft = canvas.offsetLeft;
 canvas.positionTop = canvas.offsetTop;
 if (canvas.offsetParent) {
     canvas.positionLeft += canvas.offsetParent.positionLeft;
     canvas.positionTop += canvas.offsetParent.positionTop;

function calculatePosition(event, element) {
 if (event.canvasX || event.canvasY) {
     // Already calculated
 } else if (event.pageX || event.pageY) {
     event.canvasX = event.pageX - element.positionLeft;
     event.canvasY = event.pageY - element.positionTop;
 } else if (event.clientX || event.clientY) {
     event.canvasX = event.clientX + document.body.scrollLeft
         + document.documentElement.scrollLeft
         - element.positionLeft;
     event.canvasY = event.clientY + document.body.scrollTop
         + document.documentElement.scrollTop
         - element.positionTop;

Calling calculatePosition within your event listener will attach the canvasX and canvasY properties to the mouse event.

JavaScript: the Good Parts

I’ve been trying to build up my JavaScript skills.   That statement requires a little explanation, since I’ve been avoiding JavaScript for years.  I’m not one of those guys who looks down his nose at the language.  It’s just that in the past my every experience with JS always ran into miserable cross-browser incompatibilities.  Also, there were no decent developer’s tools, so tracking down that problem with version X of browser Y was pretty painful.  My attitude was that I could solve all user-interface problems in the middle tier, where I had complete control and I could apply a proper Java debugger.

Lately, I’m long overdue for picking up a new skill.   I know that JavaScript has matured and that there are decent tools available.   Also, although I’m not particularly good at the web or GUI development, I think the user-facing stuff is more fun.  It’s just gratifying to see stuff up on the screen that other folks might use.

My first re-introduction to the language is through Douglas Crockford’s book JavaScript: the Good Parts.  This book is mercifully thin, but the information is rich enough to warrant multiple readings.   Crockford states that there were design mistakes in the language, but that a subset of JavaScript represents a “good” language.

The book covers the basic syntax and then dives straight into object manipulation, including JavaScript’s prototype based inheritance.   It covers functions as first class data types, including some of the convoluted scoping rules.  It goes over arrays and regular expressions, and includes a discussion of JSON.

As an introduction (or re-introduction) to JavaScript, this book gets my recommendation.   It has good coverage of most of the language.    It gets you into the domain and starts you on some good coding habits.  However, I have a disagreement with the basic premise of the book.  Professional developers spend more time reading other people’s code than writing their own.   For that reason, it’s not enough to learn the good stuff, you also have to understand all the other stuff too.

Guidance for Architects

Most of the enterprise architecture endeavors I’ve witnessed have turned into bitter failures.    Even with competent architects, the projects usually die from poor human interaction.  Here’s the best advise I’ve seen so far on the subject:

How to avoid being the asshole architect

Me, I try to remain positive on this subject, but just try googling “arrogant software architect” sometime.

Netbook: Ubuntu 10.04 release candidate

For the past week, I’ve been using the release candidate for 10.04.  Today I read that the update manager will automatically give me 10.04 final when it’s released.

Doesn’t this take the wind out of the big countdown?  If the final version is just incrementally different from the beta version, then Ubuntu is released continuously instead of in big milestone releases twice a year.

Honestly though, the version I’m running uses normal Gnome windows and menus, not the netbook-specific application launcher.  So at least I’ve got that to look forward to.  Also, this release candidate came without Java installed and without the Broadcom driver for wireless internet.   From these senses, it’s less of a complete consumer product than the 09.10 netbook remix.