Choices, choices

I've been thinking about the choices programmers have to make eventually. I guess this assumes that you're somewhat talented, ambitious, and thoughtful, because I'm sure a lot of coders can just sit happily at a desk and not ask themselves this. Also, it's just one way to divvy up the options, there are probably better ways.

I don't feel like writing pros and cons, so I'll just give cons and you can use your imagination.

Product company or Consulting?

Product company

You will have to work on the same thing for a long time. At first, when the product is under development, there are a lot of cool things to try and big decisions to make. But then it gets boring, you have production support to do, and some of those decisions turn out to have been wrong and can't be easily corrected. The bottom line for the business at this point depends more on sales of the existing products, so the emphasis on cool technologies is diminished.

Because you're supporting existing code, it's hard to find a chance to learn new stuff. Maybe you can deceive your manager into using the stuff you want to learn about, under the guise that the system needs to be re-written and this technology will save a lot of money or fix a lot of problems.

Consulting

You'll have to work with "non-ideal" client's employees. If the client hired a lot of really smart developers, they wouldn't need the consultants. Or maybe the really smart developers know to stay away from working there, so they need to pay the people who are willing to work anywhere as long as it's only for a year or less at a time. This makes it harder to be challenged by your peers to improve - if you're spending most of your time working with people who are not smart, you tend to go down to their level instead.

You're pretty much guaranteed to work on junky code. Again, if they had to hire consultants, they need your help to solve the challenging problems. It's probably not an interesting new business idea being implemented, more likely it's a self-created problem with their monster spaghetti codebase and vendor-provided toolset.

You have to give up some of your ideals about the quality tools/platforms you prefer. If your company gets a .NET contract, and your client insists you use Windows, then guess what? You're going to use a closed platform with second-rate versions of your favorite tools (if there is an 'N*' of your tool) and you're not going to use your Mac even though it's your more productive environment without a pile of bugs. And if the client is using the Oracle J2EE app server, then you're going to be decompiling it. This is a tough one, since a good developer stays agnostic about the tools and platform - but sometimes you have that ideal for a reason.

Many people don't want you there. Sometimes this includes your client-side supervisor, who was told to bring in consultants by their boss but thinks they could handle it fine on their own. If you produce a report that supports their pre-conceived ideas, then maybe they'll be excited about having you there, but more likely you'll be creating work for them and telling them things they don't want to hear. And of course the client's employees are mistrustful and think you'll make them look bad. And they probably value job security more than you do.

Keep coding or move to management?

Keep coding

You may get stuck as a "line programmer". This is like modern factory work. Your job is outsourcable or offshorable because there's a heavy reliance on "requirements" as an excuse to do waterfall, so the programmer doesn't have an interactive role and anyone can do it. This happens because many people think programming is typing and will never understand the art of it.

There are few places where you can move up the pay grades as a programmer. The Java Posse did a listener feedback question on this once, and said if your company doesn't promote really good developers as highly as managers, then you should move to another company. Problem is, Sun, Apple, and Google are mostly doing this work in the Bay area so that leaves a lot of people out. I'm still hoping there are more places like this.

Also, you may not get a chance to be a leader or promote your ideas. Developers don't have much decision-making power.

Management

You will probably become a bullshitter. Since you are disconnected from coding, you won't know exactly how the software behaves in a corner case, or how difficult a feature would be. So you make stuff up and make educated guesses that become random guesses over time.

You'll have to give up the dream of being the next Gavin King or Craig McClanahan or other coder hero. Instead, you will be driven to improve non-programming skills like corporate finance and negotiating.

The business environment is cutthroat, which doesn't match well with hacker ethics.

Worst, you won't do what you love, you will write proposals and performance reviews and project plans and push tickets and schedule releases instead.

Use left-side or right-side of your brain?

Right-brained - the Artist

It's hard to find a paying gig where you can invent the next great thing (or be a part of it), so you have to do one-off contracts and either manage a lot of business stuff yourself, or be a starving artist type. Until you get your job at Apple, where I think most of the programmers are the artist type.

It's not clear how to pursue an artist/hacker ethic. You can work on your brilliant idea at home, but where are the interviews for this sort of work? Perhaps there are not many openings and a lot of applicants, like aspiring to get in the NBA.

You have to gain true respect of your peers, for example, become open source committer on a well-known project. The artist rejects unearned corporate rank and title, especially when bestowed by someone who doesn't recognize quality.

Left-brained - the Analyst

You won't "save the world" - you will create predictable products that mimic others.

There may be pressure to create average-quality products, since there is no business need to exceed customer expectations, especially if it makes development more expensive.

JavaOne wrap-up

It was a long travel day yesterday to get back to DC. Here's the rest of the

Things I learned:

Glassfish 3 has a really neat-sounding architecture. The app server core is like a 20K jar and launches in half a second. It then has a maven-based dependency system that causes it to load OSGi modules that add things like JEE container, servlet container, HTTP server, PHP support, JRuby on Rails (directly from the app directory - no WAR building), Phobos, etc. So you can add a number of containers to your one running Glassfish instance and then deploy apps into it. (I think Cargo is another way to do this.) It also means it's a pretty extensible app server. There's some nice classloader magic, with a public and private classloader for each module, to make all of this possible and avoid typical J2EE classloading nightmares. Also, their build scripts create configuration files out of annotations, as opposed to introspecting them at runtime.

Rod Johnson's talk was packed. I wonder if Sun intentionally gave him a smaller room than his popularity demands because they're not sure they like him undermining their EE specs. He is more like a corporate CEO than a coder - he's very driven and seems to want market share more than hacker credentials. Interface 21 got $10M in funding at the beginning of the week so maybe this gave him big britches. Spring is adding annotation support - he did mention Guice in passing. He's also certain that OSGi is the next big thing, Interface21 is working closely with them. There's a Spring-OSGi module with the implementation. Sounds like an important thing to go learn!

Tor Norbye of the Java Posse kicked some butt! Not only did he give a talk on his Ruby contributions to NetBeans, he also was up on stage in front of the entire show at the last general session ("Toy Show") and demonstrated his speed in Netbeans and the number of keyboard shortcuts he must know. I wonder, it looked like they were all presenting on Solaris boxes, and he codes on a MacBook Pro, so wouldn't the modifier keys on the keyboard be hard to find? Maybe he practiced... I did download NetBeans 6 and opened my RoR project in it - there are some nice things. I used the local history, completion, and navigation features, and they worked pretty well. The real issue with IDE support for Ruby is the debugger - in C, there are a couple debuggers, but there's no equivalent of JPDA that lets you remote-debug a running Ruby interpreter. For JRuby, someone is working on a debugger, but the Sun guys are pushing for contribution to the Debug Commons project which will help all the IDE's have a consistent way to invoke debug operations. They're looking for help - Gabe?

Hibernate Search looks like a nice union of Hibernate meta-data with the Apache Lucene search engine. You really can't do good search with Hibernate criteria - the LIKE operator in SQL is way too slow. I ran into this with a search application at my gov't client that tried to create a massive SQL query for every user search, which was a really bad approach. Hibernate search allows you to add some annotations to your JPA class to indicate what fields should be full-text searchable, then it handles building the Lucene index, and even keeps the index up-to-date if you change the entity. There's a JMS-based solution for clustered front-end machines to share a master Lucene index. And, extra nice, the Lucene search is wrapped in a standard Hibernate Query object so your DAO code doesn't have to know it's not using SQL for some queries. Nice! I did notice Emmanuel called something a "workaround" - this made me wonder about some of the Hibernate code and whether they've allowed that mentality to get into the codebase - but this looks like a really nice way to support user search in Java.

Selenium is a great way to automate functional tests of a web app. Unlike HTTPUnit and other Request-Response level testing tools, Selenium runs a true client within the browser. This allows it to work with Ajax apps, for instance. There's a Firefox plugin that records your interactions with the app and turns it into a JUnit test case - also very handy. Selenium gets its name from the element, because it's used as a remedy for Mercury poisoning. Man, Mercury has been a terrible vendor in both of my last two clients.

Chris Richardson (author of POJOs in Action) had a nice presentation on testing, most of which I already knew. But he had a nice way of expressing why good guys don't win when it comes to unit testing and TDD - the "paradox of excellence", which is where you are rewarded for fixing the nasty bug right before production (or in production, shudder), while you are not rewarded if things quietly work and you leave every day at 5.

Feeds and REST are the community response to the enterprising of web services with XML-RPC and SOAP. There was a great graphic on one of the slides from the Rails guy, David Heinemeier Hansson:


So true.. I went to a talk on WS-Policy and pretty much wanted to shoot myself after the barrage of consultant-speak and enterprise BS. So - feeds - Rome is the Java library of choice because it handles the mess of RSS and Atom formats (RSS never had a spec!), but it's JDOM based. Watch for it to merge with the Atom-only but STAX-based feed library (A-something...)

There's even an alternative to WSDL for the lightweight REST crowd - WADL. However, there are a lot of security issues lingering in this whole mash-up environment, basically, you should only mashup to sites you really trust like Google. Assuming the stuff about Chinese censorship hasn't put you on a crusade...

Domain-specific Languages are really great for solving some problems, and I started to think of them as an alternate solution when you consider a rule engine. Rules are good for implementing a lot of business requirements that would involve big chunks of repetitive, procedural Java code. But, a lot of times, those requirements aren't globally true facts about the business, they're very contextual and too simple, and there are too many of them, and a rule engine isn't a good fit. Perhaps you want to code the solution, just not in Java. You want a language that expresses the business concepts - aha! Now, the talk from another Interface21 guy went into implementation details for writing your DSL in JRuby, where it is easy, then easily integrating this with your enterpris-y Java app. Managers don't have to freak out since it's just another JAR dependency, but it's a much better way to create and use a DSL than any solution in Java (I don't actually know of any). I think this is a great addition to the programmer tool-belt. Also, this guy actually demanded a raise from Rod during the presentation, I guess the $10M has gotten to their heads... not sure I want to work there anymore.

My overall observations from the conference:

• JRuby is going to make a major impact on Java development, both in the web and enterprise (I don't know about micro... but I think there was a presentation on that). Java makes some things hard to do, and getting new major releases into common adoption takes a couple years (still not on Java 5 at a lot of big companies!). So waiting 4 years for closures isn't going to compete with a single jar you can drop into your app to start including Ruby classes. I'm getting The Ruby Way, a recommendation from the JRuby DSL presenter.


• It's important to work with smart people. Since they share a lot of common motivations, if there are a lot of smart people at a company, all your other criteria are probably met as well. I'm starting to think there are two distinct camps of brilliance, though - the artist type and the businessman type. Consultants seem to be the latter, and innovative ideas come from the former. Maybe you can distinguish them based on who has a Mac.


• All of the good software companies are in the Bay area. I hope this isn't entirely true.


• I want to go to JavaOne next year.

Lessons from JavaOne

Here are some things I have learned so far:

• If you're a Java geek, then being next to Rod Johnson, meeting Howard Lewis Ship, seeing the Java Posse interview the Cenqua team, are probably like most normal people seeing Julianne Moore at the grocery store. (I also ran into an architect from my last client and a manager from the client before that!)
  


• With static analysis (ie. FindBugs), you find errors that are not caught by the compiler. This is a lot like a document where you make a typo that is a correctly spelled word, so it doesn't get a red underline. It's nice to have a grammar checker for those situations. That's one way to think of static analysis. It was funny how Bill Pugh ripped on Josh Bloch's JVM code for having obvious typos. Apparently there isn't much underway to do this sort of analysis on scripting languages, but he thinks it's even more important there since the syntax is more relaxed.
  


• WS-Policy is completely un-fun and sounds like consultant-y BS


• Romain Guy is a true programmer rockstar because he has to beg for breakfast donations at the beginning of his talk due to being up till 4:30 am, trashed.


• Non-RESTful web services are tunneling through HTTP. I hadn't thought of it that way before. RESTful services, on the other hand, take advantage of all the infrastructure set up around HTTP since things like caches work at the protocol level.


• The proper way to distribute business rules to your running Drools (or other engine) instances is by publishing them to a content management system. JBoss Rules will soon be integrated with Apache Jackrabbit for this purpose.


• Closures are needed in Java, anonymous inner classes don't cut it. Many APIs and language constructs would be less awkward and verbose if we had them. Neal Gafter is very opinionated.


• A workflow engine can be useful for business intelligence if the history of workflows is data mined.

At JavaOne!

I'm in SF at JavaOne this week. Have to head off to an Open Source SOA talk, but here's a quick post...

Maybe it's because I selected RESTful web services, Grails/Trails/Sails, and Closures, but I keep seeing people trying to catch up with RoR. They admit Java makes the easy problems hard (Neal Grafter's words) and want to develop more APIs and language changes that would give Java some of the same capabilities. I'd really like to hear more about JRuby and see if we can leave Java alone and admit that adding APIs isn't the same as using a different language.

Went to see the Java Posse live. They had giant foam hats, and talked about the keynotes which I missed. (I flew in yesterday morning, which meant being up 22 hours, and I couldn't stay up to get beers after the BOF with them :( ) It was great to see some of the industry rockstars yesterday. Hope today is as interesting.

XML: configuration or code?

Seems like there's a backlash against XML going on. Coding in XML is a code smell, and everyone is rethinking the overuse of XML in Spring configuration, for instance. Google Guice is the annotation-based IoC container, and now I see Spring 2.1 is doing the same.

I think there are two ways to look at the XML problem. One way is from the configuration management perspective - if you have a long process to build and deploy your software, you know how nice it can be to have a change be to configuration and not code. And maybe you'd deploy your software to many customers who configure it differently, if you're writing Tomcat for instance. There are surely some situations where the configuration of your app isn't simply key/value pairs, and XML is a nice way to structure your config files.

But of course this is abused, and Spring contexts are a good example. You don't expect that the customer is going to change the files, they're very long, and it's likely that a change in the app is going to require compiled changes as well as the XML changes. And they're really part of the code, wiring up objects in a specific way, such that changes to the context are likely to result in the app not working.

Annotations are the answer to this, it seems, since they put the configuration right there in the code. It's a more concise way to express the intent, the compiler catches your typos, and should make for better tool support. But, I have one problem with this - thinking about the Dozer bean mapping framework as an example. Here, we have two JavaBeans and the XML configuration describes how to map properties from one to the other. It seems anachronistic to give either Javabean knowledge of the other.

Maybe the better way to think about it is that XML should hold data. Facts, unchanging in time, with no logic, no tie to the classes themselves and certainly no relation to a certain runtime context. Sometimes the data is obvious, like the input to a DBUnit test. It could be some wire-protocol data, like a web service request used in SoapUI. Maybe sometimes it's configuration, like the servlet.xml file. Maybe it's even the dozer mapping file, describing how two JavaBeans' properties are related. Anything else, and yeah, XML is not a good place for it.