Where Did I Put My Encryption Keys?

As a software security consultant, I was asked to solve some interesting problems with cryptography in Java. I warn others that security is esoteric and an afterthought much of the time. So taking security into consideration early is better. But even if you are a late-comer, the solutions do not cost that much, especially when compared to losing valuable data to an unfriendly attacker. Most of them can be done with Java packages that come with the JDK (the .NET framework has them too).

For example, a super easy solution to a common problem is password hashing. Passwords need to be hidden from attackers, programmers, admins or DBA's. This can be easily done with the MessageDigest class in Java. The great thing about hashing is that there is no key to store, you simply run your password through the algorithm, like SHA, and out pops a byte array that you can store in the database or a text file. Then the next time you prompt for the password, you hash that password with the same MessageDigest function, compared it to the one you've got stored, and if they match, you are logged in. Attackers or admins cannot possibly guess what that hash is by looking at it. So there you have a 3-line maintenance-free security solution that will pass any security audit - it's the same technique most Unix systems use to store their passwords.

Software cryptography provides quite a few different solutions, and when you analyze them, you try to find the weak spot. The weak spot is usually the keys to a cipher like 3DES or AES. When you use a cipher, you have a key, and storing this key safely can be a big problem since you don't want them to be stolen or
compromised. Most security books point this out and then fall short of suggesting some solutions: e.g. this one from Microsoft Press tells you to put the key somewhere far away from the data, but the example code violates that very suggestion by putting the key and the data right next to each other in the Windows registry (with a disclaimer that says don't do this). So what hope does a novice software developer have to avoid making bad choices when storing the keys? Other books on secure software are no better, I think their publishers forbid them to give good examples for fear of being sued.

Bad practices happen, but I will suggest some better ones since I have no publisher to forbid it. Let's define some security requirements for cryptographic keys that every developer should follow:
* The keys should be hidden
* The keys should not be known to the programmer
* The keys should be changeable after installation in case the keys are lost to an attacker
That's all we need, really. Now lets try to meet these requirements.

Solution 1: Hard-coding the keys in the code. Many developers will create a static String in the code that has encryption keys in it. This gets compiled and shipped with the software. Since the customer normally gets binaries and not source code, this hides the keys pretty well to satisfy the requirement above. However, the next two requirements are impossible with this solution. Now the programmer knows the keys - imagine being kidnapped by spies and tortured for the keys, or more likely, imagine being bribed for the keys. Then imagine what happens if someone bribes a programmer and then posts the keys on a public web site. Even easier than bribery, an attacker might download the software from your own web site, decompile it (amazingly simple for Java or C#), and then post your keys to a web site. You must panic, update the code, and every one of your customers must upgrade ASAP. This is a nightmare. You might say, "Yes, but most people would not know how to decompile the code, right?" Remember, you are not protecting your keys from MOST people, you are protecting them from a motivated attacker, and motivated people DO know how to decompile your code. (Yes, of course I know how to do it, but I won't bother putting the details here.)

Solution 2: Putting the keys in a config file. Storing your keys in a config file seems unpleasant because the keys are right there in plain text for all to read. Clearly this violates the hidden requirement above, doesn't it? Its true - its not hidden very well. However, the config file DOES satisfy the other two requirements: the keys are hidden from the programmer and they are changeable later. So now when the keys are lost to an attacker who posts them on a web site, you tell the customer to change the config file to enter new keys. The other customers need not worry, and the customer who lost their keys will be secure again in seconds.

Well maybe we can encrypt the config file! Yes, then we solve the hidden requirement! Brilliant! Except for one thing - where do you store the key for THAT encryption? Hard coded in the app? In another config file? You can see where this is going... The truth about security solutions is that they can all be beaten, some solutions are simply better than others and cause the attacker to expend more effort. In the case of the keys in the config file, you can simply use Unix or Windows file permissions to limit the readability of the file to the people who need to know, thus limiting the attackers who can get the keys. Then if the keys end up on a hacker web site, you change the keys, then haul off the admins to jail and torture them - but at least the programmers are safe and your customers are safe.

Some companies actually have a security policy that says "no storing passwords in plain text on the disk". I would say that this rule is impossible to follow. Because when I encrypt the password, where would I store the keys for that? Its a rule made by people who haven't thought about it. Sometimes this rule leads to hard-coding the keys in the code, and that's even worse. But if you have to follow this silly rule, use ROT13 since it needs no keys itself.

You can further protect the keys by putting them somewhere else separate from the data - maybe store them in a JNDI server instead of a config file (note that JNDI would persist to a text file, but at least it would be on a separate machine), or in a separate database from the rest of the data. (BTW: where did you store your database password? I suggest JNDI data sources, follow Solution 2 above). Anything to limit the exposure of the keys so you can hide them from most, but not all.

You can see that hard-coding the keys is appealing because it hides the keys, and what's more important than that? But hard coding the keys totally ignores the other two concerns, and it doesn't really do a great job of hiding the keys. So storing the keys in a config file (or other external place), even in plain text, is always preferred.

-Jay Meyer

jmeyer at harpoontech dot com

Data Mining Research Project

I've decided to start a new side project. Data Mining has been an intriguing subject that I've been interested in for some time now. I wrote an automated eBay parser a while back that would alert me to new auctions via IM every hour or so. That project was fun, but violated so many of eBay's terms of service that I no longer use it.

I would like to come up with some kind of automated system for data retrieval and am in the initial phases of design/prototyping now. The design needs to be flexible enough to adapt to changing conditions encountered during the data gathering session, and also will need to be able to be controlled dynamically as my criteria change over time. Perhaps I'll integrate JBoss Rules (formerly known as Drools) into this, as a means of applying rules to the data gathering process. I'll also integrate this with our XMPP service, so I can get updates on my phone as well as controll the process from anywhere.

I'm not quite sure what I want the end product to do exactly, but I know what I don't want it to be: a Google replacement. I'm not looking to index the web, more like a "smart search" that is continuous and updates me in real time (or close to it anyway) as it discovers new data.

Anyway, I hope to have something useful in the end. If you have any comments or suggestions, drop me an email.

-Jason

jwambach@harpoontech.com

JSF Presentation: Seam leads the JSF pack

On June 8, 2006, I gave the presentation about JSF to the St. Louis Java User Group. I got a good reiview from one of the attendees, and in general, I think everyone enjoyed the presentation. I agreed to do the presentation without knowing anything about the topic, but I read a book, looked at the buzz, and worked through two tutorial code examples myself to get comfortable: JSF.ppt presentation file

My opinion, Seam from Gavin King and JBoss has some really good ideas. And the crowd at the JUG seemed to agree. We had a few JSF users in the group that were using MyFaces - they were really impressed with it vs. Struts 1.x. The biggest lessons learned are that the surrounding technologies for JSF are still taking shape, but they are a feasible alternative to Struts. When you combine Facelets, Tomahawk, and Myfaces you get a powerful MVC replacement for Struts. And if you have Java5 and EJB3, Seam will take JSF right down those last miles of the n-tier architecture with suprisingly little effort.
The next project I start will be using Seam. The JSF and EJB3 technologies are the skills that I need for the future of Java Server development.

7-Layer Dip

Java architects are insecure. They make the big decisions and try to make as many software layers as possible to maximize complexity. Somehow when entrusted with designing reusable software that is to be used by tens of Java programmers, software architects feel the need to make lots and lots of decisions, in order to confuse the poor souls who really just want to solve the problem and go home for a quiet evening of beer and Tivo. That's job security, it makes the Java architect indispensible to the project, calming their insecurities. It makes for large system that have layers and layer of code.

I call this 7-layer-dip architecture. Its is an n-tier idea gone terribly wrong, putting layer after layer of architectural bloat on the code until its nearly impossible to figure out where to make changes to add something like a new field on my object & web page. Its like a 7-layer dip, lots layers and time consuming process to create a tasty appetizer.

In a 7-layer dip - its all about the beans. If you take away the beans, you have a 6-layer dip, right? No, you do not. Instead you have some lame-o salad. Without the beans, you have something that's no longer Mexican, you have to dump out your margarita! The Fiesta is OVER! The other 6 layers are important, for without them its just beans and who wants that, its no longer fun without the other layers. Do I need 7? No. I think 4 layers is still pretty tasty, but we're enamored with the number 7, so we make up a few more.

Analgoy-stretching time: In Java apps its also about the beans- the Java beans, the POJOs, the model classes - the simplest form of code, the useful stuff. Without the beans you have no application, just empty framework stuff, no fun, no margarita. But then some Java architects like to wrap these beans in 7 layers of tomatoes and lettuce and cheese and sour cream and tortillas because they think its necessary to keep the party going. This is where the party goes wrong - Java beans do not need 6 more layers of wrappers and EJBs and services and business delegates and data accessors and views and actions. These layers do not add to the fun of writing software. Instead, they serve to alienate developers from the important parts - the beans, the app, ...the user. Developers then try to make themselves look smart by jumping on the buzzword bandwagon. I should never hear a developer utter a phrase like this in front of a user, "well, I can just make the action return a new implementation from the business delegate factory". Perhaps Java developers boast of bloat-buzzwords so they can raise their pay, but if the users are smart, they will just be annoyed and they'll hire that PHP script kiddie from the basement next door.

In the late 90s we all let Sun talk us into n-tier machines with Entity Beans, CORBA, and SNA spread out in 5 times zones, so we could write "Enterprise applications!" (and buy some E-class Sun hardware). We took in some good dot-com paychecks and the accountants and managers rode our e-coat-tails to high stock prices. Good for us! In the meantime, 8th graders were selling T-shirts with PHP apps running on old hardware in their basement. And they did it all that by reading "Learn PHP in 7 minutes". Sure, that little PHP app is not ready to handle 8000 hits per minute, but its built to be the right size for selling T-shirts out of a basement, and when he needs to add the baby-tee sizes and the 3XL sizes to the web site, he can do it in a day by recoding some PHP and adding some columns to his MySQL table.

Don't panic, I'm not advocating the destruction of design patterns, I still advocate the use of Factory and MVC and lots of other good designs. But they are simply being overdone. For example, I like to use the Factory pattern if I'm pretty sure I have two different impls to hand out in different situations. A SecurityService is a good example for just about any app, because I have changing needs from development to production. I can write one SecurityService for LDAP and one for reading a password file to authenticate users. I use a SecurityFactory to return the correct service, so I can change it out for test vs. production by changing a simple config file. Great. There, I used a factory pattern in a good way to swap out two different pieces of code seamlessly. But I don't need a Factory for every bean in my system. When am I really going to need two different impls for a Book object? I mean if the web site sells Books, there's probably only one way to sell books, do I need a Factory to pass out two different classes of Book? Almost never. But this kind of knee-jerk over-architecture decision is everywhere even in closed-source corp IT systems. If, by chance, I invent another impl for a book, I can slam a BookFactory in later while I'm putting in the new BookImpl - easy stuff (even easier, use Spring). But making a BookFactory in an application is usually just obnoxious. The only reason to do that is to pad your own resume with bloatware knowledge. I know..I know.. you can contrive the need for a PoolingBookFactory and a SimpleBookFactory and a WebServiceEnabledBookFactory. But most of the time you just need a Book - new Book(), its just that simple. Very few cases arise where I need the kind of flexibility a Factory pattern provides, but develoeprs and architects don't even remember the reason for the complex case and they abuse that pattern for a much simpler case like Books just in case they'll need it later.

This kind of over-architecture slows down developers, especially new ones who must learn this annoying bloat-itecture before they can add a new simple property to the Book class like Book.setOprahPickedIt(boolean b).

Why does this happen? What makes this bloat so prevalent? Who promotes these guys to Architect positions and lets them run big software projects? It must all come from the old waterfall addage that big code now is cheap, and change later is costly, RUP uses this principle to justify their brand of $10K tools. That theory then pushes the derivative theory: "why overarchitect later what you could over-architect today?" So IT managers believe a Java architect when he hears "well we need all this code to scale and to be prepared for change", and the checkbook writes a big check for "architecture and design" with zero expectation of a demo or milestone release coming anytime soon. And admittedly, I've jumped onto a project and I drank the kool-aid and gave the architect the benefit of the doubt "certainly there must be a really good reason for all this code...". And I changed piles of code to add a single property to a class. But when I turn to ask a peer whose been grueling with the system for 10 months, that poor coder is hard-pressed to answer the question "why?". This doesn't just happen on old EJB projects, it happens on EJB-less Web projects and thick-client apps too. Reminds me of this old parable: why we do things.
The best things to cure my pain are the Agile warriors and the POJO bigots. Both of these groups have shown me that simple is better and that change is pretty cheap. If you build your code in a simple way first, then strap on some good features one by one carefully, your code doesn't need to blow up into a confusing mess and you'll be agile when the customers change their minds or ask the system to do more. I like the Spring and EJB3 movements for reminding us that simple is good. And I like the Agile movement for the idea that change is inevitable, trust your customers and don't hire a lawyer to write that contract. Now we just need to convince all the decision makers that this is the best way.

Little piece of advice to those architects of bloat. Embrace change, keep thing simple, and read some software articles written in this decade. I recommend theServerSide and the Agile Manifesto .

Jay Meyer

jmeyer@harpoontech.com

My Experiences with JSF

I am currently converting a large Struts application to JSF. My first instinct was to simply replace the Struts tags with the equivalent JSF tags and JSTL.

Turns out things aren't so simple. JSF & JSP don't really play all that well together as detailed in the following article: Improving JSF by Dumping JSP. So this means if you mix normal HTML and JSP tags you may run into unexpected problems. To avoid this you must fully understand the rendering lifecyle of JSPs and JSF. Talk about a maintainability nightmare. If you want to avoid this, you must buy into converting your old JSPs tag for tag to the equivalent JSF tags. That doesn't seem easier to me. But wait, the tools are here to save us. They didn't save us from the nightmare of EJB 2.1, they won't save us now.

The framework Facelets was created to solve this issue by by completely replacing the JSP layer we all know and hate. It drops the need to fully convert your app over to JSF tags, you simply use HTML plus JSF tags where needed. Great, I found it frustrating to use the JSF markup to output the HTML I wanted to display in the first place. No more! Also, no more crappy code generation on the backend. Todays' date isn't 1998, lets move on.

Facelets make JSF tolerable but still I think it should be more simple. Why does JSF introduce its own dependency injection when EJB3 already has one? Because JSF came out before EJB3! Enter Seam the latest from Gavin King(author of hibernate) which vastly simplifies the JSF configuration using annotations and tightly integrates JSF with EJB3 and JBPM
The Web Beans JSR aims to standardize Seam.

My prediction is that Web Beans technology will be the BFG that allows them to punch holes in IBMs' armour and finally makes web application implementation simpler.

Rajesh Patel
rpatel@harpoontech.com
Harpoon Technologies

Webservices and Hardware

I have posted the slides and the source code for the xfire integration demo that I presented on 4/5/06. For those of you who were not in attendance, this demonstration's purpose was to introduce XFire and how it can be used to expose a web service. Rather than the typical "Hello World" example, I decided to integrate the service with a microcontroller, whose purpose is to display the temperature of my basement.

All of the source code and slides can be located here:
Client (12.8M)
Service (4.1M)
PPT Slides (308K)

I have not included all of the rxtx libraries (the .dlls) as their installation is platform specific. The hardware schematic is included in the ppt slideshow, if you need a detailed parts list, just ask me.

-Jason Wambach

EJB3 Presentation

I recently gave a presentation on EJB 3.0 at my
local javasig. They were an astute crowd
and pretty much asked me everything that
I chosen not to cover.

However they did not ask me why they should use
EJB3 over spring + hibernate. The main benefit
of EJB3 is that it will be standard based. But
functionally spring + hibernate has it beat.

Here is my presentation for anyone who is interested:
EJB3 Presentation

Raj Patel

rpatel@harpoontech.com