McDonald’s Theory

Couldn't have said it better myself. All teams needs someone to break open productive meetings. I apply the same tactic: https://medium.com/what-i-learned-building/9216e1c9da7d

Don't be a sheep!

Excellent talk about innovation. It's in Dutch, sorry about that :) http://www.jefstaes.be/keynote-jef-staes/

CQRS

Nice slideshow that's easy to understand:
http://www.slideshare.net/jeppec/cqrs-why-what-how

The surprising truth about what motivates us

How to create an enterprise with staff that evolves from following orders to autonomy and motivation:
http://www.youtube.com/watch?v=u6XAPnuFjJc&feature=youtu.be

Finger pointing

In the Agile dev world, there is a rule saying 'no finger pointing'. This means not rubbing people's nose in something when they made a mistake. This to assure people don't become afraid of touching some code or making mistakes. I think it's also a way of being professional. Now here's the thing. Most people want to know when they made a mistake. So you need to tell them. It's about how you tell them of course. Polite and calm is the manner it should be. This works for most people, but the rule has a couple of disadvantages: What do you do when someone really doesn't give a f*ck? You already told him 10 times to run the tests before check in and he still doesn't do it. What do you do then? I say finger point away! Finger point till he cries :)

DevOps

Nice post explaining what DevOps is:
http://blog.csanchez.org/2012/03/13/infrastructure-as-code/

Existing principles interestingly explained

Here's an interesting video from Gojko Adzic stating that software quality is everyone's problem. Nothing revolutionairy, but I especially like the way he talks about removing team bottlenecks and the importance of job titles.

http://gojko.net/2012/02/17/sleeping-with-the-enemy-video/

Work to the rule

I want to share a piece of text that fascinates me in the exellent book 'Pragmatic Thinking and Learning - Refactor Your Wetware'. It fascinates me because it is a general piece of text, which imo can be applied to a lot of big Agile teams on projects that take some years to finish.

[Start quote from the book]
In industries or situations where one is not allowed a fullblown strike, a work slowdown is often used as a means of demonstration. Often this is called work to rule or malicious obedience, and the idea is that the employees do exactly what their job description calls for —no more, no less— and follow the rule book to the letter. The result is massive delays and confusion—and an effective labor demonstration. No one with expertise in the real world follows the rules to the letter; doing so is demonstrably inefficient. According to Benner (in From Novice to Expert: Excellence and Power in Clinical Nursing Practice [Ben01]), “Practices can never be completely objectified or formalized because they must ever be worked out anew in particular relationships and in real time.”
[End quote from the book]

Often, when the velocity of big teams is not good, the teams itself propose a number of rules to improve their velocity. Sometimes these rules deal with check-in procedures, refactoring unfamiliar domains should be reviewed by persons familiar with that domain, adhering to certain conventions and so on... That's all needed and all fine. But if the project takes a few years to finish, it can happen that the number of rules can grow a lot, or that the people who invented certain rules, and more importantly, that knew WHY the rule was needed, are gone.

When your project has an entire wiki full of rules, and not all of these rules are clear for the whole team, there is a danger of overregulation. This can lead to 'malicious obedience'.

Testers being the fat kid up front

In the Certified ScrumMaster course, Jeff Sutherland said something that keeps running through my mind. The Agile principle is based on quick feedback cycles and quality of code (xP). A story is not finished until after a 'proxy customer' check. A 'mini acceptance' of the analist representing the customer, if you will. Later on, maybe after the sprint, the story is tested again in some form of integration environment.

To get that feedback in different phases early, he told us to treat the the proxy customers as fat boyscouts. If someone during a boyscout hiking trip can't keep up, it slows down the whole group. You should therefore walk beside them to motivate them. If they still can't follow, you take their backpack. If it's still going too fast, put them in front of the whole group. Make them determine the paste.

I think he's right. We should aid and guide our testers as much as we can. If during the daily scrum stand-up, you hear that someone is blocked from testing, this should deserve your immediate attention.

Proposal based

A month ago I followed the ScrumMaster (CSM) course of Jeff Sutherland and I'm an official ScrumMaster now. It was interesting to hear the guy that invented SCRUM - a former Phantom fighter jet pilot in Vietnam - expressing his vision on the IT industry. A couple of things stood out in the WAY he said them. He likes to formulate things differently. In the Passionate Programmer they talk about the transistion of being technology-centric to being solution-centric. You could tell that Jeff had an infinite respect for the developer (more than for management) but made an interesting statement: 'Developers are always whining and complaining about what's not right. In stead they should offer management clear choices'. For instance in stead of complaining 'we don't get the time to write tests' we as a community should say: 'Either we test and it will save (x-amount of) money in the long run or we don't and it will us cost money'. He's right. I've seen a lot of peers whining :)

Another thing a lot of developers don't do which I always try to, is the way we report issues. Before I communicate a problem to management or the customer, I try to have a list of possible solutions in my mind to propose to them. Not everyone does this, even if the answer to the report of a problem mostly is 'ok, what can we do about it?'.

Dual core processing

I'm reading one of the best books EVER right now (and I do mean EVER), called 'Pragmatic thinking and learning' by Andy Hunt. The reason I like it so much is because it links my interest in IT and in psychology/biology (I did one year of Psychology at the KUL but failed).

Andy talks about the brain as a dual core processor, where CPU1 is the L-mode (lineair or also called left mode, but this is less correct) of the brain and CPU2 is the R-mode (rich mode or right mode). There also is a contention at the message bus level that allows communication between the two, which implies that only 1 mode can dominate your brain at a certain moment in time. In this context he explains why in eXtreme Programming 'pairing' is so important. If you 'drive' you brain uses the L-mode to deal with the words, correct syntax. High level patterns and relations are made by the R-mode however. This is why sometimes your navigator says 'and if you refactor this bit here out, we can re-use it in that other place'. He also explains that this is the reason that when you're not pairing and you face a difficult problem it's important to step away from your desk, giving your R-mode a chance to kick in. I used to experience this in the past when I still smoked. Often the solution to a problem came during my sigarette break!

Fa-sci-na-ting stuff. To be continued no doubt...I'm off buying an Andy poster.

XML escaping

Did you ever want to define a constant in your code containing an XML string? For instance in a unit test to test if you can transform a piece of XML via XSLT into some desired format? You can do this the hard or the easy way: Either you take the chunk of XML, copy it in your IDE or some text editor and replace all double quotes with \" to escape them. Then you can start adding carriage returns in the String to have a somewhat readable piece of text. In case your XML contains all streets in Belgium, you will feel very, very useful doing this.

Or, you can have your IDE doing the work for you. In eclipse simply go to 'Window>Preference>Java>Editor>Typing' and check the “Escape text when pasting into a string literal“ option. As you can see it also adds the newlines and carriage returns:




I remember thinking 'Damn, I wish I knew about this before' when I first discovered this option. That might just be the criterium for any useful IDE feature I guess.

No worries, we got SSL

Last Tuesday evening a colleague and me attended an OWASP meeting of the Belgium chapter. The topics were:
-The Thinking Person's Guide to the Cloud. HOWTO: Keep your head in the clouds and your feet on the ground (by Gunnar Peterson, Arctec Group).
-Threat modeling (by John Steven, Cigital).

Especially the presentation of Gunnar Peterson was magnificent. He was funny, to the point and also explained the basic stuff, so you didn't have to be an expert to understand everything. He especially explained that a general mistake often made by non-security people, is to think that SSL secures your entire application. SSL is great, but there is a lot more to think about than just this one aspect.

There are a lot of methodologies to make your security vulnerabilities visible. This is called 'Threat modeling'. A simple and common used methodology is 'STRIDE', which stands for Spoofing, Tampering, Repudiation,Information Disclosure, Denial of Service and Elevation of privilege. These are the things you want to counter. The following table gives possible security solutions targeted at each specific problem.

Spoofing - Authentication
Tampering - Digital Signature
Repudiation - Audit Logging
Information Disclosure - Encryption
Denial of Service - Availability
Elevation of privilege - Authorization,Input validation

For each of the 6 items we have 3 categories where we could implement a security solution. You can implement it on the data level, method level and channel level. An example of the different levels:

• Data: XML
• Method: SOAP, URI
• Channel: HTTP

This means that you have 18 possibilities for implementing security. SSL however only is a solution for information disclosure, because you encryt the data on the channel level. Mostly that's not enough and you should for example encrypt parts of the xml message too.
SSL alone is not enough to secure your application!

No need to check that now, check it later...

Let's say you have an Oracle DB containing a table A with a foreign key to a table B and we want to add a foreign key constraint to ensure referential integrity. This might look something like this:

ADD CONSTRAINT "A_FK_B_CONSTRAINT_NAME" FOREIGN KEY ("COL_IN_A_REFFING_B") REFERENCES "TABLE_B" ("PK_ID")

Now if your A object has a oneToMany collection of B objects and you cascade delete these B objects whenever you remove A, you might run into a problem in the way the constraint is defined now. Hibernate deletes the A and B rows in 2 steps or statements, within the same transaction. If Hibernate first updates the PK_ID to NULL, COL_IN_A_REFFING_B will reference a non-existing row, which will cause an exception from the FK constraint checker.

This is why you need to add 'DEFERRABLE INITIALLY DEFERRED' to the constraint:

ADD CONSTRAINT "A_FK_B_CONSTRAINT_NAME" FOREIGN KEY ("COL_IN_A_REFFING_B") REFERENCES "TABLE_B" ("PK_ID") DEFERRABLE INITIALLY DEFERRED

This allows the constraint checking to be deferred or postponed till commit time. A constraint that is not deferrable (or DEFERRABLE INITIALLY IMMEDIATE) will be checked immediately after the execution of every statement. Currently only FK constraints accept this clause, other constraints are not deferrable. A good 'chicken or egg' post can be found here.

Show me your license!

In the follow up of open-sourcing our project, I spent some time today figuring out which license would be used best. This was completely new to me so I started comparing the different kind of licenses available. Apparently there are two major kind of licenses. On the one hand you have the 'GPL based' licenses, which enfore the code that use the open-source GPL code, to also be free and to also continue under the GPL 'flag'. They even have a clause stating that you cannot add additional restrictions on the 'redistribution of either the original work or a derivative work'. The goal is to not only garantuee the 'freedom' of the open-source software, but to also encourage software that uses open-source software to do the same.

On the other hand you have a bunch of 'proprietary compatible' licenses, like the MIT/X, the BSD and the Apache license which pretty much allow everything (use, modify, redistribute, ...) without enforcing extra constraints. Chances are high that we will go with an Apache license. Somewhere in between those major kinds, there also is a more pragmatic version of the GPL, called the 'Lesser GPL' which literally is a copy of the GPL exempting the fact that the propietary software should also use GPL.

I also read a lot about copyright, copyleft, dual licensing, trademark protection, and credit enforcement, but I'm not gonna bore you with that...

eID

At work a colleague and me just finished the 1.0 version of a new eID security module. It's based on the open source framework of Fedict that uses an Applet to allow you to sign in using your digital id. This framework was brought to life due to a lot of problems (stability, configuration, ...) with the current eID Fedict middleware solution.

For now it's still implemented as a stand-alone war, but it will be integrated in our larger, existing security module very soon. This larger module copes with authentication, authorization and identity management and is really mature for some time now. It came to life several years ago to replace a product from Sun called Access Manager. Onces the integration with eID is finished, the entire module will be open sourced. Exiting stuff!

PKCS#12

PKCS#12 is a PKCS (Public Key Cryptography Standards) developed by IBM and published by RSA Security. It is designed as the Personal Information Exchange Syntax Standard. This means that it serves as a standard to exchange sensitive key information between one kind of keystore and another kind. In Java you can instantiate a Keystore in the standard way Keystore.getInstance("JKS") or with a PKCS12 SPI implementation Keystore.getInstance("PKCS12"). Both implementations store private keys and certificates in a single file. OpenSSL however, stores them in separate files. Via the 'openssl pkcs12' command you can merge them into a PKCS#12 file, bridging the cap towards a Keystore. For instance:

openssl pkcs12 -export -inkey newSignedRequest.pem -certfile myCertFile.pem -name "TEST CERTIFICATE" -out myPkcs12Cert.p12

Certificates comming from a CA are almost always delivered in pkcs12 format.

Can you fetch me this thingie? Just for me please...

Kodo is a JDO implementation that it's not widely adopted and a bit more complex than Hibernate. Neverteless I found at least feature that it already had long before Hibernate did, called 'fetch groups'. With a fetch group you can for instance eager fetch a collection that is defaulty mapped as lazy for a certain scenario.

Let's say a bunch of addresses are defaulty mapped 'lazy' to a customer. We can then define a fetch group (eg 'addressFetchGroup') to indicate that this collection should be eagerly fetched when this group id is used in the query.

Here's what this would look like:

The package.jdo (mapping) file:

    <class name="Customer">
      <extension vendor-name="kodo" key="data-cache" value="false" />
      <extension vendor-name="kodo" key="jdbc-sequence-factory" value="native" />
      <extension vendor-name="kodo" key="jdbc-sequence-name" value="MYSEQUENCE" />
      <field name="name" persistence-modifier="persistent" />

      <field name="addresses" default-fetch-group="false">
        <collection element-type="address" />
        <extension vendor-name="kodo" key="fetch-group" value="addressFetchGroup"/>
      </field>

 ...

The code:

    public Customer lookUpCustomerWithAddresses(String customerId) {

        Collection fetchGroups = new ArrayList();
        fetchGroups.add("addressFetchGroup");
        fetchGroups.add("otherFetchGroup");
        ...
        return retrieveWithFetchGroups(fetchGroups, customerId);
    }

    @SuppressWarnings("unchecked")
    public <S> Collection<S> retrieveWithFetchGroups(Collection<String> fetchGroups, Object... parameters) {
    KodoQuery query = (KodoQuery) persistenceManager.newJDOQuery();
        try {
            query.getFetchConfiguration().addFetchGroups(fetchGroups);
            return (Collection) query.executeWithArray(parameters);
        } catch (Exception ex) {
  ...
        }
    }

When used correctly (and more importantly, measured correctly with a profiler), this can greatly improve performance of a specific use case, without affecting other use cases. As of hibernate 3.5 this feature is also included and known as 'fetch profiles'.

Thou shall respect the carriage return

As usual it was fairly late notice when we as the development team heard our certificates were only still valid a day and a half before expiring. Luckily the procedure to prolong them was already executed. We had sent them to our CA, Fedict, which had extented their validity and send them all back in one big zip.

Beside the complete lack of naming convention of the cert file names, a few things suprised me in the renewal process. We had at least 16 or more certificates to import for different environments and customers and had to chain each of them with an intermediate and root level certificate, also provided by the CA. Although this is a trivial task, it made me wonder why they couldn't have done this for us, you know, being a customer of their services. Second of all it was in pem format, which was nice, but it looked something like this:

Ulv6GtdFbjzLeqlkelqwewlq822OrEPdH+zxKUkKGX/eN
...
...
9801asds3BCfu52dm7JHzPAOqWKaEwIgymlk=

Importing them in our jks was impossible unless we added the begin and end phrase to make it look like this:

-----BEGIN CERTIFICATE-----
Ulv6GtdFbjzLeqlkelqwewlq822OrEPdH+zxKUkKGX/eN
...
...
9801asds3BCfu52dm7JHzPAOqWKaEwIgymlk=
----END CERTIFICATE-----

Although this is also a known practice and was quickly done, it was again something they could have done for us, you know, being a customer of their services. Lastly while doing all this, I noticed another subtlety, this time about the jks. It only allows the import if there is a carriage return after the last phrase. So you had to explicitly say '----END CERTIFICATE-----CR' before the import succeeded. I wonder if this also is a security feature.

The eagerness of JPA

JPA came to life as a result of JSR 220. This request originated to provide a uniform API and way of working across different ORM solutions such as Hibernate, Toplink and JDO. One member of the expert group leading JSR 220 was righteously Gavin King, the main inventor of Hibernate. Since Hibernate is the most widely used implementation, I thougth they would follow most of its important design decisions. For instance in hibernate the default fetching strategy is 'lazy'. Although this can sometimes lead to difficulties with casts and object equality (if you received a proxy in stead of the real object), it has as advantage that you only fetch dependencies in memory when you need them. I was suprised to see that in JPA the default fetch of a @manyToOne for instance is 'eager'?!

This lead to the fact that someone in my team unwillingly dragged half the DB in memory :) If anybody knows why they decided to change this, please let me know. Thx