Wednesday, November 16, 2011

Ec2 Getting started

My notes on getting an Ec2 instance up and running.

I created the (free) Amazon Linux AMI.

This is a base linux distro, so you will need to install any extra software you need (e.g. in my case tomcat, jdk)

Install tomcat6
> sudo yum install tomcat6


chmod a+x ./jdk-6u30-ea-bin-b10-linux-amd64-25_oct_2011-rpm.bin
sudo ./jdk-6u30-ea-bin-b10-linux-amd64-25_oct_2011-rpm.bin

Start Tomcat

Friday, October 21, 2011

Grails dynamic finders not working in Unit Tests

I ran into a problem with my unit test in Grails.

One of the dynamic finders was not working. What was strange was that it worked fine when I ran the app normally. It only failed in the unitTests.

groovy.lang.MissingMethodException: No signature of method: static com.domain.myDomain.findByCode() is applicable for argument types: (java.lang.String) values: [myCode]
at groovy.lang.MetaClassImpl.invokeStaticMissingMethod(
at groovy.lang.MetaClassImpl.invokeStaticMethod(
at groovy.lang.ExpandoMetaClass.invokeStaticMethod(


When unit testing, grails mocks out all backend access. In order to do this it must dynamically update the domain classes (e.g. save method), so that they dont try to talk to the DB.

Therefore when using any unit testing on domain object you must first mock it out. This is easily done in the unitTest class in the setUp() method

mockDomain myDomain

It can save you a lot of hassle

Tuesday, October 18, 2011

Stack trace cleanup

Groovy has a handy class for sanitize stack traces. (i.e. it removes any java, groovy classes from the classpath, so that just so called 'Business' classes remain.

Of course if you are using thirdparty libraries then this might not cleanse the stack trace enough. Therefore it might be useful to append extra ignorable packages into the ignore list.


// Add any extra package names here
"groovy.," +
"org.codehaus.groovy.," +
"java.," +
"javax.," +
"sun.," +

public static Throwable sanitize(Throwable t) {
// Note that this getBoolean access may well be synced...
if (!Boolean.getBoolean("groovy.full.stacktrace")) {
StackTraceElement[] trace = t.getStackTrace();
List newTrace = new ArrayList();
for (StackTraceElement stackTraceElement : trace) {
if (isApplicationClass(stackTraceElement.getClassName())) {
// We don't want to lose anything, so log it
STACK_LOG.log(Level.WARNING, "Sanitizing stacktrace:", t);

StackTraceElement[] clean = new StackTraceElement[newTrace.size()];
return t;

Thursday, October 13, 2011

Grails local plugin repository

If you are creating your own grails plugins, then you need to
1/ Store them on a central server
2/ Inform grails to check this server to find your plugins before going online to download them

There are 2 ways to do this (see

1/ Grails Compatible Repositories
This is easily achieved by creating (or updating if it already exists) the $HOME/.grails/settings.groovy file

e.g. to add myPluginRepository

The discovery, and distribution locations are svn repositories
grails.plugin.repos.resolveOrder=['myPluginRepository', 'default', 'core']
To publish
grails release-plugin -repository = myPluginRepository

2/ Maven Compatible Repositories (Recommended)
The maven-install command will install the Grails project or plugin artifact into your local Maven cache:
grails maven-install

In the case of plugins, the plugin zip file will be installed, whilst for application the application WAR file will be installed.

The maven-deploy command will deploy a Grails project or plugin into a remote Maven repository:
grails maven-deploy

It is assumed that you have specified the necessary configuration within a pom.xml or that you specify the id of the remote repository to deploy to:
grails maven-deploy --repository=myPluginRepository

The repository argument specifies the 'id' for the repository. You need to configure the details of the repository specified by this 'id' within your grails-app/conf/BuildConfig.groovy file or in your USER_HOMER/.grails/settings.groovy file:
grails.project.dependency.distribution = {
localRepository = "/path/to/my/local"
remoteRepository(id:"myPluginRepository", url:"http://myserver/path/to/repo")

The syntax for configuring remote repositories matches the syntax from the remoteRepository element in the Ant Maven tasks. For example the following XML:
<remoteRepository id="myPluginRepository" url="scp://localhost/www/repository">
<authentication username="..." privateKey="${user.home}/.ssh/id_dsa"/>

Can be expressed as:
remoteRepository(id:"myPluginRepository", url:"scp://localhost/www/repository") {
authentication username:"...", privateKey:"${userHome}/.ssh/id_dsa"

By default the plugin will try to detect the protocol to use from the URL of the repository (ie "http" from "http://.." etc.), however if you need to explicitly specify a different protocol you can do:
grails maven-deploy --repository=myPluginRepository --protocol=webdav

The available protocols are:

* http
* scp
* scpexe
* ftp
* webdav

Tuesday, October 11, 2011

Arrays Maps List syntax for groovy/ java/ javascript

Slight syntax differences

Note groovy maps are different to json maps, and proposed Java map literals. Grrovy maps use similar syntax to arrays

1/ Groovy syntax

def map= ['id':'FX-11', 'name':'Radish', 'no':1234, 99:'Y']
assert map == ['name':'Radish', 'id':'FX-11', 99:'Y', 'no':1234] //order of keys irrelevant
def map4= [:]
assert map == java.util.LinkedHashMap

def map2= [id:'FX-11', two:"two"] as Hashtable
assert map2 == java.util.Hashtable
assert map2.two == "two"
assert map2['two'] == "two"

Note: key may be string or not.

a= [ 11, 12, 13, 14 ] as Object[]

List (by default create ArrayList)
def list = [5, 6, 7, 8]
assert list.get(2) == 7
assert list[2] == 7
assert list instanceof java.util.List
assert list instanceof java.util.ArrayList

def emptyList = []
assert emptyList.size() == 0
assert emptyList instanceof java.util.ArrayList

// To force to type other than ArrayList use
numbers3 = new LinkedList(['One', 'Two', 'Three', 'Four', 'Five'])
numbers4 = ['One', 'Two', 'Three', 'Four', 'Five'] as Stack // Groovy 1.6+

Note Groovy also has Range type. This is inherits from java.util.List, but is a pure groovy class (e.g. groovy.lang.IntRange)

2/ Java (8) syntax (Note these produce immutable items)

Map map = {"key" : 1};
int value = map["key"];

Set set = {"item"};

String[] array = {"item1", "item2",}

List list = ["Abba", "Beatles", "Corrs"];
List emptyList = [];

3/ Javascript/ Json Syntax

var emptyList = [];
var homogenousList = [1, 2, 3];
var heterogenousList = ["one", 2, 3.0];
Var list = new Array();
list[0] = "item";
list[1] = "item1";

Use Array

var emptyMap = {};
var homogenousMap = {"one": 1, "two": 2, "three": 3};
var heterogenousMap = {"one": 1,
"two": "two",
"three": 3.0};

map = {'key':'value'}
map.key == 'value'
map['key'] == 'value'

Monday, October 10, 2011

Grails (hibernate) "Could not find a setter for property in class"

When working on a Domain class (in grails) I wanted to add a convenience method to return some information.

e.g. I had a property endDate, and I wanted to have a method isActive to return true if the endDate does not exist or is in the future.

boolean isActive(){
if(endDate==null) return true;
return endDate

However when I try to run the app I get

Solution is

def isActive(){
if(endDate==null) return true;
return endDate

If defined as boolean (or int string etc), then Hibernate interprets it as a getter, and will look for corresponding setter, and throw an exception when not finding it.
Defined as with a def, means this won't happen.
Alternative is to set it in the transients static.

static transients = [ "active" ]

Saturday, October 08, 2011

No Sql

This video is useful

At a high level there are 4 cetegories of noSql Db's

Key Value based e.g. Amazon Riak, Voldemort
Column based e.g. HBase, Bigtable. each row can have its own schema (semi structured)
Document Databases (e.g. couchDb, mongoDb).. Key Value based
Graph based:

Sunday, September 25, 2011

Configuring 2-way SSL with Oracle Service Bus

Configuring 2-way SSL with Oracle Service Bus

Must set up Identity and trust.

Identity represents the server itself. This corresponds to the Servers private key. (Its public key is embedded in its certificate)

Trust is for when weblogic is communicating with clients over 2-way SSL. It must store a list of certificates it trusts. The client must then supply one of these as part of the handshaking protocol.

Server Side configuration (ie. For terminating 2 way SSL requests)

Enable X509 as an Identity Asserter type

Error if not present:

X.509 token identity assertion is not enabled in the security realm

Goto Security/ myrealm/ Providers/ Authentication/ DefaultIdentityAsserter

Add X509 to DefaultIdentityAsserter. (Note also that wsse.PasswordDigest is also present here for WS-Security UsernameToken. (Digest Replay Detection Enabled can be also be set here). Also to enable this you also must set “Enable Password Digests” in the DefaultAuthenticator.)

Restart Server

Next Step


The X.509 username-mapper sub-plugin of the default identity asserter in not configured

HTTPS inbound endpoint inboundEndpoint specifies CLIENT-CERT authentication, therefore the username mapper properties of the default identity asserter must be configured (this is required to support 2-way SSL)


Configure the username-mapper fields of the Default Identity Assertion provider in the security realm pages of the WebLogic Server console


Configuring 2-way SSL with SoapUI (client)

A lot of these settings are configured in the Global Preferences table, so cannot be saved per project.

Firstly load the KeyStore (where the client certificate is stored) in the SSL tab of the Prferences.

Note: Requires Client Authentication box is part of the Mock definitions, and is not required for a client configuration.

You may also need to configure a proxy (I did) if your client is on the external web, and you use a proxy to access that.

Keytool commands



keytool -genkey -keystore keystorename -storepass keystorepassword

Generates a new private key entry and self-signed digital certificate in a keystore. If the keystore does not exist, it is created.

keytool -import -alias aliasforprivatekey
-file privatekeyfilename.pem
-keypass privatekeypassword
-keystore keystorename -storepass keystorepassword

Updates the self-signed digital certificate with one signed by a trusted CA.

keytool -import -alias aliasfortrustedca -trustcacerts -file trustedcafilename.pem -keystore keystorename -storepass keystorepassword

Loads a trusted CA certificate into a keystore. If the keystore does not exist, it is created.

keytool -certreq -alias alias

Generates a Certificate Signing Request (CSR), using the PKCS#10 format, and a self-signed certificate with a private key.

Stores the CSR in the specifiedcertreq_file, and the certificate/private key pair as a key entry in the specified keystore under the specified alias.

keytool -list -keystore keystorename

Displays what is in the keystore.

keytool -delete -keystore keystorename -storepass keystorepassword -alias privatekeyalias

Deletes the entry identified by the specified alias from the keystore.

keytool -help

Provides online help for keytool.

Initial Setup steps

In order to enable outbound two way SSL (i.e. MW making a two way SSL call out to a client) we need to create a Service Key Provider

Create a Service Key Provider

To use a service key provider, you must configure a PKI credential mapping provider.

To use the PKI Credential Mapping provider, you need to:

  1. Configure keystores with appropriate keys and distribute the keystores on all machines in a WebLogic Server cluster. Setting up keystores is not a WebLogic Server function. For information about setting up keystores, see the help for the Java keytool utility at See alsoConfiguring Identity and Trust, for information about keystores and keys in WebLogic Server.
  2. Configure a PKI Credential Mapping provider. A PKI Credential Mapping provider is not already configured in the default security realm (myrealm). See PKI Credential Mapper Attributes and Configure Credential Mapping providers in the Administration Console online help.
  3. Create credential mappings. See Create PKI Credential Mappings in the Administration Console online help.

Create Java Keystore in Domain

Create a Keystore. Import certificate (user identification certificate) into keystore.


(see step 6)

Configure each WebLogic Server instance to have access to its own copy of each keystore. All entries referred to by the PKI credential mapper must exist in all keystores (same entry with the same alias).

May have to convert certificate to format understood by keytool

keytool -importkeystore -srckeystore pki\GTMSWebUser.pfx -srcstoretype pkcs12 -destkeystore ./pkiKeystore

So I was able to directly import the pfx file containing the private key and certificate.

From OSB this was then accessible in the ServiceKey Provider. Not when selecting the Key, you must supply the private key password to access it (not the keystore password)

Create a PKI credential mapping Provider

Got security/ myrealm/ providers/ CredentialMapping

Select New.

Select PKICredentialMapper

Supply a name (e.g. PkiCredentialMapper)

Create Credential Mapping

Got security/ myrealm/ providers/ CredentialMapping

Select New.

Protocol= https


PrincipleName = GTMS

CredentialType = Cert

Create ServiceBus ServiceKeyProvider

Configure a PKI Credential Mapping provider

Goto Providers./ CredentialMapping. Select New/ PKICredentialMapper

Note a restart is required after this.

Create Credential Mappings

Configure a PKI Credential Mapping provider

Not sure if this is needed?

Restart server

PKI credential mapper provider

Create a new PKI Credential mapper in the console

Click on new and create KeyStore

There is no PKI credential mapper provider configured in your security realm. Service key provider management will be disabled. Configure a PKI credential mapper provider if you need service provider support. This is typically the case if you have Oracle Service Bus proxy services with web service security enabled or outbound 2-way SSL connections.

Friday, August 19, 2011

Ordered Map vs Sorted Map

This is obvious once you know it (isnt’ everything).

A little bit of a rant at declining standards first.. (its ok, I’m not lamenting the youth of today, or society; its my own declining standards I’m ranting about)..

I feel that a lot of times I rely on google to give me a quick solution to a problem. There’s nothing inherently wrong with this. I imagine 95% of developers do it and certainly it leads to quicker solutions. One negative consequence of this is that I don’t get any depth of knowledge, just a quick link to a possible solution which I often can paste in to fulfil my needs. Little effort, quick result, but little return and learnt. In some ways this blog is an attempt to expand some of these topics just a little bit more, in order to get a better understanding of them, but it too has a bit of the copy and paste solution about it.

Of course this only works if you 1/ Ask the correct question, and 2/ form your question correctly.... So today (possibly as a result of years of quick solution hunting, and no learning) I forgot some of the basics, and didn’t ask the correct question.

I had a Map instance where I wanted to control the sequencing of the keys. I knew this class existed in the java.util Collections classes (or failing that the commons collections), so I was frantically searching the web for a Sorted Map.

I found java.util.SortedMap. Perfect ...
I read the javadoc description and thought.. this isn’t what I want... It sorts on natural Order, or based on applying a custome comparator. After much head scratching about how I would create a custom comparator to order my Map in the order which it was created, I finally had my Eureka moment... I didn’t want a Sorted Map at all

Of course what I really wanted was java.util.LinikedHashMap. This stores the Keys within an internal LinkedList structure that maintains the order of the elements as they are added. Quite different to a sorted List which dynamically sorts the keys based on a algorithm (either natural or custom).

So todays lesson is simple. Learn the basic terminology before consulting the almighty google. And LinkedHashMap is quite different to SortedMap

Sunday, August 14, 2011

Gson inheritance

This article helped fix a problem I had with inheritance, and Gson (JSON api)

Basically, key is to ensure that you specify the field values as private fields, and assign their values in the constructor (not by creating a new private field with an updated value)

Monday, August 01, 2011

jconsole for high level profiling

I’ve mentioned it obliquely before but its time I highlighted the free profiling tool jconsole.

Always good to infrequently check your application especially if you are getting OutOfMemory errors.

It also doubles up as an MBean browser

Saturday, July 23, 2011

Wget (or curl) can be a magic command at the right times.

Heres some options I’ve used to retrieve a directory from svn

I wanted to download the everything below

wget -r --no-parent -nH --cut-dirs=10 --no-check-certificate --http-user=user --http-password=password https://remotehost/svn/Base/Streams/1.3/Projects/ProjectName/trunk/osb/Interface/Resources

-r = recursive
--no-parent stops it looping back to parent directory should any references point there
-nH. Ignore the host when saving any resources (removes remotehost directory)
--cut-dirs=10. Removes 10 directories when saving resources. This removes /svn/Base/Streams/1.3/Projects/ProjectName/trunk/osb/Interface/Resources, and will only save any subsequent directories and filenames.

--no-check-certificate, ignores any https cert problems. Handy for local sites with untrusted certs.

-m = mirror. Handy replacement for, -N -r -l inf --no-remove-listing

-Rindex.html : do not download index.html pages

--http-user, --http-password specify some HTTP basic auth if required

--proxy-user=, --proxy-password= specify some proxy auth if required. The actual proxy itself

To specify the proxy. Yo must set environment variables
e.g. export http_proxy=

variables are
If set, the http_proxy and https_proxy variables should contain the urls of the proxies for http and https connections respectively.

This variable should contain the url of the proxy for ftp connections. It is quite common that http_proxy and ftp_proxy are set to the same url.


This variable should contain a comma-separated list of domain extensions proxy should not be used for. For instance, if the value of no_proxy is ‘’, proxy will not be used to retrieve documents from mysite.

Monday, July 18, 2011

JSTL Gotcha

Why doesn’t this work?

<%@ taglib uri="" prefix="stripes" %>
<%@ taglib uri="" prefix="security" %>
<%@ taglib uri="" prefix="stripes-extension" %>
<%@ taglib uri="" prefix="c" %>
<%@ taglib uri="" prefix="fmt" %>
<%@ taglib uri="" prefix="fn" %>

<c:set var="a" value="A"></c:set>
<c:set var="b" value="B"></c:set>
<c:set var="c" value="C"></c:set>

<c:if test="${a=='A' }">
a= A,
<c:if test="${b=='B' } ">
<c:if test="${c=='C' }">

The answer is the space in the test B jstl tag.

Another few hours of my life wasted...
Also another time waster... Beware of <div id=”myDiv” class=”myDiv” />

Firefox didn’t like the autoclose tag. I needed to explicitly add a </div>

SoapUi Scripting

Heres a simple project with some scripting in it.

Key points are.

1/ Any output from script gets redirected to the script log. This is visible as a tab at the bottom of the main pane.

Note. This tab only appears after a script generates a log message.

2/ Examples are online at. This includes writing request to file, and dynamically generating responses

Sample project

Thursday, June 30, 2011

Weblogic Security Realm WLST import and export

This is just a reminder for myself, the code is not mine but can be found at multiple places on the web so I have no idea who the initial owner is and who to give credits.

I took it from

export configuration:

java weblogic.WLST

connect('weblogic','weblogic', 't3://somedomain:7001')



cmo.exportData('DefaultAtn','/tmp/export.ldif', Properties())

import configuration:

java weblogic.WLST

connect('weblogic','weblogic', 't3://someotherdomain:7001')



cmo.importData('DefaultAtn','/tmp/export.ldif', Properties())

Saturday, April 30, 2011

Weblogic transactions

Weblogic transactions

Handling Heuristic Completions
Heuristic error
After the abandon transaction timer expires, no further attempt is made to resolve the transaction with any resources that are unavailable or unable to acknowledge the transaction outcome. If the transaction is in a prepared state before being abandoned, the transaction manager will roll back the transaction to release any locks held on behalf of the abandoned transaction and will write an heuristic error to the server log.
A heuristic completion (or heuristic decision) occurs when a resource makes a unilateral decision during the completion stage of a distributed transaction to commit or rollback updates. This can leave distributed data in an indeterminate state. Network failures or resource timeouts are possible causes for heuristic completion. In the event of an heuristic completion, one of the following heuristic outcome exceptions may be thrown:

HeuristicRollback—one resource participating in a transaction decided to autonomously rollback its work, even though it agreed to prepare itself and wait for a commit decision. If the Transaction Manager decided to commit the transaction, the resource's heuristic rollback decision was incorrect, and might lead to an inconsistent outcome since other branches of the transaction were committed.

HeuristicCommit—one resource participating in a transaction decided to autonomously commit its work, even though it agreed to prepare itself and wait for a commit decision. If the Transaction Manager decided to rollback the transaction, the resource's heuristic commit decision was incorrect, and might lead to an inconsistent outcome since other branches of the transaction were rolled back.

HeuristicMixed—the Transaction Manager is aware that a transaction resulted in a mixed outcome, where some participating resources committed and some rolled back. The underlying cause was most likely heuristic rollback or heuristic commit decisions made by one or more of the participating resources.

HeuristicHazard—the Transaction Manager is aware that a transaction might have resulted in a mixed outcome, where some participating resources committed and some rolled back. But system or resource failures make it impossible to know for sure whether a Heuristic Mixed outcome definitely occurred. The underlying cause was most likely heuristic rollback or heuristic commit decisions made by one or more of the participating resources.
When an heuristic completion occurs, a message is written to the server log. Refer to your database vendor documentation for instructions on resolving heuristic completions.
Some resource managers save context information for heuristic completions. This information can be helpful in resolving resource manager data inconsistencies. If the ForgetHeuristics attribute is selected (set to true) on the JTA panel of the WebLogic Console, this information is removed after an heuristic completion. When using a resource manager that saves context information, you may want to set the ForgetHeuristics attribute to false.

Friday, April 29, 2011


These are the steps I followed. This was a straightforward task to configure apache to accept HTTPS traffic. Further enhancements such as forcing HTTPS only, or validating client certificates were not required.

This article is quite good

If on linux then install mod_ssl. ( )

yum install mod_ssl ...

Also is handy for checking any old CSRs

Setting up certs etc.

generate a certificate.


To do this you need a Certification Authority, CA, (openssl to DIY, or public CA). See links above

1/ Generate CSR (Certificate Signing Request)

Multiple ways to do this. OpenSSL or Keytool.

For openssl using defaults you can simply do

openssl req -new

2/ Generate certificate (using CA)

3/ Install certificate in Apache (as root)

4/ Edit ssl.conf file (equivalent to httpd.conf). Configure any VirtualHosts you require, and assign them the requisite keys and certs, e.g.

SSLEngine On
SSLCertificateFile /etc/pki/tls/http/apachecert.pem
SSLCertificateKeyFile /etc/pki/tls/http/apachekey.pem
SSLProtocol All -SSLv2
DocumentRoot /var/www/html/ssl

Create CA private Key (used for generating CSR’s)

openssl genrsa –aes256 -out 1024

In general generate a private key per domain

Create Self Signed CA cert (Also generates private Key for CA)

openssl req -new -x509 -extensions v3_ca -keyout ./private/cakey.pem -out cacert.pem -days 3650

Create Self signed CA using existing private key

openssl req -new -x509 -extensions v3_ca -key privateKey.pem -out cacert.pem -days 3650

This command demonstrates the how commands in openssl combine functionality

Req command allows switch to x509


The default locations of the of the keystore files is %JAVA_HOME%/jre/security/.keystore

To include trusted certs from the keystore in a java client you must add the following


Unsupported OID in the AlgorithmIdentifier
<ms_osb1> <[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1315908488277> <BEA-090898> <Ignoring the trusted CA certificate "CN=T-TeleSec GlobalRoot Class 3,OU=T-Systems Trust Center,O=T-Systems Enterprise Services GmbH,C=DE". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

I was doing some work with 2-way SSL, and seeing the following error in my weblogic (OSB) logs.

As it happens this is a problem with JDK 1.6_013. They included a non compatible CA cert in the keystore.

keytool -delete -keystore $JAVA_HOME\jre\lib\security\cacerts -alias ttelesecglobalrootclass2ca -keystorepass changeit
keytool -delete -keystore $JAVA_HOME\jre\lib\security\cacerts -alias ttelesecglobalrootclass3ca -keystorepass changeit

How do I remove a passphrase from a key?

Perhaps you’ve grown tired of typing your passphrase every time your secure daemon starts. You can decrypt your key, removing the passphrase requirement, using the rsa or dsa option, depending on the signature algorithm you chose when creating your private key.

If you created an RSA key and it is stored in a standalone file called key.pem, then here’s how to output a decrypted version of the same key to a file called newkey.pem.

# you'll be prompted for your passphrase one last time

openssl rsa -in key.pem -out newkey.pem

Often, you’ll have your private key and public certificate stored in the same file. If they are stored in a file called mycert.pem, you can construct a decrypted version called newcert.pem in two steps.

# you'll need to type your passphrase once more

openssl rsa -in mycert.pem -out newcert.pem

openssl x509 -in mycert.pem >>newcert.pem