Thursday, December 23, 2010

Web Security (and Weblogic)

Web Security (and Weblogic)

When defining security roles for a web app.

1/ In your web.xml
Declare how and where the authentication should occur.


2/ In your web.xml
Declare your authorization roles


For Weblogic.
In the security realm different security models are possible. This is set from the
Security Model Default: setting
Options are `DD (Deployment Discriptor), Custom Roles, Custom Roles and Policies, Advanced..
When deploying you can set mode
What security model do you want to use with this application?
DD Only: Use only roles and policies that are defined in the deployment descriptors. Custom Roles: Use roles that are defined in the Administration Console; use policies that are defined in the deployment descriptor. Custom Roles and Policies: Use only roles and policies that are defined in the Administration Console. Advanced: Use a custom model that you have configured on the realm's configuration page.

In your Weblogic.xml you can perform the role/ group/ user mappings
(one entry per role)


Or you can delegate the mapping back to the Weblogic admin console (better).. See my earlier post on defining Weblogic Roles
(one entry per role)


Oracke Soa Suite

(work in progress) I will fill this in as I progress

Instructions for Soa Suite

Best place to start is a tutorial

This tells you what you need to download

· Task 1: Install Oracle JDeveloper Studio

· Task 2: Install the Fusion Order Demo Application

· Task 3: Install Oracle SOA Suite

· Task 4: Create a Connection to an Oracle WebLogic Server


Download from

Then you need to download Extensions from (

Oracle SOA Composite Editor

Pretty print Xml

If you want to format your Xml..theres lots of ways

This url uses Xerces

(it also uses jtidy which I could not get to work with pure XML.. (kept adding HTML tags).

If you want to avoid xerces and just stick to standard java then you need to use the Transformer.

Just to note that top rated answer requires the use of xerces.

If you don't want to add this dependency then you can simply use the standard jdk libraries.

(Note if an error occurs this will return the original text)



import javax.xml.transform.OutputKeys;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.sax.SAXSource;
import javax.xml.transform.sax.SAXTransformerFactory;

import org.xml.sax.InputSource;

public class Test {
public static void main(String[] args) {
Test t = new Test();
System.out.println(t.formatXml("text D"));

public String formatXml(String xml){
Transformer serializer= SAXTransformerFactory.newInstance().newTransformer();
serializer.setOutputProperty(OutputKeys.INDENT, "yes");
Source xmlSource=new SAXSource(new InputSource(new ByteArrayInputStream(xml.getBytes())));
StreamResult res = new StreamResult(new ByteArrayOutputStream());
serializer.transform(xmlSource, res);
return new String(((ByteArrayOutputStream)res.getOutputStream()).toByteArray());
}catch(Exception e){
//TODO log error
return xml;


Weblogic DB issue

Found on Weblogic 11g

I have a few DataSources defined. Some of which are on the same DB server just different SID’s (Oracle DB’s).

One of the DB instances was removed. Then all instances of DB’s on that server failed to start, (Even though the other DB was still running. When I untargeted the problem server and re-started, then the 2nd DB started fine.)

Debugging Log4j issues


Handy for log4j issues

Set the following switch when you start the JVM to see what log4j is upto.


Check for log4j output. This will tell where it is loading its configuration from.

Note be careful with multiple webapps having multiple log4j.xml or These can override your settings.

This next section is not true.. Its was actually caused by log4j delayed write to log file.. However I'm still including it because the detection techniques are useful.

(My actual problem was incredibly simple, and I shoudl have checked this first, but I made an assumption, and as we all know assumption is the mother of all F£$k ups.. The tool I was working on was using Simple Logging Facade for Java (SLF4J).. The developpers had simply included the slf4j-jdk.jar fil in the classpath instead of the slf4j-log4j.jar. Fixing my problem was simply a case of replacing the wrong jar. USing the log4j debugger was useful however in pinpointing what configuration files were ben picked up.

Monday, November 08, 2010

Weblogic’s many jar files...

Heres some problems I encountered trying to run some applications using Weblogic.


java.lang.NoClassDefFoundError: weblogic/kernel/KernelStatus

I was banging my head against a wall with a

Exception in thread "Main Thread" java.lang.NoClassDefFoundError: weblogic/kernel/KernelStatus

While trying to write a simple standalone program to connect to a Weblogic instance via JNDI, and manipulate some JMX values. The solution was provided via this posting

To actually run the Main method class in Workshop, I right click on the class in the Project Explorer and select Run As->Open Run Dialog. In the classpath tab, be sure to add the wlclient.jar file located here \wlserver_10.3\server\lib to the User Entries section and remove the WebLogic System Libraries from the Bootstrap Entries section. If you forget to remove the WebLogic System Libraries, you will get a stack like this: Exception in thread "Main Thread" java.lang.NoClassDefFoundError: weblogic/kernel/KernelStatus

There is also a wlFullCLient.jar file that may be useful in other scenarios


Just when you thought it was safe however, you get the following

ClassCastException: weblogic.jdbc.common.internal.ConnectionEnv


the advice is as follows

I would suggest you to remove the wlClient.jar from the classpath.
Also make sure that there are no other jar files related to weblogic in the classpath.
Add weblogic.jar in the classpath and make sure that this weblogic.jar is the same jar that is in the weblogic server classpath.

This happens because of the difference in the jar used by the Client JVM and server JVM.

In fairness this does fix the problem.

Depending on what your client does, it may be necessary to include the Weblogic.jar (if its doing some funky stuff), to ensure it has all the correct classes. Be careful from oracle workshop (eclipse) however of attached bootstrap class loaders.

3/ Missing servlet jars/ JEE standard jars.

You may run into problems with missing J2EE standard jars. I for example was getting errors about HttpServletRequest etc from the Servlet Api. In previous versions of weblogic, the weblogic.jar file would contian all of these, but since 10.3.x they have stopped including it.

When looking up missing jars, try the following, check the following...


In there the MANIFEST.MF contains a list of jars that include the usual J2EE jars etc. These have been removed from Weblogic.jar so may be needed.

Finally... I've mentioned this before but it is important when working with Ears and Wars on weblogic.

If you start seeing exceptions such as

java.lang.ClassCastException: weblogic.xml.jaxp.RegistrySAXParserFactory cannot be cast to javax.xml.parsers.SAXParserFactory

deep in the Weblogic code, you have more than likely run into a dreaded classpath incompatibility.

To avoid this use the following in your weblogic.xml




This will then result in a lot of ClassNotFoudn exceptions which may mean a hunt through your libs to add missing files

For instance heres the issues I ran into

java.lang.ClassCastException: weblogic.xml.jaxp.RegistrySAXParserFactory cannot be cast to javax.xml.parsers.SAXParserFactory

This means that we need a newer version of the SAXParserFactory than provided by Weblogic (we are using Bea jdk 1.6, so thast is picking up the lastest version of javax.xml.parsers.SAXParserFactory, but then weblogic.xml.jaxp.RegistrySAXParserFactory is been instantiated and this refers to an older version of SAXParserFactory)

The problem in my case was un-resolvable. The incompatible jar was needed to perform translation of a binary Xml format, but unfortunately it also included a newer version of xmlparser. This newer version then started causing problems with taglibs in Jsps. Our fix was to move of deployment from 10.3.1 to 10.3.3

4/ java.lang.NoClassDefFoundError: weblogic/utils/NestedException

After trawling the web there are a lot of references to wlFullClient.jar

This refers to rolling your own version of all wl client jars.

What might be simpler is to simply add the required jars.

This is the list of client jars.











For NestedException the required jar is wljmsclient.jar

Thursday, November 04, 2010

Tomcat deploy arbitrary dir

To deploy a webapp from a directory not in the appBase (the webapps directory by default) of Tomcat /Catalina, then its simple.

At its most simple you simply create a file in the $CATALINA_BASE/conf/[enginename]/[hostname] directory.

The name of the file is the webapp Context name you want.

The contents must include the docBase attribute. This points the location of the webapp.

This is very useful for development, so every change can be immediately realised on the webapp without restarting, or undeploying/ deploying (which is necessary if you deploy it using Tomcats manager, since it copies the apps into the webapp directory.)

e.g. save SpringSchedule.xml in the $CATALINA_BASE/conf/[enginename]/[hostname]

<Context docBase="F:/eclipse_workspace/SpringSchedule/www">


Note: If you add properties files, or update the web.xml you will need to reload the webapp from the tomcat manager console.


Eclipse java command line

1/ You can generate the java command used by eclipse to run/ or debug an application, with all included classpaths etc. This can be useful if you want to later script the execution of the application.

It is visible by debugging the application (debug as application). From the debug perspective, right click on the debug view, to get the context menu. Select properties. The command line is visible and can be copied.

2/Also, its possible to run eclipse itself from the command line in headless mode.

eclipsec.exe -noSplash -data "D:\Source\MyProject\workspace" -application org.eclipse.jdt.apt.core.aptBuild

It uses the jdt apt plugin to build your workspace automatically. This is also known as a 'Headless Build'. If you're not using a win32 exe, try this:

java -cp startup.jar -noSplash -data "D:\Source\MyProject\workspace" -application org.eclipse.jdt.apt.core.aptBuild

Saturday, October 30, 2010


I’ve been setting up a Quartz scheduler recently. We wanted it to store the jobs in a Database. Quartz helpfully comes with a set of sql scripts for virtually all DB’s to create and manage all jobs, triggers, schedules etc via a transaction and cluster aware scheduler, so there is no need to think about that aspect of it. The scripts are all stored in the Quartz distribution under docs/dbTables folder.

Once the tables are all set up you can use the example application provided with Quartz to test out the DB persistence (Example 13). Configure it to point to your database by updating the file

We then wanted to Spring-ify it. This was straightforward


public class RunSchedule {

static Logger _log = LoggerFactory.getLogger(RunSchedule.class);

public static void main(String[] args) throws Exception {

BeanFactory springContext = new ClassPathXmlApplicationContext("Spring-Quartz.xml");



public static void cleanUp(Scheduler inScheduler) throws Exception {

_log.warn("***** Deleting existing jobs/triggers *****");

// unschedule jobs

String[] groups = inScheduler.getTriggerGroupNames();

for (int i = 0; i <>

String[] names = inScheduler.getTriggerNames(groups[i]);

for (int j = 0; j <>

inScheduler.unscheduleJob(names[j], groups[i]);



// delete jobs

groups = inScheduler.getJobGroupNames();

for (int i = 0; i <>

String[] names = inScheduler.getJobNames(groups[i]);

for (int j = 0; j <>

inScheduler.deleteJob(names[j], groups[i]);






<beans xmlns="" xmlns:xsi="" xmlns:p=""xmlns:jee=""

xmlns:tx="" xmlns:context="" xmlns:jms=""xmlns:util=""


<bean id="runMeTask" class="ie.bge.middleware.RunMeTask" />

<bean name="runMeJob"


<property name="jobClass" value="ie.bge.middleware.RunMeJob" />

<property name="jobDataAsMap">


<entry key="runMeTask" value-ref="runMeTask" />




<bean id="simpleTrigger"


<property name="jobDetail" ref="runMeJob" />

<property name="repeatInterval" value="5000" />

<property name="startDelay" value="1000" />


<bean id="cronTrigger"


<property name="jobDetail" ref="runMeJob" />

<property name="cronExpression" value="0/5 * * * * ?" />


<bean name="plannedDowntimeScheduler"

class="org.springframework.scheduling.quartz.SchedulerFactoryBean" init-method=”start”>

<property name="quartzProperties">

<util:properties location="/" />


<property name="applicationContextSchedulerContextKey"

value="applicationContext" />

<property name="waitForJobsToCompleteOnShutdown" value="true" />



All works as expected.

N.B. Note the init-method attribute for SchedulerFactoryBean. This is to automate the starting of the scheduler after it is instantiated. It is better than having a code based startup, (which would implement the Spring InitializingBean)

When we integrated it into our main application however we started having issues.


Caused by: org.quartz.SchedulerConfigException: Failure occured during job recovery. [See nested exception: org.quartz.impl.jdbcjobstore.LockException: Failure obtaining db row lock: ORA-00942: table or view does not exist

[See nested exception: java.sql.SQLException: ORA-00942: table or view does not exist


The problem was due to autowiring. We had a datasource defined for a different purpose that was getting autowired into our SchedulerFactoryBean. (We were defining the datasource via the, and not injecting the datasource).

Manually wiring the correct dataSource fixed the problem. (or turninng off autowiring should also work as suggested by jrsisson)


BeanFactory or ApplicationContext?

Taken from Spring website

Users are sometimes unsure whether a BeanFactory or an ApplicationContext is best suited for use in a particular situation. A BeanFactory pretty much just instantiates and configures beans. AnApplicationContext also does that, and it provides the supporting infrastructure to enable lots of enterprise-specific features such as transactions and AOP.

In short, favor the use of an ApplicationContext.

(For the specific details behind this recommendation, see this section.)

CSS Javascript quickie

Center (Centre) a div

Its easy, but its not necessarily obvious. To centre a div you need to set the width, and the margin-left and right to auto,



width: 500px; margin-left: auto ; margin-right: auto ;


To center text within a div use.

text-align: center ;


Useful links explaining with nice and simple explanations of objects in javascript (I wonder how I never “got it” before considering how simply its described here)

Also the prototype modifier. Basically it allows properties, and methods to be added to objects. Effectively making javascript OO, instead of purely functional.

Saturday, October 16, 2010

Recording a desktop demo video/ viewlet

Just a quick note about recording a desktop demo (viewlet). These are very handy for demo’s on using software etc. They allow you to record the screen activity, while adding voice over commentary, and popup overlays.

One of the best freeware recorders is Wink

Also some image compression stuff on here.

Svn identity/ login

If subversion in eclipse is pestering you for login credentials every few seconds, then your credentials may have got corrupted.

You can reset the stored credentials by deleting the auth directory.

This is usually located at

Unix (linux/ Osx)


USER_HOME/Application Data/Subversion/auth

Taken from

.keyring file only used when JavaSVN adapter is selected.

Also, your repository may allow anonymous read access, so browsing and
update operation doesn't require you to log in. Supclipse only prompts for
credentials when credentials are requested by Subversion server.

Weblogic JMS

Link for looking up all jms queues using jmx

You can use a JMX program, WLST, JConsole, or the console.

See the documentation on using WLST

See the documentation on writing JMX clients

See the MBean reference documentation

The JMX distributed queue mbeans have type = weblogic.j2ee.descriptor.wl.DistributedQueueBean

You can either walk down the hierarchy starting at the Domain MBean and into the JMSSystemResource MBean or you can query for specific MBean types.


One issue that I’ve seen on a client site.

An OSB proxy is set up to consume from a JMS queue. Data is injected into the queue but the Proxy does not consume it.

(Only seen when custom dispatch policies were used)

This is a weblogic bug. The workaround is to go through the configuration of the proxy, and assign it a different Dispatch Policy. This would enable consuming of the queue. Then you can simply rollback the change in OSB, and consuming will continue.

JMS Unit of Order

Message Unit-of-Order is a WebLogic Server value-added feature that enables a stand-alone message producer, or a group of producers acting as one, to group messages into a single unit with respect to the processing order.
Note that it appears to cause problems with the JMS Api's for viewing message payloads.

Editied Note.. We found an MBean API for discovering these. It involves using the MBean operation getMessages (specifying a flag). I will blog about it later

Weblogic Roles

Just a quick reminder for myself. This is easy but on initial inspection it seems like there is something missing. In fact it is easy, but appearance can be confusing.

The high level is. You are using weblogic server to host your apps. You want some security policies for your apps. It therefore makes sence to use the weblogic security realm, and its roles, groups, users and policies to manage the entity.

-. Create new Role(s).

From weblogic console, goto Security Realm, Select realm, select Roles and Policies.

Now expand the global Roles, and click on the Roles link. This will allow you to create custom roles.

-. Create/ edit Groups to include new Roles

Assign users to groups

JSTL snippets

Simple when you know how. Frustrating and time consuming when you don’t
If/ else
Because of the xml format we can’t have a
<c:if test=”${actionBean.test}”>
Instead we have to use choose, when and otherwise.
<c:when test="${actionBean.test}”>

Looping through lists
Realated to above, I was trying to test if a list was populated.
First attempt was
<c:when test="${actionBean.result.size>0}">
javax.el.PropertyNotFoundException: The class 'java.util.Collections$Synchronize
dCollection' does not have the property 'size'.
at javax.el.BeanELResolver.getBeanProperty(
at javax.el.BeanELResolver.getValue(
at javax.el.CompositeELResolver.getValue(
at com.sun.el.parser.AstValue.getValue(
at com.sun.el.parser.AstGreaterThan.getValue(
Truncated. see log file for complete stacktrace
Solution is to use Not Empty
<c:when test="${not empty actionBean.result}">

Monday, October 11, 2010

Tuesday, August 17, 2010

Getting version of Oracle WebLogic Portal Server

How to determine the exact version of Oracle Portal Server you are using?

You can determine the Portal version with the following query executed in the Portal MR database as a dba user.
Substitute your schema name where you see

select VERSION from

Wednesday, June 16, 2010

Mercurial behind a proxy server

I found some web resources that confused things when I was trying to figure this out so I thought I'd add a page.

Mercurial (hg) stores its config information in an ini file. Type
hg help config
to see the details.

Based on this its possible to have a config file per repo with differetn setting (e.g. email addresses).

The format of the config file is explained at

So to set a proxy server I used the follwoing ini file

username = My Name <>
verbose = True

Wednesday, May 26, 2010

ScreenScraping with Groovy

I'm attempting to write a screen scraper for various websites, so I will document my findings here.

A quick search on the internet finds links to examples like the following

#!/usr/bin/env groovy
// Depends on tagsoup library:
def slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())
def url = new URL("")
url.withReader { reader ->
       html = slurper.parse(reader)
       //we should now have a parsed file
       def value = html.body.div.div.div[2]
       value.list().each { f ->
               println "\nPub : " << f.toString()[0..80] << "..."
This is super simple and works really well.
My problem was I needed to also post data to the websites (e.g. to login, enter data etc).
For that you need to incorporate HttpClient. For Groovy there is a HttpBuilder library that wraps HttpClient libraries with Groovy syntax. It also allows you to use GPath expressions to quickly identify locations in the response page.
I needed to go through a proxy and this proved the first hurdle. After much messing this code worked.
(Note I added an if statement to get the current IP address of the machine, so this script would work in work (where we use a proxy), and at home (where I don't have a proxy)
#N.B. Also that httpBuilder can be got using grapes. However I had some problems getting this to work from behind a proxy. Check out this page for some tips
Also worth of note is that when using grab (grapes) all files are pulled down to $HOME/.groovy/grapes 

So you could also manually download the latest version of the HttpBuilder and manualyl install it
Eventually this worked for me (this was after grape resolve failed.. Not sure why.. Obviously it worked once it was installed)
>grape install org.codehaus.groovy.modules.http-builder http-builder 0.6
@Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.2' )
import static*
import static* 
def http = new HTTPBuilder( '' ) 
println "IP = $ip" if(ip.startsWith("10.5.") || ip.startsWith("10.2.")){
 def proxy = ""  def proxyPort = 80  //Required for HttpClient
 http.setProxy(proxy, proxyPort, "http")  //http.setProxy(proxy, proxyPort, "https")
 http.auth.basic( proxy, proxyPort,[""], System.getenv("user.password") ) 
http.get( path: '/', query:[id:'httpbuilder'] ) { resp, xml ->     
 println resp.status  
 println xml     
 xml.status.each {  // iterate over each XML 'status' element in the response:         
  println it.created_at.text()         
  println "  " + it.text.text()     
Like the simple example above (which used the TagSoup Sax Parser), the HttpBuilder parser includes an Xml Parser that can handle HTML
  • HTML response data will also be parsed automatically, by using NekoHTML which corrects the XML stream before it is passed to the XmlSlurper. The resulting behavior is that you can parse HTML as if it was well-formed XML.