Wednesday, July 13, 2005

Website notice

Just want to publicise www.billyeccles.com. This is a website I developed for, Billy Eccles (surprise surprise). Billy is a potter based in Dublin. He also happens to be my girlfriends father. So no pressure there... Anyway have a look and let me know what you think, and like in restaurants/ shops etc.

"If you don't like something on the site tell me.... If you like something, tell everyone" (and me as well).

Technical aside
You may notice that the site is a PHP site, and yet all my postings before now, have been Java related. That's correct. For this site I realised that if I was to develop anything worthwhile I needed to leverage as much free code as possible. So I'm using an open source CMS called Mambo. It saved me a lot of time.

Tuesday, July 12, 2005

Autostart Vnc

Took me a while to find this so I thought I'd write it down. Of course its easy when you know how.

Edit /etc/sysconfig/vncservers

Add the following

VNCSERVERS="1:username"

Where 1 is the display to start the vncserver on, and username is the name of the user you want to start vnc as (Not root). Simple as that. You should also run vncpasswd first and set the vnc password on the system.

Now a quick explanation

The /etc/sysconfig dir contains information on user services to be started up (unlike the /etc/rx.d dir that starts up system services). Information on these can be found in

RedHat docs


or on

/usr/share/doc/initscripts-/sysconfig.txt

Cross platform Jsp issues

I find this hard to believe, yet it seems to be true


Take a simple html tag with a jsp tag


<input name="text" value="<%=new String("Hello")%>"/%>


That works fine on tomcat. No problems...

On weblogic however, no go...

The problem is the nested quotes """

To get it to work, obviously a few solutions

<input name="text" value="<%=new String(\"Hello\")%>"/%>

or

<input name="text" value='<%=new String("Hello")%>'/%>

Both easy solutions once you know the problem. Of course it took me a while to find out that this problem. In my defence, I wasn't doing something so simple as the above example. I was having problems with a tiles definition.
After a bit of effort I now have some basic JSF pages working.

One error that stumped me for a good while was the following

javax.faces.el.PropertyNotFoundException:
Can't instantiate class ....

This came about from a backing bean I was tryign to configure from my face-config.xml


<managed-bean-name> Menubar </managed-bean-name>
<managed-bean-class> com.aol.beans.jsf.Menubar </managed-bean-class>
<managed-bean-scope> application </managed-bean-scope>
<managed-property>
<description>List of items to appear in the MenuBar and privilages, these get localised to menu.</description>
<property-name>itemsAndPrivilegeLevel</property-name>
<property-class>java.lang.String[]</property-class>
<list-entries>
<value>home,1</value>
<value>profile,1</value>
<value>performance,1</value>
<value>transaction,1</value>
<value>reference,1</value>
<value>sysAdmin,3</value>
<value>production,2</value>
<value>logout,1</value>
</list-entries>
</managed-property>
</managed-bean>


Basically I was trying to set the itemsAndPrivilegeLevel parameter in the bean.

The problem was I hadn't created a getItemsAndPrivilegeLevel method in the bean. I dodn't think I needed one (I don't), but apparently either JSF or the beans specification think different, so I have to have that method there.

The perils of globalisation

... from a programmers persepctive...

Call it globalisation or internationalisation, either way it not as straightforward as I'd like to believe.

As a java programmer it all seems easy. Use a ResourceBundle and store all your text in handy resource files. These can easily be translated as needs require. However once you've externalised all of your user displayed text to property files, thats when the fun really begins. Especially if you're dealing with a non Western script.


Funny how Japanese text looks like fractions, symbols and punctuation marks you may find yourself saying....
.....
Once you get past that stage though, then its onto the world of character encodings.

Character encoding. It's something most Western language users blissfully ignore. Its only when you try and export your freshly translated text to say Japan that things come tumbling down.

The best links I found on this are here and here.

Ah ha, maybe that 3/4 symbol isn't in fact the japanese for Hello.

In fact its garbled character encoding. Looks like its time to instruct your webserver to serve up different character sets.

I started out setting up Tomcat to send UTF-8. There are a few distinct steps involved.


1.
The JVM must be initiated using the correct charset. This sets the format of the files that java will use (includes, properties file, jsp pages etc.)

This is done with the following switch

-D=file.encoding=UTF-8

so for Tomcat, I added the following to the cataline.bat file

set CATALINA_OPTS=-D=file.encoding=UTF-8


2.
Its important to realise the standard encoding used by the OS that you are running. My version of windows is using a Western Latin-1 script. This is unable to display japanese characters. In order to get japanese characters displayed I had to install japanese character supprt.

This also means that files (for example properties files) may be stored in a different character set than you may be expecting.

In my case, I was using Crimson as an editor. Crimson saves all charcters as Ascii, thus cannot display japanese character, or even Western scripts such as é or ß characters. I had to find a unicode enabled editor. Unipad is a good one that I found.

Java properties files do not by default support unicode. They must be encoded.

Once your files are stored in a unicode mapping, you have to run it through the nativeToAscii program which escapes out any strange foreign characters and puts in the exact unicode code point instead. (Theres an ant task which can be used for this).

<native2ascii encoding="UTF-8" src="$%7Bsrc.dir%7D" dest="${build.dir}" includes="**/*.properties">.

Once your basic java environment is set, you need to be able to set the character encoding for the page. This can be set with a page directive

<%@ page contentType="text/html;charset=UTF-8" />

or in xml syntax

<jsp:directive.page contenttype="text/html;charset=UTF-8">

4.
Finally (I think) set the encoding for the HttpResponse (So you can understand any text that gets sent back)

Using jstl

<fmt:requestencoding value="UTF-8">



Now in the above example I have used UTF-8 as the encoding, this encoding maps to a variable number of bytes (from 1 to 6). Western mappings e.g. ascii will normally be covered by 1 byte, but those foreign type scripts may take up to 6 bytes to be encoded.

If you are mainly directing pages to Japanese clients then UTF-8 will be quite wasteful. A fairer encoding might be UTF-16 (doesn't seem to be supported by IE, or Mozilla browsers), or even (I going out on a bit of a limb here) Shift-JIS.



Its all very nice, if it all works well , but if it doesn't..... Can be bit nasty.

What I'm looking at now is changing the encoding deplending on the locale. The serlvet 2.4 spec has a nice mapping

<local-encoding-mapping-list>
<locale>jp</locale>
<encoding>Shift_JIS</encoding>
</local-encoding-mapping-list>

which hopefully will sort things out.

If theres no more entires on this subject then it all worked out swimmingly

Location independent loading of properties

Heres another little douzie that been slapping me across the face all day.

When writing and deploying code you don't want to have absolute paths listed for config files, resources etc, as these will invariably lead to problems when moving the application to another server.

Normally with java this entails creating a deployment directory, and including within it all classes and resources that you may need. You then load all files and properties using the deployment directory as the root.

The question then arises how do you access these properties.

For example with properties files the initial way one thinks of to load properties is as follows


FileInputStream inputStream = new FileInputStream("");
Properties prop = new Properties();
prop.load(inputStream);


The problem with this is that the is an absolute value, which knows nothing of the deployment directory that you may be running from.

The solution is to use one of the following


Class.getResourceAsStream() or
ClassLoader.getResourceAsStream()


The funny thing is that for me sometimes the first method works (e.g. when run from within Eclipse), and the 2nd one fails. Sometimes (from with tomcat) the first one fails and the 2nd one works.

In fact initially I had posted to use the 2nd method after spending a few hours tracking down why my program worked in Eclipse but failed in tomcat. It was only a day later that I discovered that the 2nd method (which works in tomcat) doesn't work in eclipse.

So now I have this slightly ugly construct to try and get around the problems

/**
* The name of the properties files. N.B. the "/" at the start means it is an absolute
* package, not a relative path from this package.
* Default propertuies location /resources/rsi.properties in the classpath may be
* overridden by setting the System property rsiserver.property.file,
* as in the test ant target, or with java -Drsiserver.property.file=myNewPropertiesFile
*/
static String propLoc="/resources/rsi.properties";
static{
// First check system property to see if specific properties file is set
propLoc = System.getProperty("rsiserver.property.file", propLoc);
prop=new Properties();
InputStream is = Utils.class.getResourceAsStream(propLoc);
if(is==null){
log.debug("Failed to load Resource ["+propLoc+"] from Utils.class.getResourcesAsStream() trying classloader.");
ClassLoader cl=Utils.class.getClassLoader();
if(log.isDebugEnabled())
log.debug("ClassLoader = "+cl);
is=cl.getResourceAsStream(propLoc);
}

try {
//Throws IOException
prop.load(is);
} catch (Exception e) {
log.fatal("Exception loading Properties file.",e);
}
}




I'm at a loss to explain it, but I suspect (hope) that its some configuration problem.

Xml schemas and Namespaces?

With the amount of headaches I'm having doing a simple task like validating an Xml doc against a schema, I'm back to wondering if I shouldn't just throw this programming lark aside..

....

In the meantime heres the root cause of my most recent heartache....

The solution is ...

String xml ="<envelope><header2>Hi</header2></envelope>";
String xsd = "http://localhost:8080/xml/test.xsd";
SAXBuilder builder = new SAXBuilder("org.apache.xerces.parsers.SAXParser", true);
builder.setFeature("http://apache.org/xml/features/validation/schema", true);
builder.setProperty("http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation",xsd);
builder.setValidation(true);
builder.build(new InputSource(new StringReader(xml)));


Simple isn't it

What could have been the problem?

The problem is that innocuous line builder.setProperty("http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation",xsd);

To cut a long story short Xerces has 2 modes for validating external schemas (i.e. not the schemas referenced in the Xml doc itself). 1/ Namespace enabled 2/ No Namespace.

Simple huh?

Well it is once you know that simple fact. I on the other hand didn't . (Mind you I do think it should be possbile to validate an arbitary Xml doc without necessarily knowing if its namespace enabled or not. But thats for another day)

So in summary people

If you want to avoid cryptic exceptions like

org.jdom.input.JDOMParseException: Error on line 1: cvc-elt.1: Cannot find the declaration of element 'envelope'.
at org.jdom.input.SAXBuilder.build(SAXBuilder.java:466)
at XmlTest.main(XmlTest.java:41)
Caused by: org.xml.sax.SAXParseException: cvc-elt.1: Cannot find the declaration of element 'envelope'.
.....


NB. This is the same exception you get if Xerces cannot locate your schema file, or if you don't configure the paramerters correctly. Something more intuitive would be more helpful.

do the following

1/ If your Xml doc uses namespaces

String xml ="<envelope><header2>Hi</header2></envelope>";
// Namespae of xsd1
String ns1 = "http://www.w3.org/2001/12/soap-envelope"
String xsd1 = "http://localhost:8080/xml/test.xsd";
//Namespace of xsd2
String ns2 = "http://kevinj.develop.com/weblog/weblog.xsd";
String xsd2 = "http://localhost:8080/xml/test2.xsd";

SAXBuilder builder = new SAXBuilder("org.apache.xerces.parsers.SAXParser", true);
builder.setFeature("http://apache.org/xml/features/validation/schema", true);
builder.setProperty(
"http://apache.org/xml/properties/schema/external-schemaLocation", ns1+" "+xsd1+" + ns2+" "+xsd2);
builder.setValidation(true);
builder.build(new InputSource(new StringReader(xml)));


You must set the Property http://apache.org/xml/properties/schema/external-schemaLocation with a space seperated String list of the schema(s) to validate against.
The list takes the following form namespace uri namespace uri .... Its composed of pairs of values, the namespace that the schema will validate and the actual location of the schema file. (It can be any uri http://, file:// etc)

This section is taken from the JDOM FAQ


Schema locations are given by setting the property "http://apache.org/xml/properties/schema/external-schemaLocation" to a list of whitespace separated name-value pairs. The 'name' is the namespace the schema is associated with, the 'value' is the location of the schema for that namespace. For example:

builder.setProperty(
"http://apache.org/xml/properties/schema/external-schemaLocation", "http://www.w3.org/2001/12/soap-envelope soap-envelope.xsd" + " " + "http://kevinj.develop.com/weblog/weblog.xsd weblog.xsd");

The above example shows how to validate against multiple schemas -- against the SOAP 1.2 schema where the namespace is http://www.w3.org/2001/12/soap-envelope and the and against a schema for namespace http://kevinj.develop.com/weblog/weblog.xsd. The files describing these schemas are in soap-envelope.xsd and weblog.xsd respectively. You can add as many of these name value pairs as necessary. The values themselves are URLs. The name value pairs follow the meaning given in the Schema recommendation (http://www.w3.org/TR/xmlschema-1/#schema-loc ).




2/ If your Xml doc does not use namespaces

String xml ="<envelope><header2>Hi</header2></envelope>";
String xsd = "http://localhost:8080/xml/test.xsd";
SAXBuilder builder = new SAXBuilder("org.apache.xerces.parsers.SAXParser", true);
builder.setFeature("http://apache.org/xml/features/validation/schema",true);
builder.setProperty("http://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation",xsd);
builder.build(new InputSource(new StringReader(xml)));


Here we set the propertyhttp://apache.org/xml/properties/schema/external-noNamespaceSchemaLocation (spot the difference) with simply the schema uri. (Not a List as there is only the default namespace) to validate against, and only a single items since there is no namespaces, (i.e. not a pair like before)


It may look simple but that wee little problem had me stumped for longer than I'd care to mention.

Heres some links on the stuff

The Jdom faq Jdom uses xerces so this applies to xerces as well.
xerces bugzilla entry. There was a bug with this prior to version 2.2.0

One aside. By default xerces will not throw exceptions for validation failures, the parser must be configured to do that.

Jdom will configure the parser to do this by default. (for a lazy programmer like me this simply means that I use jdom to wrap my calls to xerces, rather than actually looking up how to configure xerces)

More on xml classpaths

Just an extra note on the xml classpath issue. jdk1.4 comes with an old version of xerces that must be updated for the schema validation to work. I discovered this today when I started running into the same problem as last Friday after I changed the java.endorsed.dirs to be %JAVA_HOME%/lib/endorsed. Even though jdk1.4 ships with a version of xerces, and its in the endorsed directory, my validation failed, until I copied a newer version of the xercesImpl.jar into the endorsed dir. I've used xerces 2.6.1 which does work.

Java Xml and schemas growing paings


I've just spent the morning stumped on a problem I had working before. I was trying to validate an Xml doc against a schema but kept getting the following.


Exception in thread "main" java.lang.ClassCastException
at org.apache.xerces.impl.xs.XMLSchemaLoader.processExternalHints(XMLSchemaLoader.java:567)
at org.apache.xerces.impl.xs.XMLSchemaValidator.reset(XMLSchemaValidator.java:1345)
at org.apache.xerces.parsers.BasicParserConfiguration.reset(BasicParserConfiguration.java:543)
at org.apache.xerces.parsers.DTDConfiguration.reset(DTDConfiguration.java:640)
at org.apache.xerces.parsers.DTDConfiguration.parse(DTDConfiguration.java:512)
at org.apache.xerces.parsers.DTDConfiguration.parse(DTDConfiguration.java:595)
at org.apache.xerces.parsers.XMLParser.parse(XMLParser.java:152)
at org.apache.xerces.parsers.DOMParser.parse(DOMParser.java:253)
at Utils.validate(Utils.java:70)
at XmlTest.main(XmlTest.java:29)


Some investigation on the internet turned up some red herrings

  • Problem with new version of xerces. I tried older version same problem

  • Problem with JDOM. (this is the api I normally use since it is far more intuitive than the basic jaxp or xerces classes). Re doing it in pure xerces didn't help.

  • Bug in xerces.



In the end (as is often the case) I took a break from it, and I had a hunch over lunch.

The problem is jdk 1.4. It includes some xml classes by default, and you need to override these to get xerces to work correctly.

This is done with the java.endorsed.dirs
property. Set this to be the dir where your xerces is stored and Voilia it works.

e.g.

C:\dev\sandbox>java -Djava.endorsed.dirs=\apps\xerces-1_4_3\ -cp classes;\apps\xerces-1_4_3\xerces.jar XmlTest


Check out this article on Endorsed Dirs

Just be careful as well. By default <java-home>\lib\endorsed is the deafult endorsed dir and this may override it. This probably works (I'm just too lazy to bother testing it).

C:\dev\sandbox>java -Djava.endorsed.dirs=\apps\xerces-1_4_3\;%JAVA_HOME%\lib\endorsed -cp classes;\apps\xerces-1_4_3\xerces.jar XmlTest



Alternatively copy all such jars into the default endorsed dir \lib\endorsed.

Hopefully now by putting this up here, maybe I can save someone else a frustrating half day.

Let me know if I do.