New host

This blog is now hosted on my own server:

http://extremejava.tpk.com.br

Blackmagic with NetBeans and OpenOffice

This one is one of the weirdest (and useful) things I ever seen in 16 years of my life as a developer (I'm 26, BTW)... OpenOffice is a very powerful Office Automation solution... NetBeans Plaform is a very powerful application platform... Can you imagine what you get when you mix both together? Yup, this is possible. With NB and OOo installed, you just need to create a NB Module with following JARs from OOo:

• juh.jar
• jurt.jar
• officebean.jar
• ridl.jar
• unoil.jar

I created another module with a single TopComponent, and add this snippet:

private OOoBean oo;
@Override
public void componentOpened() {
  try {
    add(oo = new OOoBean());
    oo.loadFromURL("private:factory/swriter", null);
    oo.aquireSystemWindow();
  } catch (Exception ex) {
    Exceptions.printStackTrace(ex);
  }
}

But, this will NOT work... If you try to run it, an exception will be thrown, because OOo libs (.DLL/.SO) will not be found. They only exists in OOo install folder, and are automagically found by OOo classes - only if they aren't moved from the "classes" folder in OOo installation.

The workaround is a couple of classes in OOo SDK and The Ultimate Blackmagic(TM) - patent pending:

1. Grab odk / source / com / sun / star / lib / loader from OOo's CVS and add it to integration module.


2. Add unowinreg.dll to Windows Path (if you are on Linux, skip this step).


3. Alter Loader.java and add this after line 205 (after "UnoInfo" call):
   

   vec.add(new File(fClassesDir, "officebean.jar").toURL());

   
   This part of the magick adds "officebean" to classpath. Here lies OOoBean, and is optional if you are only using UNO interfaces. Other JARs are correctly listed by UnoInfo.


4. Now, the ugliest part of the evil potion: create a Facade. Using OOoBean directly will lead to some ugly ClassLoader conflicts. I've no time to fix this, but the following works:
   

   
   public static Container buildOOoBean() {
     return (Container) Loader.getCustomLoader().
       loadClass("com.sun.star.comp.beans.OOoBean").
       newInstance();
   }
   
   public static void showOOoBean(Container oo) {
     Class c = oo.getClass().
       getClassLoader().
       loadClass("com.sun.star.beans.PropertyValue");
     Class ca = Array.newInstance(c, 0).getClass();
     Method m = oo.getClass().
       getMethod("loadFromURL", String.class, ca);
     m.invoke(oo, "private:factory/swriter", null);
   
     oo.getClass().
       getMethod("aquireSystemWindow").invoke(oo);
   }
   



5. Back to the TC, I use "buildOOoBean" in ComponentOpened and "showOOoBean" AFTER the component is shown (does not work otherwise).

Yes, it is the purest evilness, but works (with some bugs):


This level of indirection is mandatory, because client module's ClassLoader is different than integration module's. If I do "OOoBean oo = new OOoBean()", the declaration and instantiation will try to use the JARs bundled. Since they don't lie on OOo install folder, the libraries will not be found. If I use "oo = (OOoBean) Loader.getCustomLoader().getClass("...").newInstance()", the "OOoBean" class from declaration will NOT be the same as the class instantiated. If you ever worked with JavaEE 4 and JSF/Struts/etc, you know that same classes from different ClassLoaders throws ClassCastException.

OOo works really well with Java, but NetBeans ClassLoader is "self-contained" (i.e. your bundled application must include EVERYTHING you use). This is a very nice architecture, but does not help if you if you need to interface with external applications.

Asynchronous Node children in NetBeans Platform

5 minutes playing with the new NetBeans 6 platform and I found a very useful feature. In previous post, I download URL. Using that code, I want to create a list of POJOs (that will be represented by a list of Nodes). In old NB5 days, I had to create a Thread and a lot of syncronization code to make the download run in background. In NetBeans 6 is easy as 1, 2, 3:

1. Subclass ChildFactory (from Nodes API), implementing createKeys and createNodeFromKey


2. Create an AbstractNode passing Children.create(MyChildFactory)


3. Compile, run and enjoy!

HTTP POST download with Java's URL class

This is easy, but I always forget the recipe. I need to download and process a file using a POST request. The code is:

URL url = new URL(FORM_URL);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);

OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
out.write("field=value&field=value&...");
out.flush();

BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;

while ((line = in.readLine()) != null) {
    // do something
}

out.close();
in.close();

The only catch is to use URLEncoder.encode to translate keys and values to the x-form-url-encoded style.

Transactions in EJB3

A little pause in the Quest for the Java Portal. I'm creating a JavaEE application, and I got a simple problem, but not well explained on the Internet. The scenario:

• EJB module has:
• Some CMP beans (with @ManyToOne and @OneToMany relationships);
• A Session Facade (stateless local bean).
• WEB module has:
• A jMaki-powered menu tree;
• A JSF-managed bean (that populates the jMaki tree with a call to Session Facade).

It is very simple, but I got a ServletException (caused by a LazyInitializationException) when I try to navigate the relationships on web container.

Googleing this problem, I found that the transaction of a stateless bean (using a "requires" or "requires new" attribute) is closed after the method returns, if the container opened it. Specifically, it happens when the EntityManager is destroyed. This is obvious, but not very intuitive. Solutions:

• Use a stateful bean. The Entity Manager will "survive" along with the session, but I think this will make the transaction survive that long, too. Not the best solution;
• Open the transaction in web container. This is strange, but logical: if I want to access the database (with lazy fetching), I must have an active transaction. EJB container is prepared to reuse the transaction by using the @TransactionAttribute annotation (defaults to "REQUIRED"). Using the brand-new resource injection, it is a matter of creating a "@Resource UserTransaction tx" attribute on my JSF managed bean. It is not nice to inject a transaction on every bean of web container, so, you only need to create a web filter that opens before chaining the operation.
  

As a conclusion, my contribution to JavaEE community. If you want to use Lazy Loading on web containers, you just need to create a Filter like this one:

public class TransactionFilter implements Filter {
 @Resource
 private UserTransaction tx;

 public void doFilter(ServletRequest request,
         ServletResponse response,
         FilterChain chain)
         throws IOException, ServletException {
     try {
         if (tx.getStatus() == STATUS_NO_TRANSACTION) {
             tx.begin();
         }
     } catch (NotSupportedException ex) {
         throw new ServletException(ex);
     } catch (SystemException ex) {
         throw new ServletException(ex);
     }
 
     chain.doFilter(request, response);
 }

 public void destroy() {
 }

 public void init(FilterConfig filterConfig) {
 }

}

The quest for the Java Portal - Profiling Liferay

It's time to run the Profiler. I start JBoss with "run-nbprofiler.bat" (created by NetBeans) and ask NB to attach profiler. I'm using "analyze performance", filtering Java core classes. JBoss runs very slow, but this is normal, since profiler is collecting execution of every method.

My first try showed that Lucene is called a lot of times, even when server is on "idle" state. Maybe it is a background job that should have a smaller frequency.

The second try filters out Lucene classes (using NB Profiler options). I found two bootlenecks: com.liferay.portal.lucene.IndexWriterFactory.decrement (takes 58% of startup time) and JBoss classloaders. And I found that Liferay auto deploy is running too often (~5 seconds of delay). Nice for develoment, awful for production use.

I had to remove JBoss classes from profiling, and I got amazed on things I discovered. I got a OutOfMemoryError (perm gen space) - solved by adding "-XX:MaxPermSize=256m" to run script - but I could find a lot of interesting things:

1. getResourceAsStream was called 12068 times;
2. Xerces' ChildNode was instantiated 853808 times (does anybody imagine why Java is "slow"?);
3. Stacktrace got really deep. About 50 levels or more, not including JBoss, Lucene and JDK classes;
   
4. A chain of 7 filters was called before hitting Liferay's MainServlet. If you consider a little forward made after MainServlet, we reach 14 filters before hitting a JSP file;
5. JSP compilation took 19s on this environment - total execution was 180s;
6. LR's VelocityTaglib has 8 iconXXX() methods that took 8 seconds each - detail: each one forwards another request;
7. Everytime a "include" is made, the chain of 7 filter is called. And there's a LOT of includes.
   

Remember that this was only ONE hit and the request wasn't complete, because of the OOM error. After that, I'm going to have some fun in the "real world". I'll try to go deeper tomorrow. Maybe I can send a RFE to Liferay team after I organize the arguments.

The quest for the Java Portal - Running Liferay

Now, I have a profiled environment. But, when I tried to run JBoss (without profiling), I got an error in counter-ejb module. Its classpath is not correctand, so, I added this line to my build.[user|computer].properties:

classpath.manifest=[original line in build.properties] lib/doubleclick-filter.jar

I reported this bug on Liferay JIRA [#LEP-2406]. After another 16 minutes of compilation, I forgot to initialize the database. I had two options: use a diferent connection pool or initialize it before deploying Liferay. I prefer the second, so, the easiest way was to create a MBean that depends on LiferayPool. Hypersonic is smart enough to allow multiple SQL commands in one Statement. I will upload the code later, but it is a matter of create a MBean that reads the script provided by Liferay and runs it on the poll.

This leads to an strange error about Spring transactions with EJB, Hibernate, JBoss and all. I don't remember the details, but the solution is to change the debug attribute of CachedConnectionManager. This is something I can't do in my shared JBoss server, so, I used an dirty trick:

1. Start JBoss with no Liferay, but with the original data source (liferay-db.xml configured);
2. JBoss translates your -ds file into and -service file and logs it with debug level;
3. Grab the translated file in log, create the -service, delete the -ds and indent the file - this will help you understand its structure;
4. Now, the funny part: copy CacheConnectionManager definition from jboss-jca.xml, paste into liferay-service.xml (inside the "CachedConnectionManager" optional attribute), and rename the MBean to an unique name - I put a ",name=xxx" suffix.

This will create a custom CCM to Liferay, without violating the original instance. That's what I love in Java (specially in JBoss): you can create a Lego-like software that is just a matter of do the bindings.

Liferay runs fine, but I found two bugs: a ClassNotFoundException about ical4j, and the contents of Guest community are blank. The first, I solve by adding lib/ical4j.jar to manifest classpath (as above). The second I don't care, since I will clean everything when I deploy the real application.

The quest for the Java Portal - Compiling Liferay

I got some weird exceptions with JBoss Portal, so, I decided to play around with Liferay. I have it running on my shared JBoss, and I have some ideas to their structured articles feature, so I will try harder on Liferay.

I've decided to check the bootlenecks on Liferay, because it is too slow. I'll use NetBeans Profiler. Using NB's ability to create projects using existing Ant scripts, I've done some setup:

1. Create an Java Application project using an already existing Ant script;
2. Add all "src" folder to the source folders list - about 20 of them (I guess this isn't necessary unless we want to change anything);
3. Adjust compilation build to the "start" target and run to "deploy" (Liferay does not set "start" as dependency to "deploy");
   
4. Create a "build.[user|computer].properties" and "app.server.[user|computer].properties" to customize some build parameters. I dislike Jikes, so, I'm using "javac.compiler=modern". The rest of properties are straighforward to customize, but folders on app.server must be correctly configured - JBoss predefined values does not work on an out-of-box installation;
5. After building a lot of modules (about 10 minutes on an almost empty Windows box powered by an Athlon XP 3200+), more than 2000 classes - yes, two thousands - are compiled without errors. Running the "deploy" target will install Liferay on JBoss. It installs some JARs on server's lib folder - I will change this later, before uploading to the real server.

Now, you can run JBoss. But first, some one-time configuration:

1. Add a datasource. To use profiling, I created a memory-only HSQLDB:

   <?xml version="1.0" encoding="UTF-8"?>
   <datasources>
   <local-tx-datasource>
     <jndi-name>jdbc/LiferayPool</jndi-name>
     <connection-url>jdbc:hsqldb:.</connection-url>
     <driver-class>org.hsqldb.jdbcDriver</driver-class>
     <user-name>sa</user-name>
     <password></password>
   </local-tx-datasource>
   </datasources>

2. Use NetBeans Profiler (Profile | Attach Profiler) to prepare a special run script to active profiling on JBoss - this file will be called "run-nbprofiler".

Now, we have an working environment. Next step is test and profile.

The quest for the Java Portal - Testimonial

I was supposed to be an user of Java Portals, not a developer. Things like JBoss management portlet are annoying, but it is just a matter of user experience. At least JBoss Portal deploys with minimal changes. Actually, I only need to change deployment descriptors.

Portals like Gridsphere, Jetspeed and others are unusable to normal users, since they need to know Java very well. How can I replace Zope/Plone in my job if every environment admin will need to know how to use Maven/Ant/etc?

Is that hard to develop an JavaEE application that makes users' life easier?

If I need to do any developer task, I prefer to develop my own portal that conforms to JSR-168 without violating JavaEE specifications.

BTW, I'm talking as a Java Portal user (i.e. portal deployer/admin). I'm not talking as the user of the portal I will create with a Java Portal (i.e. my "costumers").

End note: I'll try JBoss Portal beta-1, since it was released this week, but I won't use it if I still needs that buggy management layout.

The quest for the Java Portal - Gridsphere Tests

GridSphere compiled in an zero-error-in-deploy WAR file. But, with my well-known luck and my incredible skill of catch the weirdest bugs on Earth, I got the classic "error-page-that-throws-error" bug. A page throwed an error. This triggers the error page. But, the same error was throwed. This triggers the error page. I love when this happens. My luck was the StackOverflowError. In some EPTTE bugs the error page leads to HTTP infinite redirection.

The origin of the error? This little piece of code:

public synchronized Object deepCopy(Object oldObj) throws Exception {
   ObjectOutputStream oos = null;
   ObjectInputStream ois = null;
   try {
       ByteArrayOutputStream bos =
           new ByteArrayOutputStream(); // A
       oos = new ObjectOutputStream(bos); // B
       // serialize and pass the object
       oos.writeObject(oldObj);   // C
       oos.flush();               // D
       ByteArrayInputStream bin =
           new ByteArrayInputStream(bos.toByteArray()); // E
       ois = new ObjectInputStream(bin);                  // F
       // return the new object
       return ois.readObject(); // G
   } catch (Exception e) {
       throw e;
   } finally {
       if (oos != null) oos.close();
       if (ois != null) ois.close();
   }
}

I will not comment the weird "catch e throw e" fragment, but this was the most expensive way to follow the Prototype pattern (deep copy variant) I ever seen. You don't need to be an extreme Java programmer to notice this code will fail on the first non-serializable class found by oos.writeObject. Now, guess what was the exception I got?

java.io.NotSerializableException: org.apache.log4j.Level

After digging into the code, I discovered that GS team knows the use of transient keyword, but didn't used it on BasePortletComponent.java, line 42:

protected Log log = LogFactory.getLog(PortletPageFactory.class);

How this works on Tomcat is a mistery to me. Changing it to a final static attribute and adding some dependencies of dependencies (Jackrabbit needs Lucene and yet another logging implementation - SLF4J), made it work, but not as expected.

Result: an installation page with some really ugly URL rewrite techniques.

<link ... href="/tmp54563GridSphere-exp.war/...css" ...>
<img src="/@GRIDSPHERE_DEPLOY@/...png" ...>

The second one is a side-effect of using my own build script. The first one (along with cool design patterns) gives GridSphere an one-way ticket to my attic.

The quest for the Java Portal - Gridsphere Compilation

Current version of Gridsphere, 3.0.3, supports Tomcat and Jetty only. Since I'm extreme (-lly nuthead), I've tryed to run it on JBoss. I haven't found any live demo on its site - too bad.

Like Jetspeed and eXo, there's no WAR/EAR/SAR/CAR/etcAR file to download - I must download the source and compile (there's a corrupted, hidden WAR file on server, if you edit by hand the URL of download). Good news is: it uses Ant (not Maven) - that means I does not need to dig into obscure and unstructured XML. IMHO, Maven is good for the main developer, but not for the bug hunters, like me.

As expected, it wasn't straightforward. The steps:

1. Set CATALINA_HOME to JBoss' server folder - "$JBOSSHOME/server/$instance". Compilation script needs a "lib" folder with JavaEE API. Tomcat 5 stores on "$C_H/common/lib", Tomcat 6 stores on "$C_H/lib". Gridsphere "detects" tomcat version by verifying if "$C_H/common/lib" exists.
   
2. This leads to a "$C_H/bin" not found. After some digging in build scripts (I love Ant), I found an strange task that adds Tomcat's bin dir to classpath:

   110 <fileset dir="${appserver.home}/bin/">
   111     <include name="*.jar">
   112 </fileset>
   

3. Removing it and running the undocumented "ant war" task, creates the web archive.
4. The archive will not deploy: a ClassNotFoundException nested in a WSException. The class that was not found was the CaptchaServlet. The CNFE is strange, since the class is on "WEB-INF/lib" folder. Since I will not allow new users, I removed the reference from web.xml, but the same exception was thrown, with class GridSphereServlet.

At this point, I got tired of editing files by hand, and I created a NetBeans project to handle everything to me. This is the approach I'm doing with Liferay, and, likewise, I found a lot of dependencies, like Castor (yet another binding framework), JFreeChart, and others.

BTW, JFreeChart deserves some lines. First, I thought one of JFree JAR files were corrupt in GS distribution. I've decided to download it from JFree site and I can't see the documentation because IT IS NOT FREE. Yup. I can download the sources, but not the developer documentation.

And, like every giant project, I found useless things, like Perl5Util. I don't want to discuss if P5U is useful, but use it to replace regex like "\.xml$" with "" is a waste of resources, since String.replaceFirst can do the job.

Finally, I could compile with zero errors (and a lot of unchecked warnings). But it didn't run. I got an weird exception in Castor. After some reverse engineering, since there isn't architetural documentation, I found I need to add six XML files to their correct folders. In original GS structure, these XML lies on a "config" dir, but are copied to different folders by one of the 10 Ant scripts. Copying them made a deployable WAR file.

This is a thumbs up no NetBeans: users will always have an well organized project structure. There are no lost files on lost folders needing a lost task on a lost script to copy them to the correct place.

BTW, the problem was a static Class.getResource returning null. I had to change the code (adding some assertions) to figure it out.

Tip to Gridsphere team: use NetBeans to manage the project...