Archive for the ‘Java’ category

Tomcat 5 not reloading changes in JSP

2011/07/24

Part of the beauty of JSP development is the easy of trying out the change. All you have to do is edit the JSP file, save it and reload the page. Yesterday I got caught by surprise by not being able to do exactly that.

I saw some strange behaviour in an EL expression used JSP tag. As it was nothing ATG specific, I decided to try it out in plain old Tomcat rather than full ATG application. So I located the instance of Tomcat 5.5 (I always have couple of Tomcats lying around) and being lazy, I decided to inject the change into out-of the box jsp-examples Web application that comes with it.

I picked a victim JSP to be modified (jsp2/el/basic-comparison.jsp), made my change, reloaded page – nothing. I must have forgotten to save the file, did it again – nothing. Weird. Maybe browser caching. Cmd-Shift-R – nothing. Tried different browser – it showed unchanged file as before. Definitely not a browser issue.

The most logical explanation would be that I am editing different file than reloading: so I verified that the JSP file I edited is indeed under WebApps, that correct version of Tomcat was started. It was. There was no context redirection in place but to be sure, I unzipped pristine copy of apache-tomcat-5.5.31.zip (I have couple of those around as well), copied the JSP there, started it – same thing.

Really weird.

Few more experiments showed that no change to any JSP page under jsp-examples was visible. Removing JSP page caused 404 error as expected, but as soon as the page was there, it reappeared in original pristine version.

At this point, I started to suspect there is something fishy with the jsp-examples application itself. And it was. Brief check of the web.xml showed that it is quite long (24K) and that it contains declaration:

<servlet>
  <servlet-name>org.apache.jsp.jsp2.el.basic_002dcomparisons_jsp</servlet-name>
  <servlet-class>org.apache.jsp.jsp2.el.basic_002dcomparisons_jsp</servlet-class>
</servlet>

....

<servlet-mapping>
  <servlet-name>org.apache.jsp.jsp2.el.basic_002dcomparisons_jsp</servlet-name>
  <url-pattern>/jsp2/el/basic-comparisons.jsp</url-pattern>
</servlet-mapping>

and suddenly it all made sense.

As long as the JSP file was present (Tomcat checks), it’s content was completely ignored and the request for /jsp2/el/basic-comparisons.jsp was served by COMPILED class /webapps/jsp-examples/WEB-INF/classes/org/apache/jsp/jsp2/el/basic_002dcomparisons_jsp.class – which of course did not contain any of the JSP file changes. When I removed file, Tomcat reported 404 a- the check for file existence comes before check for URL mapping apparently.

Moral of the story: being lazy causes one has to do more work but often learns something new. Had I created fresh new WebApp, none of these mappings were present and I would never realized what is Tomcat 5 doing behind the scenes.

Btw, in Tomcat 6 the mappings are gone and everything behaves as expected.

Although this is not technically a bug – but a feature – it is unfortunate that this optimization is used exactly in one application that can be used by newbie developer learning JSP for experimentation and not seeing it’s change can confuse the hell out of them. Especially when googling the issue will very likely sending him/her in completely wrong direction – suggesting reconfiguring Jasper, playing with valves and internals of Tomcat configuration.

JSP compiler configuration in JBoss 4.2.x for ATG

2010/02/24

The post is on corporate blog – http://blog.thinkwrap.com/blog/2010/2/24/jsp-compiler-configuration-in-jboss-42x-for-atg.html

Bye bye Ruby, hello Groovy

2009/03/17

I first time discovered Ruby back in 2006 (yes, I know, I was late to the game), and immediately fell in love with it. The dynamic nature of the language, the consistency, pure esthaetics and practicality certainly changed the way how I saw software development and programming languages.

Since that time, I made several attempts to integrate Ruby into my professional life and make it part of the toolbox. It was cool to play with Ruby in my spare time, but I wanted to use it on projects whose main development language/platform was mainly Java. Use it as scripting, glue language. Use it as toolkit language to e.g. generate test data, access databases, convert files, build projects and maybe even build a piece of Web applications (admin apps for example).

It never worked. The main problem was availability of the Ruby platform in all environments. While JVM was there by default, Ruby had to be installed and sometimes compiled for the more exotic platforms. And that can be a big deal if you have not full control over the environment – scenario which is pretty much guaranteed in enterprise environment. It is hard to argue with the sysadmin saying “You want to install that in production just to run scripts ? Why do not you use Perl or Bash or Java that are already there ?”

For a little while I thought that JRuby may be the way. After all – all you need is JVM and JRuby is just another JAR, right ? As Goethe said, grey is all theory and green is the tree of life :-). A language is as good and useful as are components and libraries available. One certainly does not want to write everything from scratch. Libraries in Ruby are Gems and Ruby provides very nice, mature and IMHO superior system for component management to Java JAR’s – because it handles different versions of same Gem very well (maybe some day there will be Gem hell after DLL hell and JAR hell 😉 ). Unfortunately, some Gems (by Murphy’s law most of the really interested ones) are for performance reasons built as thin Ruby layer around native (written in C) library. And JRuby does support that, making most of the Gems unavailable.

Even if JRuby had all the gems available, there would still be a problem that the Gem system and Jar system are different and do not quite fit together. Also, from language point of view you certainly can use Java objects in JRuby and vice versa, but doing that makes you feel slightly schizophrenic – what reality am I in? Is this a Java Java class or Ruby Java class ?

Third problem that I have encountered after coming up with some Web App in Rails is that the deployment model is very different from Java deployment model which myself and people in organizations we work with understand really well. We know how to deploy so that it scales, we know how to monitor and maintain a Java enterprise app. But not a Rails app with all those Mongrels, lighttd’s and other creatures :-). This leaves many open questions like “How do we size hardware for expected load ?” for which I do not have answers, and judging by well publicized issues with Rails apps scalability, even the best and brightest in Ruby world may not have either – or at least some people say so.

About at the same time I discovered Ruby, I also become aware of the strange Java dialect called Groovy. It sort of tried to do the same thing I hoped to use Ruby for, only from firm within Java environment. The original reason I did not want to look deeper at Groovy was that compared to straight elegance of Ruby, it looked kind of ugly. The Java skeleton was sticking out in wrong places and alltogether it just did not feel as good as Ruby.

I have to publicly admit I was wrong.

Being a Mac user, I have license for going after good looks and white shiny objects, but when it comes to programming languages, the good looks may just not be enough. The reality is the proof.

During last 12 months, we have quietly and very successfully used Groovy components and pieces on three large projects. It fitted perfectly, never running into anyof the issues above.

Through these projects, I learned to appreciate the Groovy way, my sense of aesthetics stopped to be offended by certain syntax constructs in Groovy and I even started to like them better than Ruby ones. For example, I am now convinced that Groovy categories are safer and better approach that explicitly alerts programmer about using class extension, than re-opening any class in Ruby (which is still possible in Groovy by assigning closure to member in metaclass). Imagine how confusing it can be for software maintenance when reopening and using happens far apart in the source code.

But the most important, the painful realization ” how the heck do I do the XYZ thing in this language ? If I only were coding in Java, it would be so much simpler ” is history with Groovy. Everything that I was used to use in last 12+ years in Java is still there, all the goodies of Jakarta Commons and way more.

Groovy community seems to be less opinionated, less self-righteous than Rails/Ruby community and more understanding for weird requirements and idiosyncrasies of enterprise environments. Rather than telling you “you should not want to do this” and “DHH thinks it is wrong”, you actually may get a helpful pointer to useful website or blog how to do that stupid thing in Groovy or Java or combination of both. Because you know, when one needs to accomplish something that seems to be wrong and illogical, being told that it is wrong and you should better forget about it does not really help. People who worked with real enterprise system’s integration understand, that cost of touching or changing certain systems is so prohibitive that it is out of question and doing the technically wrong thing may right (and only) option for given situation and customer.

Therefore – bye bye Ruby, Hello Groovy. Next things to embrace and embed will be Grails.

It's alive !

2009/02/18

I was aching to blog about this since December 18th, when our system quietly and gently slipped into public visibility. Marked as Beta (thanks, Google for making this a legitimate way how to go live) it comfortably made it through Christmas shopping season into 2009.

Now when the site has been announced and mentioned few times in the media, I guess it’s OK to mention it here too.

What is “it” ? A new, fresh eCommerce site selling music. A lots of really good music. Without any DRM or any other nonsense, as plain good high quality (mostly 320 kps) MP3’s. The selection is actually very good – starting from several hundred thousands in December to several few million songs when full catalog is loaded. More great music being added every week.

The design of the site is pretty, modern, leveraging lot of jQuery and flash magic. In the backend powered by the probably most powerful eCommerce platform – ATG eCommerce Suite.

We go back long time with ATG. Starting in 2001 when we (we means Montage at that time) decided to bid solution based on ATG for two major RFP’s in federal government and won them both. During following years, we have digged deeper into very rich and powerful platform and built more functionality and added few more customers. This project was our first full eCommerce implementation based on ATG 2007.1. But definitely not the last one – it looks like despite the economy maladies, demand for eCommerce and specially ATG based eCommerce solution is surprisingly large and interesting amount of work is coming our way …

I am tremendously proud of what our team was able to deliver. It took lots of dedication and sweat: we had pretty aggressive deadline (full store was implemented in under two months) and complexities of the environment. To make me even more proud is that the system is running in production environment architected and developed by our team. It is not often that developers team has opportunity to be involved into complexities of the large enterprise system deployment and putting to production.

Ah, almost forgot – the URL is http://www.zik.ca/. See for yourself. Right now, only in French (primary target audience is French speaking Canadian population), English coming later.

picture-2

I would like to thank everybody who helped to make this an amazing project experience. We were lucky to build great relationship with both our customer and the end-customer, as well as with our development partners in Montreal working on different stores in Virtual Shopping Mall.

Thanks to ATG for such rich and powerful product suite. It is like great sports car: very powerful, requires skills to master it, but once you get it, you can do amazing things with it.

And last but not least, big thanks to everybody from our delivery team that made this possible. You guys rock.

Btw, if you would like to work with people that can build things like this, and have either ATG experience or at least solid J2EE, Spring, Hibernate and JSP/JSTL skills, send your resume to careers at thinknostic dot com.
We are hiring again :-).

No telecommuters please – you must be able to live and work either in Ottawa or Toronto. Speaking French is not required but is definitely a plus.

After the Spring Storm

2008/09/25

Few days ago, I have received an email asking to attend the SpringSource conference and offering $300 discount if I decide to buy before end of month. The timining was pretty ironical, considering the recent contraversy about SpringSource Entreprise Maintenance policy change. The was a heated discussion on TheServerside.com and even FAQ was compiled by Rod Johnson to explain the whole thing.

So what is the issue: SpringSource, commercial entity behind community driven Spring Framework decided to capitalize on the investment and push the community toward paid service suport contract. Whether or not it was related to funding round and VC’s entering the game, is not important. Unfortunately, the terms were not really well explained and SpringSource did initially very poor job explaining and communicating the actual change. The community overreacted and lot of people started to call for code fork, project Summer or similar – or switching to different framework.

What the change really means is that maintenance releases will not be available (as binary, official downloads) after 3 months, unless you pay for the support. It means that using new policy after let’s say release 2.0, only for three months everybody could download binary distributions of 2.0.1, 2.0.3 etc. If at the end of month 3 you would need fixes in the 2.0 branch, you would be on your own. The source code repository with 2.0 branch would still be available, with fixes committed (see later) but without tags (or labels) and certainly without pre-packaged download of 2.0.4, 2.0.5. You could compile the actual head of the 2.0 branch yourself, but it would not be clear what version you actually run.

This is actually not such terrible limitation as it may seem. Maintaining old code base and backporting the changes from trunk into fixes in branches is lot of work, requires testing and so on. SpringSource did invest a lot into the codebase and as commercial entity provides service to the users. The release management and software maintenance is slightly different type of work that developing new great release, and I can imagine that there are even less contributors and volunteers for this (seldom appreciated) kind of work. Multiply that with many branches and you’ll see the magnitude of the problem.

What did SpringSource do wrong, in my opinion is that they are taking away something that was available for the community for free and that created lot of negative sentiment. There are two ways how to push people to pay for free service – and cutting unless you pay is wrong one. Right way is to offer more on top of what is available for free, something that has value specifically for enterprise clients. There are several areas where there is a need: more and much better samples, template applications, guides, tutorials. Their availability has huge value for customers in saving development time and better quality of the products using the Spring platform.

There are several technical subtleties that are in open: the FAQ says that the changes and fixes from trunk will be available in the maintenance branches, but does not say when. Delaying changes can be powerful argument … Lack of tags in public repository also implies existence of private repository with these tags – how else could the SpringSource build the “supported releases”. The existence of two repositories in project (unless you use distributed VCS, of course) is often starting point of splitting project to payware and limited “community edition”. I hope this will not happen to Spring.

The most unfortunate impact of the maintenance policy decision may be for the Spring based opensource projects, that helped a lot to establish acceptance of the platforms over last few years. Dilema of facing either cascading changes of permanent upgrades just to keep in sync with latest trunk version, or permanently monitoring the branch to see which of the changes are important – in absence of “official” maintenance release with defined bug fixes etc can be an issue for many projects. I hope SpringSource will find workarounds and solutions for these.

Despite of that, forking Spring would be very bad idea. The very last thing enterprise Java needs is another framework that is esentially same or very similar than existing one. SpringSource employs brilliant engineers, Spring community also has great contributors – what is the point of splitting the forces when there is no architectural disagreement, the issue is “just money” ?

Forking also does not solve the core problem: who will be providing the community service of maintaining and building the old releases. The maintainers of Summer codebase (or whatever the name of Spring forked successor would be) will face the same dilema – choose between working for free forever or finding way how to employ enough manpower to keep the balls rolling.

I would recommend that instead of forking Spring codebase, whoever feels like it, try instead to fill in the gap and provide the “inofficial official” builds, a “community maintenance releases” as counterpart and alternative to official SpringSource maintenance releases for enterprise (== paying) customers. This would actually help the project and all projects that depend on it much more than asking them to switch.

It would also not hurt SpringSource – I am convinced that they would loose very little revenue (if any) because of that. It has been done (Centos vs RHEL). If my project would be based on Spring platform and not on commercial Java server (which it is not) and the support cost would be reasonable (which it IMHO is), I would happily pay for priviledge to have access to the people of Juergen Holler’s caliber (and many others).

Let’s face it, folks: when it comes to support and software maintenance and similar less attractive areas of our profession, there is definitely no such thing as free lunch …

Trying out Groovy with Oracle database

2008/04/07

There is a saying that necessity is the mother of invention. Such necessity happened last week and forced me to try out the Groovy language.

The trigger was need for creating good data set for testing changes in a full text search. I had to locate few hundred of obsolete technical documents to be used as test data, dump the metadata as well as the BLOB data to disc, create INSERT sql scripts and loader that would from Ant pre-load the database with the test set and insert the BLOB’s. It is fairly simple task, the issue were boundary conditions:

Database in question is Oracle. I have no OCI8 installed on my MBP and refuse to install Oracle on OS-X – it was little fun getting it up and running on Linux. Usually I run Oracle in Windows or Linux VM and with lack of OCI8, only option how to communicate with it is Level 4 JDBC driver.

Because of missing OCI8, Ruby was out of question (the gems for Oracle access needs OCI). So was PL/SQL, for the reason of administator priviledges. I am not exactly a PL/SQL programmer, but know enough to write stored proc that loads or unloads BLOB, assumed that UTL_FILE and DBMS_LOB packages are available and accessible. Which was not the case.

Before falling to default and writing it all in Java (which would work, but take certainly longer than I was willing to invest), I decided try out this Groovy thing :-). My exposure to the language was minimal – I never programmed in it (not counting 2 hours spent browsing Ruby in Action), but I hoped that similarity with Java and almighty Google will help me out …

Connecting to database in Groovy is simple and elegant:


import groovy.sql.Sql

db = Sql.newInstance(
            'jdbc:oracle:thin:@myoraclehost:1521:XE',
            'USER', 'PWD', 'oracle.jdbc.driver.OracleDriver')

      def sql = """
            select d1.document_id, d1.version, d1.filename from document d1
            where d1.document_id in ...

           .... REST OF SQL DELETED ....
        """
        list = db.rows(sql)

The result is hash-like structure that can be used to drive both generating the DELETE and INSERT SQL statements into text file as well as retrieving the BLOB’s and saving them to HDD:


    def writeBlobToFile(db, docid, version, filename, reallyWrite=false) {
        def sqlString = "SELECT d.DOCUMENT_BODY FROM document d where d.document_id = '$docid' and version = $version"
        def row = db.firstRow(sqlString)
        def blob = (oracle.sql.BLOB)row[0]
        def byte_stream = blob.getBinaryStream()
        if( byte_stream == null ) {  println "Error for ${docid} : ${version}"  }

        int total = blob.length();

    // Write to a file
       if (reallyWrite) {
            byte[] byte_array = new byte[total]
            int bytes_read = byte_stream.read(byte_array)
           def fullname =     "/Users/miro/tmp/BLOBS/$filename"
           def fos= new FileOutputStream(fullname)
           fos.write(byte_array);
           fos.close()
       }
        println "Document $docid:$version, file: $filename, size $total"
        return total
    }

Loading the documents and metadata from disk to database is done by Ant script


  
    
  



    
        
        

    



    
        
        
        

    

    
        
        
    


The Groovy task must be defined before used (with proper classpath). The groovy
task shows passing arguments and executing script. The most important part of
the script (loading binary file and inserting the BLOB) is here:


// each line in METADATA.TXT looks like this:
reg =  /document_id: \'(\w+)\', version: (\d+), filename: \'(.+)\', size: (\d+)/

db = Sql.newInstance(
        'jdbc:oracle:thin:@myoraclehost:1521:XE',
        'USER', 'PWD', 'oracle.jdbc.driver.OracleDriver')
counter = 0
new File(args[1]).eachLine { line -&gt;
    line.eachMatch(reg) { match -&gt;
        try {
            str = "${args[0]}/${match[3]}"
            println "Opening file: ${str}"
            FileInputStream fis = new FileInputStream(str)
            int size = fis.available()
            byte[] data = new byte[size]
            fis.read(data)

            // get the BLOB
            row = db.firstRow("select document_body from document where document_id = ? and version = ? for update",
                    [match[1], match[2]])
            my_blob = (oracle.sql.BLOB)row[0]
            if( my_blob == null ) println "my_blob is null!"
            outstream = my_blob.getBinaryOutputStream();
            outstream.write(data);
            outstream.close()
            fis.close()
            counter = counter+1
        } catch (Exception e) {
            print "Exception: ${e}\n"
    }
}
println "Processed files: ${counter}"

The loader is controlled by the file METADATA.TXT that contains information linking document metadata in database (inserted by SQL statement) with file on disk. This indirect way allows easily “inject” document body with required search phrases, reload database and run tests.

The whole experience was quite pleasant and considering how little I knew about Groovy, it took very little time to create something useful. I wish I could have done some thing in Ruby (which I still like better) – but I was amazed how powerful the combination of Groovy + Ant can be.

The most valuable feature is no need to add anything really new or exotic to Java environment – all you need is Jar file and Eclipse plugin, no new/different library packaging scheme – Jars are still jars, not gems.

The only minor hickup and weird thing I found was passing arguments between Ant and Groovy script. It looks like that the groovy script should be a script and not use classes + static main method (which would be natural instinct for Java developer). Thanks to Christopher Judd for the hint.

Showstopper issue with JRuby ?

2008/03/16

After some break, I dusted off the Ruby to try out one interesting idea Peter presented yesterday that literally ASKS for Rails. So I grabbed the latest and greatest RubyNetBeans from Ruby Hudson. For some reason it stopped updates on January 26th so the latest version I have got was build 6327. It is bundled with JRuby 1.1RC1 and the Rails version that comes preinstalled is 1.2.6. Using the menu Tools -> Ruby Gems, they can be easily upgraded to latest and greatest 2.0.2.

The trouble begins when you want to install database connectivity gems such as sqlite3-ruby. The installer fails with the message:


trying to install

INFO:  `gem install -y` is now default and will be removed
INFO:  use --ignore-dependencies to install only the gems you list
Building native extensions.  This could take a while...
extconf.rb:1: no such file to load -- mkmf (LoadError)
ERROR:  Error installing sqlite3-ruby:
    ERROR: Failed to build gem native extension.

/Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby</a> extconf.rb install sqlite3-ruby --no-rdoc --no-ri --include-dependencies --version &gt; 0

Gem files will remain installed in /Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1</a> for inspection.
Results logged to /Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out</a>

The missing file mkmf.rb is indeed missing from the JRuby distribution. This is entered as a bug in JIRA 1306 with resolution ‘Won’t fix’. Tough luck.

I understand the reasons and motivation for this decision – the JRuby team decided not to support native extensions in Gems, to keep the platform Java only. I also understand that in this particular case, there are workarounds – using ActiveRecord-DBC gem and JDBC drivers for the database will most likely work. Unfortunately, this decision makes choice of JRuby as platform very questionable.

I really liked JRuby for the comfort of having platform that is portable and safely wrapped within boundaries of the good old trusted JVM. I feel much more comfortable maintaining possibly several different versions of JRuby and corresponding Gems sets than maintaining same several configurations on the OS level and sudo-ing just to install Gems. I was more that happy to trade the lack of speed for this security.

The two main attractions of Ruby (from my point of view) are elegant, powerful language with beautiful syntax as well as sheer amount of code available as Gems to be reused. With the bug 1306, many of this code may not be available for JRuby – unless Gem authors make specific provisions for Java version of the Gem. I cannot think how this is a good idea and certainly not a good news for the future of the language.

One way out is use native Ruby interpreter, of course and make sure you do not mess up your installation by trying out new things. This does not allow the easy way into enterprise that JRuby was promising – by being basically ‘just another jar’ and running on Tomcat.
The other way out is to reconsider the Groovy. I still do not enjoy the syntax anywhere close to Ruby, but every Groovy class is Java class, there is no need for artificial bridges. It has own clone of Rails – GRails – that seems to provide lots of Rails magic and goodies and is based on Spring which I am very familiar and quite happy with. I still do not know whether the amount of “gems” in Groovy world is in the same league as Ruby (which is still limited compared to Python or even CPAN Perl bonanza) – but as long as I can find that what I need, it may be just enough.

For now, I will revert back to Ruby (no time to start learning Groovy+Grails), but I definitely will look at it later on.