Progriff

BatchJHaml Update

2012-02-14T00:00:00+11:00

BatchJHaml Update

14 February 2012

I've updated BatchJHaml to support sub directories. If you set the last argument to true, it'll generate the Haml file in subdirectories as well. Do note that the layout folder will be omitted.

Here is a sample from my build.xml

<target name="run">
  <java fork="true" 
    classname="com.progriff.jhaml.BatchJHaml" 
    outputproperty="javaoutput">
    <classpath>
      <path refid="classpath"/>
      <path location="${dist}/BatchJHaml.jar"/>
    </classpath>
    <arg value="${haml.path}"/>
    <arg value="${haml.layout.path}"/>
    <arg value="${haml.output.path}"/>
    <arg value="${haml.output.extension}"/>
    <arg value="${haml.javascript.path}"/>
    <arg value="${haml.stylesheet.path}"/>
    <arg value="${haml.recursive}" />
    </java><echo message="${javaoutput}" />
</target>

For example if you have the following in your haml folder,

haml
  |-- someDir
        `- baz.haml
  `foo.haml
  `- bar.haml

The resulting jsp folder will look like this

jsp
  |-- someDir
        `- baz.jsp
  `foo.jsp
  `- bar.jsp

Resin 3.1 and Azure SDK for Java

2012-02-08T00:00:00+11:00

Resin 3.1 and Azure SDK for Java

08 February 2012

I was writing a REST service that will send a Cloud Queue Message to Azure. After testing my utility class with JUnit, it passed. However, when I tried to send a Cloud Queue Message via REST service (hosted in Resin 3.1), I got the following error:

java.lang.IllegalArgumentException: property 
  "javax.xml.stream.isReplacingEntityReferences" not supported
at com.caucho.xml.stream.XMLInputFactoryImpl.setProperty
  (XMLInputFactoryImpl.java:265)
at com.microsoft.windowsazure.services.core.storage.utils
  .Utility.createXMLStreamReaderFromStream(Utility.java:321)

Resin Web Server uses it's own XMLInputFactory implementation called com.caucho.xml.stream.XMLInputFactoryImpl. According to this article, you can override the implementation by having <system-property/> following in your resin.conf. Put the following to use the JDK's version of XMLInputFactoryImpl

<system-property javax.xml.stream.XMLInputFactory=
  "com.sun.xml.internal.stream.XMLInputFactoryImpl" />

Problem solved? Not really. Tried running the REST service again and I got the following:

com.microsoft.windowsazure.services.core.storage.StorageException: 
  XML specified is not syntactically valid.

After looking at the Azure SDK for Java's source, I've found out that the XMLOutputFactory and XMLStreamWriter was unable to generate the XML request body in com.microsoft.windowsazure.services.queue.client .QueueRequest.generateMessageRequestBody

Why? If you do a simple system out on the class name, resin is using it's own implementation for XMLOutputFactory and XMLStreamWriter. So add the following as well into your resin.conf

<system-property javax.xml.stream.XMLOutputFactory=
  "com.sun.xml.internal.stream.XMLOutputFactoryImpl" />
<system-property javax.xml.stream.XMLStreamWriter=
  "com.sun.xml.internal.stream.writers.XMLStreamWriterImpl" />

tl;dr? Add the following in your resin.conf

<system-property javax.xml.stream.XMLInputFactory=
  "com.sun.xml.internal.stream.XMLInputFactoryImpl" />
<system-property javax.xml.stream.XMLOutputFactory=
  "com.sun.xml.internal.stream.XMLOutputFactoryImpl" />
<system-property javax.xml.stream.XMLStreamWriter=
  "com.sun.xml.internal.stream.writers.XMLStreamWriterImpl" />

After that, my code works perfectly. I'm sure there are other weird problems as well, I'll keep you posted if I've found anymore weird stuff.

Hibernate won't update my object!

2012-01-06T00:00:00+11:00

Hibernate won't update my object!

06 January 2012

Yesterday, I was given a Hibernate related bug to fix. We have an object, say object A which has a list of object B. This list of object B are stored as a JSON string in table A.

So here's the problem, after calling a session.update on object A with the updated object B, the change was not persisted into the database. After hours of Googling and some help from Michael who is a fellow colleague of mine, I found the problem.

According to this article, Hibernate will only do a UPDATE statement after session.flush(). During this operation, Hibernate will compare the original object and the object to be updated.

This article explains how Hibernate compares the object.

The fix? In your Hibernate user type, override the deepCopy and return another instance of list.

@Override
public Object deepCopy(Object value) throws HibernateException
{

  // if value is null, return null
  if(value == null)
  {
    return null;
  }

  // convert the value to a List first
  List<ObjectB> objectBList = (List<ObjectB>)value;

  // create a new Arraylist and add all the objects
  List<ObjectB> newObjectBList = new ArrayList<ObjectB>();
  for(ObjectB objectB : objectBList)
  {
    newObjectBList.add(objectB);
  }
  return newObjectBList;
}

Normalize your video with FFmpeg

2011-10-22T00:00:00+11:00

Normalize your video with FFmpeg

22 October 2011

It's been a while since I've updated my blog. I didn't encounter any weird things until last week.

At Movideo, we've been haunted by videos that goes out of sync with it's audio. The symptom goes something like this:

Plays a dynamic bitrate video that was encoded using Movideo's Encoder application.
In the beginning, audio and video is in sync.
A crackling noise appeared and then the audio and video goes out of sync.

We've decided to use MEncoder to normalize the original video which is in .mov format.

The command

mencoder -ovc copy -af volnorm=1 -oac libmp3lame input.mov -o output.mov

creates a mov file which Quick Time won't recognize at all. Tried viewing it with Windows Media Player and there's audio but no video. After that, I upload the video via Movideo's Admin interface. The re-encoded video still goes out of sync.

The solution to this is to re-encode the audio with FFmpeg with

ffmpeg -i input.mov -vcodec copy -acodec libmp3lame output.mov

Now, output.mov can be viewed by Quick Time and works fine on Windows Media Player. Then, I tried uploading the output.mov via Movideo's Admin interface and the video doesn't go out of sync anymore.

Generating JSPs With BatchJHaml

2011-07-10T00:00:00+10:00

Generating JSPs With BatchJHaml

10 July 2011

Last year, I was assigned to build a web app using Struts. I wanted a Haml interpreter and found JHaml. The problem is it only converts Haml strings to HTML. I need something to convert .haml files to .html or .jsp files. Why should you use Haml? See here.

Due to this limitation, I created BatchJHaml. If you're using BatchJHaml as a standalone app, it's pretty straight forward. But what if you want to include BatchJHaml into your Ant build script? Here's what you need to do.

1.) Run git clone git@github.com:pugnusferreus/batchjhaml.git in your terminal

2.) cd to BatchJHaml

3.) Run ant and BatchJHaml.jar will appear in the dist directory.

4.) Copy and paste BatchJHaml.jar in your local lib directory.

5.) Copy the following into your local lib directory as well. You can find them in the BatchJHaml/lib folder. If you already have the following, you can ommit this step. If you have a later version of the following jars, you can use them as well.

commons-io-1.4.jar
commons-lang-2.5.jar
guava-r06.jar
jhaml-0.1.2.jar
markdownj-1.0.2b4-0.3.0.jar

6.) Copy the following build target into your build.xml

<target name="compile-haml">
  <echo message="Converting haml files into jsp ..." />
  <java fork="true" classname="com.progriff.jhaml.BatchJHaml">
    <classpath>
      <fileset dir="${library.home}">
        <include name="**/commons-io-*.jar" />
        <include name="**/commons-lang-*.jar" />
        <include name="**/guava-*.jar" />
        <include name="**/jhaml-*.jar" />
        <include name="**/markdownj-*.jar" />
      </fileset>
      <path location="${library.home}/BatchJHaml.jar"/>
    </classpath>
    <arg value="~/YourProject/haml"/>
    <arg value="~/YourProject/haml/layouts"/>
    <arg value="~/YourProject/jsp"/>
    <arg value="jsp"/>
    <arg value="~/YourProject/javascripts"/>
    <arg value="~/YourProject/stylesheets"/>
  </java>
</target>

7.) In your build target, add compile-haml to your depends. Example:

<target name="build" depends="clean, prepare,compile, compile-haml">

And enjoy your Haml.

Modernizing Lunchpicker Part 2

2011-07-09T00:00:00+10:00

Modernizing Lunchpicker Part 2

9 July 2011

When I wrote lunchpicker, I was obviously not yet enlightened with TDD. While migrating lunchpicker from Rails 2 to Rails 3, I took the opportunity to learn RSpec. So again, here is my journey with RSpec. Since lunchpicker is a Rails app, I'll be using rspec-rails

1.) Some stuff from the Sass plugin has been deprecated in Rails3. Remove the plugin by removing the directory in vendor/plugins and add https://github.com/jasoncodes/hassle.git in your Gemfile.

2.) Move filter_parameter_logging in application_controller.rb to /config/application.rb

3.) Remove the test folder because we'll be using Rspec

4.) Add the following lines into your Gemfile. These gems should be loaded in Dev and Testing environments only.

group :development, :test do
  gem 'rspec-rails'
  gem 'spork'
  gem 'awesome_print', :require => 'ap'
  gem 'mocha'
  gem 'shoulda'
  gem 'vcr'
  gem 'webmock', :require => false
end

5.) Run bundle install and then run rails g rspec:install. This will create the rspec folder and the rake file to create a test database.

6.) The structure of your spec/ directory should mirror that of app/. For example, the spec file for app/models/location.rb would be spec/models/location_spec.rb.

7.) Create your rspec test! Need example? See here.

8.) Now, in the lunchpicker directory, type bundle exec rspec spec. This will run the tests. Since we're testing the controllers and models, rspec would need to bootstrap all the Rails stuff. The result? Slowness.

9.) To overcome this, we'll be using spork to bootstrap the Rails libraries. Open another terminal window and cd to the lunchpicker folder. Type in bundle exec spork rspec.

10.) Open another window and cd into the lunchpicker folder. Type in bundle exec rspec --drb spec. Notice that the tests run faster. If you change any Rails related config, remember to restart spork.

What's VCR? VCR records your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests. We're interacting with Google Weather which is an external API.

ps. lunchpicker is now Rails 3

Modernizing Lunchpicker

2011-06-26T00:00:00+10:00

Modernizing Lunchpicker

26 June 2011

Lunchpicker is my first project with Rails. Back then, there was alot of hype regarding Rails and that was why I started Lunchpicker to have a taste of Ruby On Rails. Yes, I fell in love with it immediately.

Wife had some plans for the entire afternoon and I've decided to dedicate my entire Saturday afternoon on migrating Lunchpicker from Rails 2 to Rails 3. Here's a blog post on my adventure. Thanks @jasoncodes for helping!

1 . Install RVM. RVM allows you to install, manage and work with multiple ruby environments. For example, you can have project A running in Ruby 1.8.x and project B running in Ruby 1.9.x

2 . For lunchpicker, here is my .rvmrc file

    $ cat .rvmrc
    rvm --create 1.9.2@lunchpicker

3 . Now, we need to intall Bundler. Bundler manages your application's dependencies via Gemfile. Here's a sample of lunchpicker's Gemfile

source 'http://rubygems.org'

gem 'rails', '3.0.9'
gem 'google_weather', :git => 'http://github.com/Ennova/google-weather.git'
gem 'informal'
gem 'haml'
gem 'httparty'
gem 'authlogic', :git => 'http://github.com/radar/authlogic.git'
gem 'pg', :require => 'pg'
gem 'sass'
gem 'dynamic_form'

4 . After creating a Gemfile, run gem install bundler and then run bundle intall to install all the dependencies in your rvm.

5 . Go into the lunchpicker directory and run rails new .. It'll obviously overwrite your old configurations with Rails 3's. Replace all the files.

6 . Do a diff and merge all your codes (controllers, models, js, css, haml etc.)

7 . Remove config/initializers/new_rails_defaults.rb

8 . Remove scaffold related html from the public folder

9 . In your helpers, mark your HTML fragements as safe. For example you should change

return <<-HTML
  <p>
    blah
  </p>
HTML

return <<-HTML.html_safe
  <p>
    blah
  </p>
HTML

10 . If you have a non ActiveRecord model, and if you're using Validatable, change it to Informal . For Lunchpicker, search.rb is not an ActiveRecord model. Also, we need to change

validates_presence_of :sheltered

validates_inclusion_of :sheltered, :in => [true, false]

We're using validate inclusion rather than presence for booleans. This is due to the way Object#blank? handles boolean values. false.blank? # => true

11 . Add the following lines to config/application.rb. This will prefix the tablename infront of the PK. eg. venue_id

config.active_record.primary_key_prefix_type = :table_name_with_underscore
config.active_record.schema_format = :sql

12 . default.html.haml has been changed to application.html.haml in Rails 3. Run git mv default.html.haml application.html.haml and don't forget to remove the application.html.erb as well.

13 . Switch rails.js from Prototype to jQuery. Remove controls.js, dragdrop.js, effect.js and prototype.js. Please see rails.js.

14 . Add rails.js in the application.html.haml for Unobtrusive Javascript support.

= javascript_include_tag 'rails'

15 . If you run rake db:migrate before step 11, run rake db:reset and then rake db:migrate again.

I hope that this will be useful to you if you need to migrate any old Rails 2 application to Rails 3. I'll use this opportunity to add unit test with RSpec.

The Rails 3 version of lunch picker won't be in production till unit test is done!

Mocking with Mockito

2011-06-18T00:00:00+10:00

Mocking with Mockito

18 June 2011

At movideo, we're using Mockito as our mocking framework. Mockito provides a clean and simple API to mock your Java objects.

Let's say Class1 has a couple of arguments for it's constructor and you do not want to provide each arguments, you can simply do:

import static org.mockito.Mockito.mock;

then, you do

Class1 class1 = mock(Class1.class);

If you want to mock a certain method call in class1, do the following:

import static org.mockito.Mockito.when;
import static org.junit.Assert.assertEquals;

then, you do

when(class1.methodReturnsString()).thenReturn("foo");
assertEquals("foo", class2.callClass1Method());

Assuming that

 class2.callClass1Method()

will call

 class1.methodReturnsString()

the assertion will be successful.

To get a better picture of what Mockito can do for you, you can checkout my Mockito Test project from my Github Sample Project.

I'll assume that you have Git, Java 1.6 and Ant installed on your machine.

git clone https://github.com/pugnusferreus/mockito_test
cd mockito_test
ant
Open docs/unitTest/index.html in your browser

You can see that all the unit tests pass.

Thanks @cstrzadala for introducing Mockito to all of us!

Unit Test Your Java DAO With Hibernate And HSQL DB

2011-06-05T00:00:00+10:00

Unit Test Your Java DAO With Hibernate And HSQL DB

05 June 2011

Unlike Rails which allow you to do unit test on your models, there's no easy way to actually unit test your Java DAO. Sometimes, we do not have access to the development database or we do not want to polute the development database with our data.

Sure, we could run a local db on our machine. What happen if it's a huge RDBMS like Oracle or DB2? Yes, the previous sentence sounds enterprisey but as a Java developer, you can't get away with enterprisey things.

Take a look at this sample project. Assuming that your current project is using Hibernate as your ORM, you can use HSQL DB to load up a "in memory" database for your unit test. Hibernate and hsql will automatically creates the table for you.

Here's how. I'll assume that you have Git, Java 1.6 (or whatever it's called now) and Ant installed on your machine.

git clone https://github.com/pugnusferreus/dao_unit_test
cd dao_unit_test
ant
Open docs/unitTest/index.html in your browser

You can see that all the unit tests pass.

Make sure that your hibernate config points to the HSQL DB and your model is mapped. Open up "com.progriff.dao.UserDaoTest" and you can see that it's your typical Java Unit Test

Things to do before you start programming

2011-05-23T00:00:00+10:00

Things to do before you start programming

23 May 2011

I've been reading Pragmatic Thinking & Learning by Andy Hunt. In one of the chapters, the author talks about UML. Yes, even I myself find that UML can get a bit boring. He suggests that I literally draw out the diagram. And so I did. The author also suggests that you can do the same with Lego bricks or by role playing, but I won't go into that.

At movideo, I've been assigned to make a program that runs on the server and upload videos to YouTube. In the future we might externally syndicate our videos to another provider (eg. Vimeo). So, I come up with this diagram to see how it can aid me with my software development process.

This diagram actually helped me by:

Acting as a check list
Making Test-driven Development(TDD) easier. Why? By looking at this diagram, you'll be able to see the hollistic picture. By having a hollistic picture of the entire application that you're about to write, it's easier for you to write unit test for it's expected behaviour
Writing pseudo codes. I don't actually write pseudocode à la COBOL era, but for every function that I write, I'll write comments of the things that I need to do.For example:

    public void create()
    {
      // do validation
      
      // check if record exists
      
      // save to database   
    }

Most importantly, before you start scribbling, get away from your computer. Computers can be distracting at times (emails, IM, Twitter). You'll be able to see a hollistic view without those distractions.

How to install jekyll

2011-05-18T00:00:00+10:00

How to install jekyll

18 May 2011

This is my first blog post entry for progriff.com. Since this blog was made with Jekyll, I'll walk you through how to setup a basic blog with Jekyll. Why Jekyll? Here are a couple of points I could think of :

It generates static pages for you.
You do not need a database in your server
You do not need a webserver (Apache, IIS, etc ...)
Without all those mumbo jumbo on your server, your server will be more secure

Here's how:

1 . First, install Jekyll

    $ sudo gem install jekyll

2 . Then install rdiscount

    $ sudo gem install rdiscount

3 . Make a directory. Let's call it blog.

    $ mkdir blog
    $ cd blog

4 . Create the directory structure as of below.

      blog
        |-- _layouts
                `-- default.html
                `-- post.html
        |-- _posts
           `-- 2011-05-18-foo-bar.md
        |-- _site
        |-- css
        `-- index.html
        `-- _config.yml

5 . This is optional. If you need code syntax highlighting, you'll need Pygment.

    $ sudo easy_install Pygments
    $ pygmentize -S default -f html > /path/to/your/blog/css/pygment.css

6 . Take a look at the source for this blog and copy whatever you need.

7 . After you're done, you want to preview your blog. In your terminal, type

    $ jekyll --server --auto

8 . Go to your browser and go to "http://localhost:4000"

9 . If you need to compile the pages without starting the server, just type "jekyll"