Sunday, May 29, 2011

set editor for crontab solaris 555

I was trying to use "crontab -e" under Solaris and I was getting back a number "555". This is related to EDITOR variable missing. Here is how it gets solved (Using bash shell)
$ vi ~/.bash_profile
export EDITOR=/usr/bin/vi
$ source ~/.bash_profile
$ crontab -e

Sunday, May 22, 2011

No message found under code '...' for locale

This error can occur even if the code is in the resources file for example when rendering error pages resulting from Tomcat internal errors (500) like JSP errors. We better configure Spring to use the Reloadable Resource Message Source:
<bean id="messageSource"
         class="org.springframework.context.support.ReloadableResourceBundleMessageSource">
         <!--  <property name="basename" value="classpath:messages" />  -->
         <property name="basenames">
             <value>/WEB-INF/i18n/messages</value>
         </property> 
         <property name="cacheSeconds">
             <value>60</value>
         </property>
         <property name="fallbackToSystemLocale" value="false" />
     </bean>

JSP Spring Taglib for your error pages must contain a default value (not very i18n friendly) otherwise when the error pops up the JSP will fail with "No message found under code 'password' for locale 'en_US'"
<spring:message code="password" text="Password"/>

A special case for this issue is when Spring tries to parse an empty code. To avoid the issue in that case just define an empty code line in message.properties. Something like "=" or "=Empty" or "=Null" should do the trick.

Spring Invalid remember-me token cookie theft attack

There are errors that simply sound too scary. From time to time Tomcat was returning the below error message:
org.springframework.security.web.authentication.rememberme.CookieTheftException: Invalid remember-me token (Series/token) mismatch. Implies previous cookie theft attack.

To recreate this issue you can use Firefox "Live Http Headers" plugin to capture sessionId and remember me token cookies once the user is logged in. Then restart Firefox and use "Modify Headers" plugin to force the previous cookie values.

To resolve this issue you can forward the request to the login page passing an error code. Order in web.xml is important, here is a working example:
<error-page>
        <exception-type>org.springframework.security.web.authentication.rememberme.CookieTheftException</exception-type>
        <location>/WEB-INF/jsp/login.jsp?error=sessionExpired</location>
    </error-page>
    <error-page>
        <exception-type>java.lang.Exception</exception-type>
        <location>/WEB-INF/jsp/uncaughtException.jsp</location>
    </error-page>
    <error-page>
        <error-code>500</error-code>
        <location>/WEB-INF/jsp/uncaughtException.jsp</location>
    </error-page>

Friday, May 20, 2011

java.sql.SQLException: Column 'name' not found.

In OSX Tomcat MySQL 5.1 my query was failing from Java with the below error:
java.sql.SQLException: Column 'name' not found.

The query was perfectly running from the command prompt. Developers using Windows and Ubuntu did not have the same issue so I decided to upgrade to latest version both MySQL and the JDBC connector. The issue did not go away.

Problem explained

The below will fail ...
SELECT DISTINCT c.name AS `Client Name`, ...
But this will work:
SELECT DISTINCT c.name, ...
Whether we need the aliases in the query is a good point. Especially for internationalization this is a bad practice but I needed to be sure my MAC could support this :-)

The Solution

Use a flag in the connection string:
SELECT DISTINCT c.name, ...
jdbc.url=jdbc:mysql://localhost:3306/nestorurquiza?useOldAliasMetadataBehavior=true
From MySQL Reference Manual:

"Should the driver use the legacy behavior for "AS" clauses on columns and tables, and only return aliases (if any) for ResultSetMetaData.getColumnName() or ResultSetMetaData.getTableName() rather than the original column/table name? In 5.0.x, the default value was true."

Mysql Got a packet bigger than 'max_allowed_packet' bytes

  1. Edit section [mysqld] in mysql.ini (Windows/Linux) or /etc/my.cnf (OSX):
    [mysqld]
    ...
    max_allowed_packet=100M
    ...
    
  2. Restart MySQL

Thursday, May 19, 2011

Run Sql from Script

If you must run commands from a script and make sure they all run within a transaction Spring JdbcTemplate can help. The below code should be self explanatory and it contains some findings that could save you some time next time you need to do something like this:

package com.nestorurquiza.utils;

import java.io.InputStream;

import javax.sql.DataSource;

import org.apache.commons.dbcp.BasicDataSource;
import org.apache.commons.io.IOUtils;
import org.springframework.core.io.Resource;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.transaction.annotation.Isolation;
import org.springframework.transaction.annotation.Transactional;
import org.sqlite.SQLiteDataSource;

public class SqliteUtil {
    @Transactional(rollbackFor = Exception.class, isolation = Isolation.READ_UNCOMMITTED)
    public static void runSqlFromScriptResource(Resource resource, JdbcTemplate jdbcTemplate) {
        try {
            //The below is not recommended for DDL and it is not transactional
            //SimpleJdbcTestUtils.executeSqlScript(jdbcTemplate, resource, continueOnError);
            
            InputStream is = resource.getInputStream();
            String[] sql = IOUtils.toString(is).split(";");
            
            //Trimmed or not trimmed batchUpdate will not work with SQLITE:
            //[SQLITE_MISUSE] Library used incorrectly (not an error); nested
            //exception is java.sql.BatchUpdateException: batch entry 40
            //jdbcTemplate.batchUpdate(sql); 
            //String[] trimmedSql = new String[sql.length]; 
            //for(int i=0; i < sql.length; i++) {trimmedSql[i] = sql[i].trim(); }
            //jdbcTemplate.batchUpdate(trimmedSql);
            
            for (int i = 0; i < sql.length; i++) {
                jdbcTemplate.update(sql[i]);
            }

        } catch (Exception e) {
            String resourcePathFromDataSource = resourcePathFromDataSource(jdbcTemplate.getDataSource());
            throw new RuntimeException("Unable to run sql script against " + resourcePathFromDataSource, e);
        }
    }
    @Transactional(rollbackFor = Exception.class, isolation = Isolation.READ_UNCOMMITTED)
    public static void JdbcTemplateUpdate (JdbcTemplate jdbcTemplate, String query, Object[] params) {
        try {
            jdbcTemplate.update(query, params);
        } catch (Exception e) {
            String resourcePathFromDataSource = SqliteUtil.resourcePathFromDataSource(jdbcTemplate.getDataSource());
            throw new RuntimeException("Query failed for " + resourcePathFromDataSource, e);
        }
    }
    
    public static String resourcePathFromDataSource( DataSource dataSource) {
        String filePath = null;
        String url = null;
        if (dataSource instanceof DriverManagerDataSource) {
            url = ((DriverManagerDataSource) dataSource).getUrl();
        } else if (dataSource instanceof BasicDataSource) {
            url = ((BasicDataSource) dataSource).getUrl();
        } else if (dataSource instanceof  SQLiteDataSource) {
            url = ((SQLiteDataSource) dataSource).getUrl();
        }
        if(url != null) {
            filePath = url.substring(url.indexOf("sqlite:") + 7);
        }
        return filePath;
    }
}

Wednesday, May 11, 2011

sqlite database locked

If you are using sqlite you will notice that locking issues are frequent. While doing some sqlite from Spring JdbcTemplate I noticed that from time to time the table would get locked:
org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [INSERT INTO employee (`first`, `email`) VALUES (?, ?)]; SQL state [null]; error code [0]; database locked; nested exception is java.sql.SQLException: database locked

The statement was run from a loop and sometimes the total of 70 records were inserted but from time to time only a fraction. Only one thread was actually accessing the table but I could only get the issue reproduced when using Jakarta JMeter to simulate several users. This made me think that somehow the driver version (3.7.2) I am using (Xerial SQLite JDBC) has a problem. Further investigation allowed me to conclude that the code was missing the @Transactional annotation and once I added that I could not longer replicate the database locking issue. Furthermore the INSERTs go way faster as well
@Transactional(rollbackFor = Exception.class, isolation = Isolation.READ_UNCOMMITTED)
    public void populateDB(...) {

As you can imagine the reason why this is solved this way is that nothing gets committed up until the last record is inserted avoiding that way a possible locking coming from the underlying "setAutoCommit(false)/setAutoCommit(true)" operations.

Spring abstracts once again a big problem from the developer. Handling this locking issues could take your breath away but with the use of Spring you just rely on an annotation.

There is a second issue I have seen related to the way Data Sources are instantiated. Basically be careful and sure you instantiate the Data Source just once.

Reusing datasources advice might look trivial and someone might say I got that under control as I use JNDI. The problem is when your datasource is to be discovered on the fly.

I have a project where a sqlite database is used per user. Clearly I cannot use JNDI but instantiate the data sources instead. I need the datasource to populate data and at the same time to allow Jasper Reports use that data. In order to accomplish that I instantiate the datasource from a Controller using a Service and I pass the Datasource to my Report Services and Jasper Engine.

Saturday, May 07, 2011

VCD or SVCD from multiple avi mp4 and more

DVDs are great but for small videos it is just a waste of media. SVCD is a good well supported by all modern DVD players in the market.

Making a VCD or SVCD on MAC OSX can be achieved using a combination of ffmpeg, vcdimager and Burn. In Linux you can use k3b instead of Burn.

My son's teacher wanted to give as a present for Mother's day a video of each child to his/her Mom. She had several mp4 and avi files (one per student) and she thought creating a VCD from those short videos was going to be easy so after some attempts using Windows Vista Movie Maker she asked me for help. I told her that I could do that easier in my MAC and then she agreed for me to do the job.

Here are the steps to follow if you run OSX and you want to burn several VCDs, one per media file:
  1. You will need to install ffmpeg and vcdimager. I use MacPorts so:
    $ sudo port install ffmpeg
    $ sudo port install vcdimager
    
  2. Put all AVI, MP4 or any other supported by FFMPEG format in a folder and from that folder run the below command. If you have less than 30 minutes then pick the highest quality (SVCD)
    $ for i in *.*; do ffmpeg -i "$i" -target ntsc-svcd "${i/.*}.mpg"; vcdimager -t svcd -l "${i/.*}" -c ${i/.*}.cue -b ${i/.*}.bin ${i/.*}.mpg; done
    
  3. If you have to burn more than 30 minutes in just one CD you will need to lose quality and then go for VCD
    $ for i in *.*; do ffmpeg -i "$i" -target ntsc-vcd "${i/.*}.mpg"; vcdimager -t vcd2 -l "${i/.*}" -c ${i/.*}.cue -b ${i/.*}.bin ${i/.*}.mpg; done
    
  4. Now you will end up with some mpg, bin and cue files. The cue files contain the information about the (S)VCD image (bin files) which in turn are a packaged version of the mpg files. All you need to do is to burn the bin files and for that in OSX you use the Burn program. Just open it, click on the Copy tab and drop the cue file into the container area. Insert a blank CD-R and hit Burn.

Custom JSP Tags to find if a String is Numeric

JSP scriptlets should be avoided (Even though I confess I have used them for years) especially to avoid Null Pointer Exceptions in JSP.

There are many JSP tag libraries with a lot of functionality already implemented but no matter how much is already there you can always have a scenario where the function simply does not exist in any of the existing tag libraries.

In those cases you need to define a Custom Taglib. This is actually easy and I will demonstrate here how to use a custom taglib to implement a function to find out if a String is numeric. I will provide a quick English only solution, if you need a multi-locale solution just change the implementation adding try/catch for Integer.parseInt(), Float.parseFloat() and so on.

  1. Create your utility method
    package com.nestorurquiza.utils;
    
    
    public final class StringUtils {
        
        private StringUtils(){}
        
        public static boolean isNumeric(String value) {
            if(value == null) {
                return false;
            } else {
                return (value.matches("((-|\\+)?[0-9]+(\\.[0-9]+)?)+"));
            }
        }
    
    }
    
  2. Create file /WEB-INF/tld/StringUtils.tld
    <?xml version="1.0" encoding="UTF-8"?>
    <taglib xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemalocation="http://java.sun.com/xml/ns/j2ee/web-jsptaglibrary_2_0.xsd"
        version="2.0">
        <tlib-version>2.1</tlib-version>
        <uri>StringUtils</uri>
        
        <function>
            <name>isNumeric</name>
            <function-class>com.nestorurquiza.utils.StringUtils</function-class>
            <function-signature>
                boolean isNumeric( java.lang.String )
            </function-signature>
        </function>
    </taglib>
    
  3. Include a directive in your JSP to declare the prefix to use with the taglib
    <%@ taglib prefix="su" uri="/WEB-INF/tld/StringUtils.tld"%>
    
  4. Use the function in JSP
    <c:choose>
        <c:when test="${su:isNumeric('44.4')}">
            Is a number
        </c:when>
        <c:otherwise>
            Is not a number
        </c:otherwise>
    </c:choose>
    

Followers