Solitaire in GWT

I recently tried my hand at building a Solitaire game using GWT.   This game gave me a chance to work with the drag and drop as I continue to explore the GWT framework.  The game still has a couple of small bugs in it but the basics are in place.

Checkout the game at the following URL:
http://markfarnsworth-dev.appspot.com

The code is on my git hub space.

http://github.com/farnsworth2008/Webitaire

Let me know what you think.

app engine goes clunk

Today the Google AppEngine went offline for an extended period of time.

I am sure the problem will be quickly resolved but thinking long term there is a big risk for the internet if the trend towards “Cloud” solutions drives organizations to replace their own data center’s with virtual solutions driven by a small number of big vendors (i.e. Google, Amazon, Rackspace, etc).   Having a large amount of the internet served by a single organization has a real risk that a attack and/or accident will have a very wide impact.

GitLink Plugin for Eclipse

I have been using GIT for a while now and really enjoy the way the tool works.

My one area of frustration is the lack of a good plug-in for Eclipse.  The EGIT and Gitclipse projects are great in concept but for my needs these plug-ins are not yet usable.  I expect that in a few months the projects will be better but for now, all I want is a very simple tool that does one thing well.

I decided to solve my frustration by creating a simple plug-in to show git status in an eclipse view.  This plug-in is useful because it allows you to quickly monitor the status of your repositories without leaving the IDE.  The plug-in’s ONLY feature is showing current GIT status.  If you want to add, commit, branch, clone, checkout, push, pull or perform any of the MANY git commands you will use the standard GIT tools.    For me this plug-in supports the model of work that I prefer since I personally like the command line and using the standard GIT tools for most operations.

The “GitLink” plug-in that I created does try to act as a “Eclipse team provider” and does not provide advanced features. The implementation runs git-status as an external process and then parses the result to populate a view. The plug-in also provides some icon and menu options to launch the standard git-gui and gitk tools. The plug-in runs git status automatically when resources change and as such generally will provide an up to date view of the system status. A simple refresh button is also available if the user wishes to manually trigger a run.

Source code for my plug-in is on git hub:
http://github.com/farnsworth2008/GitLink

You can also use the update site to install the plug-in in eclipse:

http://markfarnsworth-dev.appspot.com/update

If you try out the plug-in please send me a note to let me know how it works for you and if you think it needs any features. I have been using the plug-in for a couple of days on my Debian/Linux system and tested an earlier version of the concept on a windows box that was running msysgit.

To use the plug-in, setup the Git repo under your project folder using the standard Git tools for your platform.   Make sure the git tools are in the system path and if you are running windows make sure you have mysysgit under your system path.

The git status view provided by the plug-in can be activated in Eclipse via the windows view menu.

UPDATE: i have decided not to complete work on this because frankly the approach does not work well on the Windows OS and for an eclipse plug-in a solution that is not pure java is likely more trouble then it is worth.

Looking at NetBeans

I spent a few hours yesterday with NetBeans.

My primary IDE for Java development has been Eclipse and prior to that I worked with Eclipse’s predecessor (IBM Visual Age). When I first started working with Visual Age in 1998 or 1999 I was very impressed and each release has been better then the one before. Visual Age was rebranded as Eclipse; the pace of improvement has increased in recent years via the power of the Open Source community. Eclipse continues to be the leader in Java IDEs but at the same time the other players have been improving their offerings. InteliJ IDEA, JDeveloper, and NetBeans are well designed systems and I am sure that in some usage examples thes environments are better then the status quo functionality provided by Eclipse.

The last time I looked at NetBeans was several years ago. A lot has happened since my previous look. Spending time yesterday with NetBeans showed me that the project is alive and well and includes many innovative features. I enjoyed seeing how the tool has grown and see a lot of potential. Compared with my last look the tool is much more polished and given the improved speed of modern hardware performance was not an issue. The UI for NetBeans is still not quite up to the Eclipse standard but it is much better then it was in prior releases. There are pros and cons to the Swing vs. SWT approach and for the most part the Eclipse environment rendering is better due to the native widget approach that SWT uses but the gap is much more narrow then it was in the past.

I configured several of my projects for development in the NetBeans tool and created a few test projects from scratch to get a feel for the process. In addition I installed plug-ins for working with GWT, JIndent, Jython, and a couple of other community features. Overall I found that NetBeans continues to be a strong player in the IDE marketplace. Right now I still prefer Eclipse but over the next few weeks, I plan to play with NetBeans a bit more in order to fully evaluate the feature set.

Since my GarinDriver project works with any IDE I also decided it was a good time to put together a quick slide show to provide an overview of using GarinDriver with NetBeans.

I am not sure how much I will use NetBeans in the future. I may decide to stick with Eclipse but even if that is the case spending a little bit of time every few months to look at other options is a good thing.

Playing with GWT

My wife and I find it is much easier to remember a phone number if you create words out of the letters.

A while ago, she asked me to create an application to help with the process of figuring out what words you can make out of a given phone number.

Since I had some free time today, I created the Phone Words App using GWT.

Web Frameworks, AJAX, and GWT

Over the years, I have worked with ALOT of web frameworks in various environments.

The web was origionally designed around a document sharing paradigm.  User intractivity in the web started with CGI requests and basic HMTL forms.  The early CGI mode was very lacking in terms of application interactivity due to the long request and respone cycle and the lack of state management.   Basically CGI is not much more then what you have with a command line text interaction and in some ways the CGI model is more restrictive.  Compared to the C++ GUI frameworks I worked with in the early 90′s the move to web based CGI was a big step backwards as far as design goes but the push to the web required a move away from abstract widget based approaches. Java Servlets improved the CGI by making the interactions faster and providing a crude session management system but at the it’s core the Servlet model of request response leaves the same fundemental problem present in CGI.   The session is also bad since state management is on the server and as such can be lost and can come out of sync with the user’s activity fairly easy.   This lack of a true component and widget oriented model is a big problem for complex applications.

The solution to the request/response problem is AJAX. Using AJAX lets applications in browsers act like traditional client server systems. With AJAX you can keep per user state on the client, perform background server communications with overlapping transactions, control the user’s experiance and generally produce a better applicaiton experiance. That said, it is not magic and the cost is that the browsers for AJAXs must do more work vs. the dumb term type interactions of the pure HTML3.2 model. With AJAX you need at minimum a browser with script support and the ability to perform background communications (i.e. hidden iframes). The browser also must be able to dynamically control all rendered content (i.e. DOM/DHMTL). The requirement for a rich browser makes AJAX heavier then the old school flat html model. It also makes development of AJAX more complex but thankfully frameworks like GWT go a long way towards containing the complexity.

The GWT framework addresses the issue by packaging features as widgets and using AJAX RPC behind the scenes for background comunications. With GWT you get support for the various browsers for free and if properly designed a GWT application. State for the application can be properly maintained at the browser level in the widget context and as such the server can be a true stateless environment. GWT and systems like it with a AJAX and/or future RPC techniques allow for much richer web applications.

I recently put out a GWT version of my SQL color coding demo application. With the GWT framework adding support for a new features is much easier and the end user has a much better experiance. The SQL color coding demo is very small and simple but it is fun to use and shows the basics of the GWT model. You can try out the GWT version of the SQL Format app here. This is a very simple bit of GWT but it shows how AJAX interactions work and provides a taste of the programming model for GWT. The source code is available here.

A slightly more complex example of GWT is something I put together to help my wife and I make words out of phone numbers.
http://markfarnsworth-dev.appspot.com/phoneWords.html

SQL Formatter and Pretty Printer

A couple of weeks ago I started work to build a universal parser for SQL.  The work started with a basic need I had for minimal parsing of SQL statements in the GarinDriver JDBC project.  I was unable to find any open source parser code that would fully support the basic structure used by MySQL, PostgreSQL, Oracle, and other similar systems.   In particular, I wanted support for q’<>’ used by Oracle in addition to the SQL standard string encoding, MySQL’s use of C style string escaping, and $tag$ style strings for PostgreSQL and H2.  Conceptually I wanted to build ONE parser that could be configured to properly parse ANY dialect and that would support a framework for adding additional exceptions in the future.

The basics of my parser framework is now complete. It is smart enough to parse MySQL, H2, HSQLDB, Oracle, PostgreSQL, and ISO SQL2003 variants. The shallow parsing framework uses a base Token class and a range of subclasses to describe the most basic elements of the SQL grammar. The parser identifies the core keywords, statement boundaries, string, comment, and identifier limits but does not look into deeper language issues like statement structure.   One benifit of the shallow parsing is that it allows for flexibiltiy and does not require a complete BNF style grammar.  Even with only this basic level, it is possible to leverage this code for useful stuff beyond the GarinDriver, LiquiBase data migration solution.

The following bare bones test page demonstrates how the parser can be used to format and add color to SQL batches.  The code would also be useful for someone building a universal SQL editor.
http://markfarnsworth-dev.appspot.com/RenderSql

Overall, I am happy with the approach and while I plan to use a more established framework like ANTLR or XTEXT for future deep parsing efforts I feel my homegrown framework provides a better approach for shallow parsing vs. the larger and more complex language tools that I have reviewed.

A Universal Parser for SQL

I recently completed some work on the GarinDriver to research a more flexible model for database schema change management. This work gave me a chance to dig a little deeper into the nuts and bolts of Java database interactions. Working to track schema changes and support a wide range of databases increased my appreciating for powerful open source database platforms available today. In particular, working with H2 Database gave me a chance to explore what I now think is one of the hidden treasures of the open source DB marketplace. Overall it was a fun bit of hobby coding and I believe it will be useful in future real world projects.

Working on the driver, one challenge that went beyond my expectation was parsing the various dialects of SQL. In a review of current open source projects, I did not find any public license parsers that can parse statements from ALL database systems. In a sense, we are supposed to have a standard but it is really well followed by the vendors and as such writing a universal parse is a bit difficult. I had some free time this weekend so I took on the challenge since it is an area where the open source community really does not currently have a good solution. For example the tools that ship with Eclipse can not parse the $tag$ style strings from PostgreSQL or the q’[O'Brian]‘ style supported by Oracle. In any case,  I had some free time and interest so I decided to build my own parser.  Without such a parser it would be impossible to provide full support for the statement execution model needed to support the GarinDriver desgin.  Workarounds were possible but a proper parser felt like the best approach.

Building a parser is complex and often involves specialized tools (i.e. ANTLR, BISON, JAVACC, FLEX, LEX, YACC, etc). There are benefits and drawbacks to the traditional grammar definition and parser generator approach.   My initial review it seemed that supporting the often contradictory approaches used by different systems would be quite difficult with these tools.

Since all that I needed for the GarinDriver/LiquiBase project was “shallow parsing” my approach was to use a small framework of Java classes. The use of my own framework allows me to share logic across the SQL dialects and provided a chance to explore idea for a more dynamic approach to “shallow parsing”. So far the approach seems to be working out nicely although I am considering building a deep parser at some point later on with the Eclipse XTEXT project but for now the flexiiblity of a home grown hand coded parser has proved to be a viable approach for “shallow parsing”.

Parser Fun:

• Standard SQL comments have EOL style (–) and the block style (/* */).
• Block style comments nest within each other so parser must count the nesting levels.
• MySQL supports pound sign comments in addition to the standard forms.
• HSQLDB supports // style comments in addition to the standard forms.
• Standard SQL uses double quotes for identifiers and single quotes for text strings.
• The parser must support both including quote symbol doubling for escapes (i.e. O”Brian or My “”big”" table.
• Oracle supports q’[O'Brian]‘ in addition to the standard style.
• MySQL uses C style string encoding (i.e. ‘O\’Brian’ ).
• PostgreSQL supports $tag$O’Brian$tag$ style in addition to the standard forms AND the mysql format.

My parser uses object oriented techniques to define a base token concept, extend the base concept for SqlStatements, and then extend the SqlStatements to define the dialect variants in what I hope will be an extendable framework. This approach provides more flexibility for future growth vs. more static models and parser generator tools. In any case the approach seems to work and the ability to use object orientation to extend the parser in new directions seems like a good thing.

As with other hobby projects, my work in this area is EPL and hosted on Google code. The documentation is sparse but if you are looking for SQL parser code you may find that this code can help you develop new and interesting tools for working with SQL systems.

http://code.google.com/p/garinparser/

If you decide to use the parser please let me know. Also, if you can define a legal SQL statement that does not parse correctly with my parser let me know since I like to eat tasty bugs.

GarinDriver plus LiquiBase

LiquiBase Schema Manager GarinDriver with Enhanced Schema Manager Distributed Locking Model YES YES Multiple Context Streams YES YES MySQL DB YES YES Oracle DB YES YES PostgreSQL YES YES HSQLDB YES YES H2 YES YES Liquibase XML Change Set YES YES Vendor SQL (i.e. ENGINE=InnoDB) PARTIAL YES Complex SQL batches (i.e. from mysqldump style tools) NO YES SQL/DML Recording (record from any JDBC tool) NO YES Manual, FAQ, etc. YES PENDING

GarinDriver is an JDBC driver and schema manager that I created to supports a schema deployment. The driver wraps your backend driver and provides a framework for SQL/DML script recording and playback to streamline distribution of changes across environments. The driver can also be configured to run with the enhanced schema manager or use the standard LiquiBase SchemaManager via an option in it’s JDBC URL.  With both models, GarinDriver ensures that schema changes are automatically deployed BEFORE client code connects to the system. With the enhanced manager, the schema change stream is defined as a “stack” of numbered SQL and/or XML files. This model works well with mysqldump, Toad, SQLPlus, and similar tools. Tools like mysqldump are especially useful for creating a complete system script in one simple step. Subsequent scripts can also be written as general SQL standard files and/or use the full flexibility and power of the native dialect. The driver level DML recording feature provides an option to record DML statements you issue in your JDBC based tools and automatically add them to the schema stack. This option means that you can configure ANY schema tool that uses JDBC and automatically captured the numbered series of scripts to build your schema distribution table. In addition at any point you can place LiquiBase XML files in the stack to provide flexibility for vendor neutrality while also having an option for running the vendor specific scripts when and where they make the most sense. The combination of GarinDriver and LiquiBase provides a lot of power and flexibility for complex envirionments. For more information about GarinDriver please visit the main project site. http://code.google.com/p/garindriver/ We will be providing additional documentation in our wiki in the coming weeks.

GarinDriver

I recently released the GarinDriver as an open source project published on google hosting.
http://code.google.com/p/garindriver/

The GarinDriver is a JDBC driver that includes the ability to run a stack of scripts for initial creation of a database schema and incremental updates to that schema.

By working this logic into the JDBC driver layer, it is really very easy to add the capability to exiting projects.