Google App Engine in Java

The promise of the Google App Engine has been intriguing from the start. In essence, the app engine is a virtual environment for server-based applications hosted by Google. Any developer can write an application and deploy it on the app engine. The hosting is free up to a certain limit. When an application gets traction and traffic gets heavier, Google will ask a fee and scale the hardware for you. The app engine was originally a Python environment, but since a few years it’s available as a Java environment as well. In the past week, I had some spare time to take a closer look at the app engine and write a small sample application. This post documents my initial experiences with the app engine, with hurdles to overcome and thoughts on the technology.

The Java environment of the app engine is a very custom version of a J2EE servlet container. For example, core Java APIs, Java Servlets, and Java Server Pages are working mostly as expected, but a few other components, such as Enterprise Java Beans, JDBC, and RMI aren’t available. The limitations on Java have been documented on this site. Perhaps a more significant difference with other Java containers is the persistency model. App engine has a data store based on Google’s big table, which is very different from a standard relational database. Although the app engine supports standard APIs for persistence, such as Java Data Objects (JDO) and Java Persistence API (JPA), the semantics of these APIs are very different from when a relational database is used. The emphasis of the app engine data store is on read performance and all queries need to be pre-indexed. This puts certain limitations on the use of relationships. For example, the application can’t do inner joins. In order to scale the data store effectively, the transaction model is also different from a traditional database.

The development environment provided with the app engine is Eclipse-based and comes with an embedded servlet container. This allows for rapid local prototype development before publishing the application on the app engine. It was fairly easy to get started with the SDK, although there are a few caveats to cope with:

• JSP code completion doesn’t work out-of-the-box, because the app engine plug-in doesn’t add the JSTL libraries to the classpath of the project. The libraries are packaged with the app engine SDK and can easily be added to the project classpath to fix JSP code completion;
• Generated JSP code isn’t visible in Eclipse, but for the generated files can be found in the temp directory when running the development container. For a Linux setup with the latest app engine SDK, the path is: /tmp/Jetty_127_0_0_1_8888_war____.g0qk00/jsp/
• JUnit testing is supported, but the documentation is somewhat confusing. The trick is to have a normal Java project next to the app engine project in the same workspace. The normal Java project has a reference to the app engine project and the testing libraries as described in the documentation. JUnit tests can run directly from Eclipse.

The sample application I wrote consists of a simple data model, three JSP pages, and a background HTML Web page scraper. The code of the sample application I wrote is available here and the live version is available at visa-bulletin-tracker.appspot.com. The code is available under an Apache 2.0 license, so feel free to re-use and/or reference the code.

Packing light – System on an External Drive

In the coming months, I will be traveling a lot. During this travel, I would like to take my work laptop as well as a personal system. It is too much to take two entire systems, so I decided to pack light and only take the work laptop and keep my personal files on an external SSD Hard Drive.

I created two partitions on the SSD Hard Drive. One with the personal data, and another one with a Linux Ubuntu operating system. With the operating system on the external drive, I will have a complete personal system without the need to take a second laptop with me. I can use the same set of hardware, but my personal data and applications will stay completely separate from the work laptop. On the work laptop, I will load the personal operating system directly from the external hard drive. The performance turns out to be really reasonable with USB 2.0 interface and an SSD drive.

On my PC at home, I need a way to synchronize my personal files on and off the external drive. My PC runs Windows 7, but the operating system on the drive is Linux. The data partition has a FAT32 format and I can synchronize files with Windows 7 using the freeware tool PureSync. This is better than a simple drag-and-drop, because changes can come two ways. I can modify the files in the original location on the Windows 7 PC, and I can also modify files on the external hard drive using the Linux system. PureSync makes it easy to update both sides and resolve any conflicts.

Additionally, I would also like to run the operating system on the external drive as a virtual machine in Windows 7. This makes it easier to set the system up the way I like it to be without needing to reboot to a different drive too frequently. I have been using SUN VirtualBox for a while now, but I have mostly been using it with virtual drives that represent themselves as a file in the host system. It turns out VirtualBox can also load a guest operating system directly from an external hard drive. Instructions on how to do so are located here. Don’t forget that VirtualBox and associated utilities need to run as Administrator in Windows 7 if you’re going to try this yourself too.

*Ville gets $180 mjn funding

Word came out today that Zynga (the company behind FarmVille and other social games) received $180 mjn in funding. Another hint that the social Web gaming is about to explode.

MMO: Monopoly City Streets

Today I spent some time playing with the newest variation of the traditional Monopoly game. Monopoly City Streets is a Web-based multi-player version of the game which uses real street names from Google Maps. So far I’ve been able to avoid ‘social’ games such as Farmville and Maffia Wars on Facebook, but I was really intrigued by the idea of Monopoly City Streets. What’s interesting to me is that the game includes real-life data from Google Maps. This makes the game much more attractive as people can relate to the game better. The concept of the game is simple: each player buys streets, puts real-estate on his/her streets and receives rent each day. Players can also sabotage each other, for example by destroying buildings. After signing up for the game (which is free), I found that I was not the only one interested. Many of the streets in my neighbourhood and in the place were my parents live, were already taken. It looks like the game is a big success for its developer, Hasbro.

Unfortunately, the game will end next week on December 9. The developers state on their blog that “[the] objective was to design and deliver a free game that would be a fun, interactive, promotional introduction to our new board game, MONOPOLY City.” (Monopoly City is just a new version of their traditional board game.) The success of their ‘promotion’ came completely unexpected. It would be interesting to see how serious Hasbro will get into this on-line gaming market now that so many people have expressed interest.

Another game being brought on-line is Civilization Network which is expected to come out next year. Technically, Civilization IV already had an option to play over the Internet, but Civilization Network is very different from this, because it uses real-life data and is Web based. For Monopoly City Streets, the real-life data comes in the form of a map with street names from Google Maps; For Civilization Network, the real-life data is the social network from Facebook. Civilization Network will be built on the Facebook platform so you can play with or against your (real-life) friends on Facebook. I’ve personally been a big fan of the Civilization games and I can’t wait for Civilization Network to come out.

My Favorite PC Applications

This is just a quick list of the software I have installed. All of them can be used for free.

On-line backup: MozyHome

This service is free for up to 2 Gb automatic on-line backup and only 5$/month for unlimited! The photos, documents, and other media stored on my PC is invaluable to me. From the three aspects of security (availability, secrecy and accuracy), availability matters most for this data. I evaluated different systems for data backup and figured on-line backup is the best fit. Once it has been setup, on-line backup services provide the most robust protection against data loss. MozyHome provides one of the most competitive services for on-line backup. It’s really much easier and error proof to sign up for the service than setting up remote backups to a friend’s place or storing your data on an external hard drive or NAS at home.

Browser: Mozilla FireFox with adblock and X-Marks

FireFox is getting more competition from Google Chrome and Apple Safari, but both lack good plug-ins. The problem is that both browsers are based of the WebKit engine, which doesn’t have a good plug-in architecture. I use FireFox with adblock and X-Marks. Adblock removes ads which improves the readability and speed to load Web pages. X-Marks synchronizes my bookmarks between the my five computers (PC, Mac and Linux). Whether I’m at work or at home, I can always access all of my bookmarks.

E-mail: Mozilla Thunderbird

I keep all my e-mails (since 2004) in this e-mail client. I haven’t noticed any performance problems, even with about 4000 messages in my inbox. And it’s totally free.

Virtual Machine: Sun VirtualBox

I had a great experience with this tool. I’ve used VMWare in the past, but VirtualBox is a free alternative that works at least as well as VMWare. The feature I like most is the integration with memory paging in the host system. In older virtualization software, a chunk of memory in the host system is reserved for the guest system. With memory paging integration, the virtual machine will only take the amount of memory from the host system that is really used by the guest.

Communication: Skype

Perfect, for chat and PC-to-PC or PC-to-Phone calls. I use this extensively for calls with my family in Europe. PC-to-PC is free, and PC-to-phone is charged, but much cheaper than via a regular phone line. I’m no longer using MSN Messenger, Yahoo! Messenger or Google Talk. The only problem I’m seeing with Skype now is that recent versions of the application take a while to load on start-up.

Photos organization, editing and publishing: Picasa

This works well for me to manage my photo albums. It is easy to use and photos can be published directly to picasaweb from the application.

Virus scanner: AVG

The free edition is good enough and seems to work well.