NetCDF to Web 2.0 via THREDDS

Lately I have been working on an extension to THREDDS so you can do OpenDap style queries on NetCDF files but return a JSON format instead. Specifically so you can have a Google Visualization Source JSON format returned. Currently THREDDS allows easy access to OpenDap forms like this, I have created a dataset access form for creating the visualization source query here. This form can be used to create and test queries. I will be working to integrate this form into the THREDDS catalog page as an access type.

We (my lab) are hoping this will be useful for pushing scientific datasets, among other things, to the web as well as adding the ability to quickly create online visualizations for these datasets. The google visualization source json protocol is a simple protocol with an extensive library of widgets behind it. Creating new visualizations in JavaScript or tools such as GWT is relatively easy as well.

Google Code Project - Esri Gwt Library

Started my first google code project. http://code.google.com/p/gwt-esri/ . Its a gwt library built on the esri js api see here. Looking for help if anyone is interested. It's built from this start with the map loading and event handling reworked. I have also added the basics of a controls layer that works much like google maps controls layer. Still a lot to do. But it's a start.

More GWT Gadgets

CIMIS Senor App
Cal-Atlas - Imagery Search
Cal-Atlas - Search Light

And for anyone not using GWT 2.0, you really need to upgrade. In browser 'hosted mode' is the greatest thing to come to web app development since, well GWT.

Client Side Google Protocol Buffers & GWT

I have implemented a, crude, but working 'complier' to turn Google Protocol Buffers into GWT ready
java. It uses the base from the google code project here ... http://code.google.com/p/protobuf-js/, which I translated to Java so it could be included and complied by GWT (I added the ability to parse floats as well).

To test I accessed ~3500 protocol buffers and parsed them on different browsers (each containing a String and ~7-10 floating point numbers). I found varying results...

Chrome: ~2500ms
Safari: ~2800ms
FF: ~18,000ms
IE: ~72,000ms (stellar)

... this result had to use event chaining as to not choke the browser (FF & IE).

With such variation in parse time, this was not a reasonable solution. Parsing the protobuffers server side and sending them in JSON turned out to be MUCH faster. With all browsers having a reasonable JSON parsing time (under a second).

For the optimal solution I just modified my 'compiler' to spit out GWT overlay classes for the JSON instead of the proto classes. I then send the JSON wrapped up in a GWT-RPC. All works great now, very snappy performance.

GWT Products

Here is my first round of GWT Products.....

http://www.atlas.ca.gov/view/gallery.html
http://www.atlas.ca.gov/download.html
http://justinmerz.homeip.net/swim/gwt/RemoteCeic.html
http://projects.atlas.ca.gov/ (the embedded project search)

... with a couple more in the works ...

Couple notes, inserting as an embedded 'widget' can be interesting. Moreover, when you have a template w/ xhtml doctype or want your gwt code to exist in a different directory than the sites code. I have a simple solution using a $wnd variable for imgs. Looks something like this....

<script type="text/javascript">
var gwtRoot="themes/atlas/gwt/";
</script>
<script type="text/javascript" language="javascript" src="themes/atlas/gwt/gov.ca.ceres.gforge.ProjectSearch.nocache.js"></script>

<div id="ProjectSearch"></div>

.... now I can move my code wherever I want.

GWT and more

Back in spring I attended the Google IO event. After being impressed with several of their new apps and services (eg.. app engine, gears, etc), I must say as a developer I find myself using the Google Web Toolkit or GWT the most.

The idea is genius and for a smaller group of web developers with a lot on their plates the GWT has stepped in to become a huge time saver. With our (CERES) first project completed using GWT, here is short list of what it has added to our developers.

1) IDE developement w/ eclipse... its Java... its eclipse... enough said.
2) CROSS-BROWSER SUPPORT!!! ... can't say it enough
3) Easy sharing of packages w/in the gwt community.

And these are just three of the major additions the gwt has brought. This is not even touching on good OO development best practices that can be easily implemented in Java and not-so-easily implemented w/ JavaScript.

On to the new stuff.... Google has released a package for creating gwt gadgets that can be easily dropped into opensocial containers. Now you can use the gwt to create embeddable apps for the web.

We are currently setting up an Oauth server (for getting rid of the tiring ... please create an account ... for every new app we launch) and are going to look at integrating all three (GWT, OpenSocial and Oauth) to create easily deployable web apps.

New CaSIL

with new and improved YUI tools an new CaSIL (and CERES) site has been launched!

image zoom and table row expansion

Working on merging CaSIL with the new California State Template, I have put together an "eye candy" selection method to show which row your mouse is currently over. The idea is derived from application docks on your desktop. And if you hate it, there is checkbox to turn it off.

http://archive.casil.ucdavis.edu/casil/etc/website/stateTemplate/

OpenLayers and getCapabilities

here is a OpenLayers plug-in I am working on ... type in a URL of an OGC standard service. It parses the getCapabilities response for (currently WMS/WFS) services, then shows available layers. These layers can then be dragged (via YUI widgets) to the add layer pane and added to the map. Color and opacity of WFS services can then be modified along with image type and opacity of WMS services. Layer order is determined by the order of the list (drag to reorder).

California...
http://olgcs.casil.ucdavis.edu/captureTool/test.php

World...
http://olgcs.casil.ucdavis.edu/captureTool/test4310.html

I hope to wrap this tool up and have the source available via svn at gforge.casil.ucdavis.edu at some point.

**Note: This thing only works in FireFox at the moment.

Firebug

So with all the online AJAX/web 2.0/(cool net catch phrase here) I have been doing lately, I must say Firebug, an extension for firefox is one of the most useful debugging tools I used in a long time. Its very intuitive and allows for quick diagnosis of errors in asynchronous calls which can be tricky to catch. I highly recommend it for anyone doing online development.

Putting JS API's together

Here is an online app that shows off the merging of OpenLayers, CERES, CaSIL, and Google Maps API's and online services. The actual service that generates the land use report should be up soon.

cmluca.casil.ucdavis.edu

OpenLayers

alright haven't been on this thing forever. I have been doing lots of work with OpenLayers an all javascript mapping tool. I have been running OL w/ google maps base layers and CaSIL WFS overlays. More on this to come.

Java Threading

This weekend I decided to clean up some of my threading skills in java. I bought the O'Reilly book, Java Threads, which has been a good read. I always find the O'Reilly books very insightful. Anyway I went ahead a wrote a small Reader/Writer program using threads. It follows a simple algorithm using a basic FIFO policy with the exception of reads, if a read has 10 "time units" left and a second read comes that only needs 5, that read is allowed to take place at the same time. Here is the applet, with the code in the base dir.

http://wwwcsif.cs.ucdavis.edu/~merz/RW.html

Finished ...

I finished. Well not completely, still one more quarter of school. But with the end of this last quarter I have finished my final Computer Science/Engineering class. My final project was for ECS 152b (computer networks B) for which I wrote a Bulletin Board and Chat Client/Server. It was written in C. We had to create the client and the server from the socket level up. This became interesting when I implemented the second connection for file transfer. See you need to be able to chat and send/receive files at the same time (Which makes since). I used pthreads on the server side and fork client side. Disconnect was my favorite part since the file client and the chat client are two different processes at that point and time did not permit for a nice pipe implementation to be used. Instead the user chat client sent a kill command to the chat server for that clients connection, which sent a kill to the clients file server, who then intern sent a kill to his file server connection. Once the 2 processes and two threads were all ready, they all shut down.

Anyway now that I am done I am putting the final touches on my resume before I send it out. I am excited to finally get to use some of the stuff I have been learning over the last five years. I am going to be a little sad to move out of Davis, but as Steve Zissou says in The Life Aquatic “This is an adventure.”

Feedlounge Review

Well after a week of using feedlounge’s public release I must write something about it. First and foremost you must remind yourself that this is not a desktop application. Which is hard to do with the amazing UI that feedlounge implements. Not only does feedlounge excel at being ‘easy on the eyes’ so to speak, but I think it is part of a large milestone for the internet. In today’s world of wireless synchronization is key. The problem with that, synchronization is time consuming, and annoying. I believe feedlounge shows us the way the true American solution to synchronization… have someone else do it for you. Centralized web apps are just the way to accomplish this. With the all the backend benefits of the web and a front end that looks like _____ (insert your favorite car here) it is hard to go wrong.
Now not everything in life is perfect and I know many have been upset that feedlounge has become a paid service. And at first I had the same thought as you, damn. Though with further thought into the matter I have gotten past this slight dilemma. Mainly it came down to the fact that I can’t even eat lunch at taco bell for under $5 anymore. Carl’s Jr. is almost a sure $7, and mocha’s are creeping up on that same marker. And when I thought about comparing feedlounge to that of eating the low grade beef from taco bell, I am upset that I even thought about making such a comparison. The other solution of course (and I believe feedlounge has flirted with) is adds. Now I don’t know about you but I am getting a little sick of the fact that ever time I turn around there is somebody trying to sell me something. No I don’t need a new mop, no I don’t have erectile disjunction, no I don’t need to lose weight with your miracle bottle of legalized crack. I just want to read my flippin friends blog.
Anyway enough said, feedlounge, in my humbled opinion is superb reader. And I am excited that this door has now been opened for everyone to walk through. Let the evolution of the internet continue.

JR

So I have been away from this thing lately. Between running/playing on the UC Davis Mens Club Water Polo team and school, I have been very busy. One of my class's ecs140b - programming languages (part 2), I have been doing some work in JR. JR is an extended java that has all sorts of commands built into it for parallel process operations. I must say at first my lack of knowledge on how multiple process sync together made me struggle with the language. But once I got over that hurtle, I must say JR has some very nice features. Creating new process's on remote hosts can be done in two lines of JR code. I am looking forward to writing a pong - win brick hybrid game in JR for my final project in the class.

Ethics of Online Gambling

So I was 'googling' my name the other day and found that a paper I wrote was one of the first links that came up. It is a paper on the ethics of online gambling for the class computer ethics in the information age. I then proceeded to google 'Ethics of Online Gambling' and my paper was FIRST on google, and that is out of 227,000 results. Ha, I have made it to the top. Not bad for a guy who has only taken 2 English class's in college.

Maven plugin for gump

I have almost completed the parsing of the maven descriptor file for all the information that gump needs. I have gone ahead and built the plug-in for gump that will allow gump to build maven projects when the --do-build command is run. The command line call still needs a little fine tuning but things are looking good. So far the only additions one must add for gump to run maven is in the module tag a type="maven" and a optional goal=" mavengoal" may be added.

ex:
〈 project href="jakarta-turbine-2/project.xml" type="maven" goal="jar"/

Finally a id config file has been added to gump/util to allow updates of known id mismatches between gump and maven.

Round and round

I am starting to round a first corner where I think I am ready to start testing my first test case. Well test one of my first test case ... failed! :( Well at least gump is listening (It was a gump error not a gump crash). Time to build up some debug code and get test underway.

Maven I read you - gump

Well after a couple hundred print statements I think I have figured out the Gump3 process for parsing workspaces. I have inserted a check for a new maven flag into the gump module file/descriptor that allows maven files to be transformed in gump project files (which I actually create). I have made a skeleton output that gump will run, although many dependences are still waiting to be identified and transposed.