Parsing CSV files with Grails

One of the arguments that I often make for my use of CFML is how you can do so much with so little code. Seemingly every time I attempt to do something that I haven’t done previously with Grails, I find that argument holds less water than I thought, as I can often do it even easier in Grails.

For a current project, we have an occasionally updated CSV document that contains codes related to the customer’s industry. Given that this file will be changing with additional codes being added, while this app is in early development, we decided that we would just keep it in our application config directory and ensure that any new codes are added during the application bootstrap routine.  Here is what I came up with:

// Insert new codes
def csv = new File("grails-app/conf/code.list.csv")
def code
csv.splitEachLine(',') { row ->
   code = Code.findByLabel(row[1]) ?: new Code(
      code: row[0],
      label: row[1]
   ).save(failOnError: true, flush: true)

Essentially, the above is saying:

  • Read the CSV file
  • Loop each line, where each line is referred to as “row” in the closure.
  • Search in the database for codes with the same label
  • If the code does not yet exist in the system, create a new instance of Code passing in property values from the row in the CSV file.
  • Save the new code to the database.

As you see, I have added line breaks for readability, but I was able to get the result I was looking for in THREE lines of code! I figured I would share in case anyone is looking for a similar solution.

Video blogging on the cheap – not as easy as it should be!

I just recorded two screencast videos last night that I wanted to use as video blog entries. Seems easy right? Just find a video host!

Unfortunately, “easy” is far from the way I would describe my experience, and I am somewhat exasperated by the process at the moment. So here is the detail: I have two videos, one being 9:17 long, and the other being 15:02. Both of these are recorded as OGV files, which is part of the free, open, cross-platform OGG media container format.  All I need to do is find a service to host them and stream them.  So far so good right?

I decided that I would try to look around for a video hosting solution other than YouTube, since I have posted screencasts on there before and the video degradation was horrendous.  After some googling and reading reviews, I started down a spiraling path of services leading to nowhere, beginning with….

  • Vimeo (verdict: fail) – Vimeo seemed like a great place to start.  Any time I have seen their videos, I have never noticed a degradation.  They offer HD and the service is free – kind of.  In all actuality, there were three issues for me here.
    • bad: They do not support the OGV file format, so I had to convert the OGV to an AVI before uploading it.  Of course they don’t actually tell you this until you have sat through an entire upload first! There was a degradation that occurred during that process, so even after uploading it, the quality wasn’t as good as I liked.
    • bad: Free accounts are only allowed to upload a single HD video per week.  Already in my first try I had two, so that is a show stopper.
    • good: The HD version that was uploaded was better than many of the alternatives,
    • bad:  you can’t embed the HD.  If the user wishes to see it, they have to click through the player to the site and watch it on the Vimeo site.
  • Viddler (verdict: fail) – Viddler seemed like a good alternative to Vimeo.  However, ultimately it doesn’t seem to be the direct fit either.
    • bad: Just as with Vimeo, they do not support OGV.
    • good: As opposed to Vimeo, at least they tell you about the lack of OGV support  as soon as you attempt uploading!
    • bad:  Since I had already converted one of my videos to AVI, I went ahead and tried it.  Even in full screen mode, the degradation was bad enough that I couldn’t see what I was typing in the video, which is kind of the point!
  • YouTube (verdict: fail) – After nixing Viddler,  I thought “why not at least try YouTube again?”, and I was soon reminded of exactly why not.
    • good: They support OGV!
    • bad: Even my 9:41 video was deemed “too long” and was promptly removed.
    • bad: I couldn’t even get far enough to report on degradation!

So just as I began typing this blog entry to air my dissatisfaction with things in general, I came across this post, praising the combination of using Jing to record, and to host the video.  The video clarity on his example was really impressive.  “Ha!” I thought, “finally!”.  So I now have one more to add to my list:

  • (verdict: fail)
    • good: They allow you to upload any file type whatsoever! (I think anyway)
    • bad: They only embed a few different file types into players.  OGV is again not supported.

So here I sit, still without a good solution to what initially seemed like it should be a no-brainer of a problem to solve.  The amount of time that I have wasted to still be sitting at square one is terribly aggravating.  Between upload times and service-specific encoding times, I am more hours deep into this than I care to think about.

HTML5 to the rescue?

One thing that came out of this search is that I learned that HTML5 natively supports OGG/OGV using the <video/> tag.  (more here), and based on an example that on this page, it looks very cool!  The only fundamental thing that is holding me back at the moment is that there doesn’t appear to be any option to allow your user to ‘full screen’ your video out of the player.  So close, yet… still no solution!

If anyone has any good recommendations, feel free to leave them in the comments.

How to: write the last N linux terminal commands to a file

Sometimes blog entries are for you.  Sometimes they are for me.  This one is the latter.

The other day I asked the following question on Twitter:

Anyone know a way to write out the last N commands run in the #linux terminal to a file?


I got a plethora of responses within minutes, but by far the most complete and tricked out response came from Joseph Lamoree @jlamoree, who gave the following solution:


history | tail -n 10 | sed -E 's/^ +[0-9]+ +//' | grep -vE '^history$' > cmds


In that command, the “10″ represents the last 10 commands, and “cmds” is the filename that the output will write to. Since there isn’t the remotest chance in hell that I would ever remember this, and Twitter is about the worst place for me to go back later to find technical information, I am putting it here on my blog for future reference. Thanks Joseph!

Open Letter: Stepping down as DFWCFUG Manager

At Tuesday night’s meeting (3/8/2011),  I announced that after 55 meetings at the helm, I am stepping down as manager of the Dallas Ft. Worth ColdFusion User Group. 

Am I tired of doing it?  Am I leaving the language?  NO, and NO!

As an Adobe UGM, one of my responsibilities is to endorse and evangelize the product of Adobe ColdFusion (ACF).  For numerous reasons over the past year or so, I have found myself at growing odds with this task.  As competing open source engines such as Railo and OpenBD are gaining in functionality, stability, and performance, as well as being made freely available to the CFML community, it is impossible to ignore them as true contenders in this space.  Where they were once viewed as free alternatives, they have moved to the position of driving change and driving features that I would like to see in ACF. I wholly feel that these engines are the future of our community, and should be given equal attention rather than be viewed as just an alternative.  Based on that fact, it is disingenuous for me to continue in my role as an Adobe UGM.

As of its inaugural meeting on April 5, 2011 at the Paladin Consulting office in Dallas, I am going to serve as coordinator of the DFW CFML User Group, a non-product-specific user group composed of enthusiasts of the CFML language, regardless of the engine that runs it.  Without the pressure of promoting one company’s product over another, we can focus on what is really important to us, which is the power of the CFML language and the diverse ways that it can be used across various platforms.

It is important to note that the new group will not be strictly an “open source” group, nor is this a swipe at any kind of Adobe itself.  The group is simply not going to endorse a single product as the only viable solution to writing enterprise level applications in CFML.  Our content will doubtlessly include Adobe ColdFusion, but will not be exclusive to it.

So where does this leave the DFWCFUG?  Adrian Moreno has served as co-manager of the group for several years now.  Adobe mandated this hierarchical approach to how their groups will be organized so that in the event of the departure of a manger, the group can carry on without interruption with the co-manager taking over.  I have spoken with Adrian at length on this topic, and he does not share my vision on the DFW CFML User Group, and feels that it is important to have a product-focused user group under Adobe.  As a result, he has opted to take the role of group manager effective immediately and will be leading the DFWCFUG.

I want to make it abundantly clear that this will not be an “us vs. them” scenario between the two groups.  We are in this together as one community with varying interests and it is in all of our interests to positively promote both groups.

Fortunately, I think that this leaves the DFW CFML developers with some excellent options!

I plan on sharing much more about the new DFW CFML User Group in the near future.  Please follow us on Twitter at @dfwcfml and look for upcoming announcements in the next few days.

Lastly, thanks for letting me serve as leader of the DFWCFUG all these years.  It has been an honor and a privilege to do so.

~Dave Shuck

Invisible blog post

This is my first post after moving my blog over to WordPress.  Considering that the new feed isn’t in the Adobe Feeds, ColdFusion Bloggers, FullAsAGoog, etc., this will effectively be a tree falling in the forest with no one around.   Quite honestly, I am not even sure that I will request the update to the aggs.  When I am not listed in tech feeds, I can have stupid meaningless posts like this. :)

Presentation Summary and Files: Mach-II User Authentication and More

Last night I gave a presentation to the DFW CFUG on Mach-II.  I spent the first part of the presentation talking about all the new features that have been added in both versions 1.5 and 1.6.

The features we covered were:

  • Mach-II 1.5
    • XML Includes and Modules
    • Subroutines
    • <redirect /> enhancements
    • Complex properties defined in config XML
    • URL Management Features (SES support) -buildUrl() & buildUrlToModule()
    • Bindable property placeholders for params
  • Mach-II 1.6
    • Broadcast style listeners – putting the II in Mach-II
    • Application.cfc enhancements – much simpler and new plugin points for onSessionStart/End
    • handleException() plugin point enhancements
    • Enhancements to tracing/debugging
    • New cache manager (granular)
    • Lot’s of goodies slated for 2.0.  Anyone interested in a dashboard module?  This will allow for programmatic reloading of the entire application, modules, managing the caching engine in addition to future enhancements to the framework.

Next we went into code examples where I had three stages of a sample application.  The first had two public events, one of which we would secure with user authentication in the next step.

In the second phase, we added in necessary CFCs to support our User – User.cfc, UserDAO/Gateway/Service/Facade, and tied them together using the newly available ColdSpring property.   We then secured one of our events by using a CheckIsLoggedIn filter

In our third and final phase, we took a very simple approach to introducing roles based security by securing a new event limiting access to only users who had a role of type “Site Admin”.  As mentioned in the presentation, there are certainly more robust ways to implement roles based security, but hopefully the example will help kick start ideas.

You will find the following in the zip file below:

  • Presentation in both ODT and PPT
  • Three sample applications representing all stages covered
  • Sample http-vhosts.conf showing how I had the apps set up in Apache.
  • MySQL backup file of the database used.  You will need to add a ColdFusion DSN named “machii_auth”

Thanks to all that attended!

Flood pictures from Lake Texoma 4th of July 2007

I took my family up to Lake Texoma over the past couple days where both my brother and my parents have cabins right on the water in the Texins Club, which is mostly made up of current and former TI employees and Family.  As of this today, the water is 22 feet above normal and my brother’s cabin is actually now *in* the water, which has been rising 1″ hour for the past week with no sign of relief in the near future.   The full set of pictures is available here, but since you are already here, I will post a couple of notable pictures:

The baseball diamond there in the park.

Steps leading up to my brother’s cabin.  There is usually a nice green yard between the porch and the water.

Same porch the next morning.  Notice the steps are gone.

The neighbor’s porch swing.

In case there was any question…

Usability – It’s not just for websites!

As a web developer, usability is something that we consider on a daily basis.  When it comes to functionality that has become a standard, we don’t jack with it!  For instance, we have long established that when you mouseover a link the cursor turns into a pointy finger.  Can we alter that?  Sure!  But people would be confused and would likely miss some content in your application.  See that navigation bar at the top of my page?  I could easily move to align with the bottom of my site, but the result would lead to more confusion.

This leads me to a mini-rant about my new phone.  A couple weeks ago I dropped my LG phone one too many times.  It seemed to always think I had a headset plugged in, although I have never actually owned a phone headset.  In addition, when I turned it on, it would randomly call my Mom and Dad.  That one I still can’t figure out!

So, being a generally cheap guy, I stopped by the Cingular store to pick up the the next-to-the-bottom-line model for about 60 bucks and was on my way without paying too much attention.    After unwrapping it and charging for the first time, I discovered something that baffles me.  Check out this keypad!

Sony Ericsson flip phone keypad

What genius thought it would be a good idea to design the keys like that?  After about 35 years of a consistent and well-defined pattern of telephone keypads, Sony/Ericsson decided it would be a good idea to alter that by offsetting the middle column.   This serves no functional purpose either.  You can see they could have quite easily shifted the middle column of buttons up so that they would be aligned.  I am sure my fingers will get used to it, but dialing without looking at my fingers presents a new challenge.  I suppose that is one of the costs of being cheap.  :)

Mounting drives in Windows… just like Linux!

Before continuing I must concede to the fact that I am *not* a fan of Windows. I use it where I have to, but by and large I feel that Linux, specifically Ubuntu, is just a more pleasant experience and is a better tool for the kind of jobs that *I* need a computer to do. I must also admit that I was Microsoft certified about 9 years ago (the NT4 track!), so what I “discovered” last night might not be entirely new to many people, but it was certainly new to me so I thought I would share. Plus, considering how rarely I have compliments for Windows, I feel obligated to share this so that my steadfast Microsoft fanboy friends will quit saying “Why do you hate Bill Gates?”, which incidentally I do not.

Now that I have gotten that out, let me tell you about a cool feature I found within Windows last night. It actually does something the way that Linux does!

One of the Windows web servers that we interact with has its webroot on the D: drive, with a path D:inetpubwwwroot. At the time that this application was created, hard drives were not the size they are today and 8GB seemed like a reasonable partition for a data drive. However the application has grown, as has its need for hard drive consumption and it finally reached a level which needed to be addressed.

I originally set out to add a new drive (E:), then move the wwwroot over to the new drive, update all mappings in IIS, including virtual directories, and update any mappings within ColdFusion. This was not a very exciting prospect considering this is a live production server. However, this seemed like a fairly logical approach so I began.

First I added the new drive and initialized it in the Disk Manager. I now had this 80GB empty partition which I planned on turning into E: After choosing to to make it a “Primary Partition” and selecting the size, I got to the point for choosing the drive letter. This is where an option jumped out at me that I had never noticed before, which is a testament to both my lack of observance and to how fast I normally cruise through this section! I was presented with the following:

WHAT??? “Mount”???

The solution became abundantly clear immediately. Rather than have to re-map paths and risk blowing up whatever buried physical paths might lurk under the covers of this legacy application, I would simply mount the new drive as: d:inetpubwwwroot – just like Linux but with backwards slashes and the funny letter/colon thing on the front!

So, I renamed the existing wwwroot folder to wwwroot.old, mounted the drive to that position, and copied over all files from the old wwwroot to the new wwwroot. I restarted ColdFusion and IIS and the application picked up right where it had left off without a hitch!

So (get ready to write this down, because you won’t hear it often from me)…. YAY for Windows!