Parsing CSV files with Grails

One of the arguments that I often make for my use of CFML is how you can do so much with so little code. Seemingly every time I attempt to do something that I haven’t done previously with Grails, I find that argument holds less water than I thought, as I can often do it even easier in Grails.

For a current project, we have an occasionally updated CSV document that contains codes related to the customer’s industry. Given that this file will be changing with additional codes being added, while this app is in early development, we decided that we would just keep it in our application config directory and ensure that any new codes are added during the application bootstrap routine.  Here is what I came up with:

// Insert new codes
def csv = new File("grails-app/conf/code.list.csv")
def code
csv.splitEachLine(',') { row ->
   code = Code.findByLabel(row[1]) ?: new Code(
      code: row[0],
      label: row[1]
   ).save(failOnError: true, flush: true)

Essentially, the above is saying:

  • Read the CSV file
  • Loop each line, where each line is referred to as “row” in the closure.
  • Search in the database for codes with the same label
  • If the code does not yet exist in the system, create a new instance of Code passing in property values from the row in the CSV file.
  • Save the new code to the database.

As you see, I have added line breaks for readability, but I was able to get the result I was looking for in THREE lines of code! I figured I would share in case anyone is looking for a similar solution.

6 thoughts on “Parsing CSV files with Grails

  1. Just for fun I rewrote that as one line (my perl roots are showing).

    new File(“grails-app/conf/code.list.csv”)splitEachLine(‘,’) { row -> Code.findByLabel(row[1]) ?: new Code( code: row[0], label: row[1]).save(failOnError: true, flush: true) }

  2. Oops, left out a dot in there.

    new File(“grails-app/conf/code.list.csv”)?.splitEachLine(‘,’) { row -> Code.findByLabel(row[1]) ?: new Code( code: row[0], label: row[1]).save(failOnError: true, flush: true) }

  3. Have you tried doing this with 10′s of thousands of lines. I have found that after a few thousand lines it gets real slow.

    • No, I have only put through a few hundred lines, and given the fact that it is wrapped up in the bootstrapping, I haven’t really paid attention to the speed of it. I will keep that in mind though, thanks.

      • I would suspect that the slowness might be caused by the flush:true in the save. I haven’t tested this but it seems like the logical issue. In case you didn’t know, flushing forces hibernate to write the changes to the DB immediately rather than waiting. I would suggest trying it without the flush and see if that improves speeds.

        (I know it’s an old thread but better late than never I suppose)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>