JRun wsconfig error- Security alert: attempt to connect to JRun server from host

I was experimenting with the Railo 3.3 installer, which includes an IIS connector to Tomcat, which works really well.   Too well in fact!  When I ran it, it actually unmapped all my existing IIS ISAPI mappings to JRun and was sending all requests to Tomcat.

I decided the quickest fix to this would be to simply open up /JRun4/bin/wsconfig.exe and remap the sites that were no longer connected.  However, when I did this, I received the following error:

Could not connect to JRun/ColdFusion servers on host localhost.

Knowing perfectly well that I had an instance of JRun running, I went to the terminal to look at the standard out and saw this:

Security alert: attempt to connect to JRun server from a host

In case that is too hard to read, it says: “Security alert: attempt to connect to JRun server from a host”.   I suspect that because I am attached to a WIFI connection with an IP Address on 192.168.*, and then VPN’d into my company with a second address of 10.252.*, JRun assumes that the connection attempt is coming from outside the subnet.

I went digging through files in JRun4/lib and came across security.properties.  In this file, there is a default setting:


I altered that restriction  setting from “″ to “*” like this:


Once I did this and restarted the server, I was able to use wsconfig without issue.  And my ACF sites are pointed to JRun, my Railo sites are pointed to Tomcat, and all is right in the world again.

NOTE: DO NOT DO THIS ON A PRODUCTION MACHINE!   If you do, I strongly recommend that it is a very temporary change.

End of an era – Turning the lights off at InstantSpot

As of this first week of 2011, we will be permanently turning off the lights at InstantSpot. I felt that it was fitting to give it at least some type of obituary, and share my reflections of what has been a 4-and-a-half year story of the process.  We have learned much through the process, and hopefully we can share some of what we learned with others.

Starting in 2006, Aaron Lynch and I started on modest plan to revolutionize the blogging world.  OK, so that isn’t really true – it happened like this…

In the beginning… a simple CMS

In 2005, Aaron was making a career change and after years of being friends through the Jeep and 4×4 community we found ourselves in the same line of work as ColdFusion developers.  Around the beginning of 2006, he joined my company which gave us way too much time to discuss and conspire on how we might take over the world.  We decided to begin by making a simple no-database CMS in ColdFusion that we could distribute and sell for a nominal fee.  Of the ones that we deployed, we found that no one was really interested in hosting it themselves and we ended up putting a few instances on our server.

A network was born

Once a small handful of them were running, we created a single directory page that listed and linked to all of the individual sites.  Even though it was exceptionally simple and rudimentary, we realized that in essence we had a “network”.  Given the fact that no one really had an interest in hosting themselves, we quickly realized the fallacy of duplicating the code base for each new site and set out to more effectively externalize the settings and make the configuration centrally persisted in a shared database and run all of the sites from a single set of code.  We quickly developed methods of skinning individual sites, and using subdomain URLs.  The methods we came up with are almost exactly the same methods that we would end up using the next 4 years moving forward.

The addition of blogs

Almost immediately we decided that we needed to add blogs to the sites.  In a move that would haunt us to some degree for a long time to come, we chose to implement the open source BlogCFC application created and maintained by Raymond Camden.  While there was nothing inherently wrong with BlogCFC itself, it was just a poor choice on our part for several reasons:

  • We already had a functioning CMS build using the Mach-II framework using its own database.  Adding in multi-site support to the blogs (which I don’t think it had at the time, if I recall correctly) and tying that into our existing application proved to be a ton of work and felt like such a goofy solution.
  • Combining the two databases into a single one, and the vastly different approaches of data persistence (naming schemes, UUID vs integer keys, etc…) made our code base look schizophrenic.
  • We actually had two complete applications and had to manage session replication between them.
  • We ended up using a really a small portion of the actual BlogCFC code base.  If we would have just created the functionality on our own from scratch, we would have spent far less time and ended up with something that seemed like far less of a hodgepodge.

Eventually we got it working, and did so somewhat successfully, but it really felt like a hodgepodge of methodologies.  However, we started growing our user base, got some great community feedback and support, and had a relatively well performing blogging network.

The dirty secret

One of the seriously embarrassing dirty secrets that we had was the original hardware/server architecture of InstantSpot.  When Aaron and I both started out writing the code, we were not the Linux nerds that we grew into being later.  We were both Windows developers and had never paid as close of attention to casing issues as we grew to later.   Unfortunately the first time that we tried to run Spot under Linux, it blew up severely.  We started trying to work through issue by issue,  but decided we would do a temporary fix by throwing it into a Windows VM on our co-located Linux server.  However, this temporary work-around ended up sticking around much longer than we anticipated.   So, for the first year or so, InstantSpot chugged along on a single PC that was little more than a common desktop, running Ubuntu Server, with a Windows VM in VirtualBox!  The fact that it supported as much traffic as it did was always just a little bit comical to us.

The inevitable rewrite

As we continued working to bring in a continual array of new enhancements – many of which were implemented but never even seen by our users! – we found more and more that our architecture sucked badly enough that we would actually have to do something about it.  In addition goofy hardware/OS issue, the maintenance of dealing with the two separate applications became this growing elephant in the room.   As our user base and our exposure continued to grow, we decided it would be worth a complete re-write from the ground up.  When we first started slapping stuff together in the befinning, we didn’t have a defined target in mind, so surely a rewrite would be less time consuming since we actually had a goal now right?!   We set out about mid-2007 to find out.  Around that time, Railo had hit RC 2.0 and Aaron and I were both enamored by the idea of a completely open source version of InstantSpot.  We busted our asses the last quarter of the year and were finally ready to roll right at the turn of the year in 2008.   I will never forget how many people scoffed at us using Railo.  I mean seriously, given the enormous growth of it in 2010, you wouldn’t believe all the snickering we put up with about it early on.  However, we found huge performance increases over ColdFusion and felt like we were really breaking new ground.

Disaster strikes

In January 2008, we rolled out the newly rewritten InstantSpot with HUGE expectations.  As soon as we flipped the switch at 1:00am CST on January 13, we immediately started seeing very strange errors.  Some of these errors were coming in the fashion of ColdSpring returning improper service object bean types, Mach-II throwing errors that made absolutely NO sense at all, and errors in code that was too simple to fail!  Often when someone would visit a blog, someone else’s content would show up.  (not cool!)  It began looking like we had some serious threading issues.  Immediately we discovered one issue in which the getBean() method of ColdSpring (1.x?) was not thread safe!  (Here is an email thread discussing it).    We also realized that we were using getBean() in a few inappropriate places, and moved code around so that the only place this was being used was in the bootstrapping process and we made those processes threadsafe on our own.   Even then, however, we continued to see things spiraling out of control in what became increasingly clear as a concurrency issue within Railo itself.   Gert Franz actually contacted us as soon as we brought the issue to light and wanted to help us resolve it by us sending him our application.  Given the complexity of the application, however, we felt that we didn’t have the option to spend time going down that road without knowing that we would see a solution come out of it quickly.  We made a snap decision to purchase a license of ColdFusion 8, and cut over as soon as we could.  Two nights later we made the cut-over to CF8 and all of our threading issues completely disappeared.  (blog entry from January 2008 discussing this in detail)

Immediately we breathed easier knowing that our months worth of work was actually paying off, but unfortunately this gave some “I told you so” ammo to all the previously mentioned skeptics of our use of Railo, and we were eating crow.  As it turned out, Railo put out a patch within the week that would have completely fixed the problem we had, and in hindsight, I really wish that we had the foresight to hold out and continue down that path.

Easy days

So there we were… we had our new codebase running… and running… and running… with almost zero time spent on maintenance issues.  Before too long we had amassed somewhere around 1000 blogs, with less than 5% of them being relatively active.  We couldn’t have been more pleased with the results of the rewrite as it made our lives so easy.  This period, for better or worse, allowed Aaron and I to put a huge focus on client work in our moonlighting hours.  While this was a temporarily lucrative venture, the focus was definitely off of InstantSpot since it really didn’t demand any of our attention on a daily basis.   One result of this that we essentially quit marketing the network.   Aaron and I are both really cheap dudes… I mean really… ask our wives.   Subsequently we never spent money to push InstantSpot beyond word of mouth.  We had this idea that if we made it an obvious tool, that an organic growth was inevitable.  While this was true to some extent, the flaw of this was the real industry giants that we were up against.  I mean, what had we made really?  A blogging network that from a feature perspective was a competitor against services like Google, WordPress, etc.   These giants that obviously had no idea that we even existed and were only competitors in our wildest imaginations and only on paper from a feature-by-feature perspective, rather than a true competition of any kind.  We had an obvious niche in the CFML development community, but the fact is that many developers want a more hands-on solution than what an out-of-the box solution such as a blogging network would provide.

What about revenue?

So we have never really talked publicly about InstantSpot revenue.  We spent a lot of time trying to figure out how to make money of it, but never really got it worked out right.  I mean, it paid for itself (until recently), but it never really made any money at all.  In periods of our largest traffic, we were still only making about $20.00/day.  So, with our colo fees of about 160/month, we were netting somewhere around 440.00/month.  For those that wondered why we didn’t bust our asses a little more, perhaps you understand our lack of motivation for doing so now!  It was kind of frustrating to have worked so hard, and to put so much time and effort in, for such a small monetary reward.    The thing that was really frustrating was that about 75% or more of that revenue was being generated from one very spammy blog that was set up with keywords all driving around prepaid wireless.  It felt kind of dirty, but it was the one consistent blog that kept us in the black.  Although technical blogs like mine, Aaron’s Sam Farmer’s, and others certainly brought more traffic, the click-through and click-rates on that stuff was dismal.

The decline

Over 2010, we have really seen a solid decline of anything positive from InstantSpot.  What was initially one of the HUGE benefits that we created with InstantSpot ultimately turned into a negative against us.  Our users enjoyed really ridiculous search engine exposure.  A brand new blog could be created and with a relatively well crafted title and content could easily hit page 1 of Google on a wide array of subjects.  This was due in large part to the fact that we started off with a few blogs that already had PR5 ranks, and then we used what we felt was a really smart way of linking pieces together so that new blogs made more relevant by being tied do solidly ranked blogs as they came in.   If you think I am overstating this, here are a couple of examples I can think of…

  • When Saddam Hussein was executed, we had a blogger that put up a youtube video of it ( yeah, the appropriateness  of that was definitely discussed, but we really never established a TOC that restricted specific content ).  This blog entry made page one of Google, and was picked up by El Tiempo, which is a huge latin news source, and linked as the first blog to carry the info.  We hit somewhere around 100K impressions by mid morning and our sever absolutely croaked.
  • We had a blogger that posted about State Farm Life Insurance and for a period, that blog link was higher in Google search results than statefarm.com with the search “state farm life insurance”.  That one was mind blowing to me!

There were a lot more anecdotal examples like this and we were continually shocked by what we were seeing in the way of search relevance.  Unfortunately we were not the only ones that noticed it.  We ended up with a couple of worthless spam sites that capitalized on this.  They would create a blog and then generate scores of blog entries every day with complete bullshit content stuffed with keywords and links to various products.  Over time, our valid content was overshadowed by our spam content as more and more (and more… and more… and MORE) spam sites were created.   Along with them came thousands upon thousands of spam comments.  We were pretty effective at blocking the bots, but there were a surprising number of manually typed spam comments, which were much more difficult to stop.  Eventually when you looked at our recent entries across the network, it was just shameful.   The more spam issues that came up, the more performance issues we had as well.  Unfortunately both Aaron and I were so tied up with our families and our jobs that neither one of us could deal with effectively and things just spiraled negatively.


At some point it really became quite clear that we were either going to have to invest much more effort and certainly much more money to make a real run at it, or we could just continue sailing along as long as it would go and see what happened.  It became clear to us that the kind of traffic that we would need to actually make InstantSpot something that could support us was so far beyond reality that we didn’t even know what to do about it.  Considering that both Aaron and I have full time jobs and each supports a family of 5,  quitting our jobs, taking loans and going full bore never seemed like the right decision.  “And who knows… maybe someday it will just take off!“.  Obviously, we never made that jump or commitment and both feel that we have hit “… as long as it would go”.  With the drop in revenue we have seen, we are actually spending money on InstantSpot to keep it up now.  We have experienced more and more performance issues that we don’t have the available time – and to be honest, the will – to keep things on track and at a level they should be.  After some discussions, we have decided that it’s time to pull the plug.

But, it’s not all bad

It really has been a great experience, and we have learned so much about so many things.  It has given us not only a great tool that has boosted our education and skill set, but gave us a great reason to be passionate about what we do and strive to do it well.  Hopefully we can find another project on the near horizon that gives us the same opportunity.


In closing, I would like to say thanks…  Thanks first and foremost to my family that were supportive day in and day out, and believed in what we were trying to do simply because we believed in it.  Thanks to all of the early adopters and people who stuck with us even when things were less than ideal with our system.  Thanks to many in the ColdFusion community that gave us really great support.  This has been a really great run, and I am happy to have been a part of it.

Solved: Strange sun.awt.X11.XToolkit exception with Mach-II/Railo/Tomcat/Ubuntu

I am running a BER version of Railo to experiment with the Hibernate ORM functionality for a new project.  I set up a Mach-II app from the 1.8 skeleton using Mach-II 1.8.1 on Railo under Tomcat6 on Ubuntu… whew!  That’s a mouthful huh?

The simple skeleton came up just fine, but after a little bit of customization, I ran across a strange issue.  I had created an event, in which a listener pulled a new entity from a ColdSpring bean, and persisted it using EntitySave().  Somewhere in that process, I started getting exceptions related to sun.awt.X11.XToolkit.

The first time the error would occur, I would see this:

Can’t connect to X11 window server using ‘:0.0′ as the value of the DISPLAY variable.

 /var/lib/tomcat6/webapps/railo-orm/MachII/properties/HtmlHelperProperty.cfc: line 142

    140: <!--- Configure auto-dimensions for addImage() --->
    141: <cfif StructKeyExists(serverInfo, "productLevel") AND serverInfo.productLevel NEQ "Google App Engine">
    142: <cfset variables.AWT_TOOLKIT = CreateObject("java", "java.awt.Toolkit").getDefaultToolkit() />
    143: <cfelse>
    144: <!--- Some hosts (such as GAE) do not support java.awt.* package so replace with mock function --->

On subsequent requests, I would get the following:

Could not initialize class sun.awt.X11.XToolkit again with the specitic exception pointed to HtmlHelperProperty.cfc: line 142.

After some Googling I came across a similar sounding issue in which a guy had added params to his app server.  I added it to mine, and the error went away.  If you come across this yourself, try adding: -Djava.awt.headless=true to the JAVA_OPTS (in catalina.sh for Tomcat).

Implicit constructor and CF9 style implicit accessors without ColdFusion 9

One of the broadly welcomed features of ColdFusion 9 has been the addition of implicit getters and setters to CFCs.  What this means is that you no longer have to hand code the repetitive boiler plate getXXX() and setXXX() methods for each property in your model objects. However, you don’t need ColdFusion 9, (or even ColdFusion for that matter) to enjoy the benefits of the new implicit accessors with the version 9 release.  Since ColdFusion 8, developers have had access to the OnMissingMethod() method which makes tasks like this very simple to implement on your own.  The OnMissingMethod is also available in current versions of OpenBlueDragon and Railo.

For my implementation of this, I use a standard BaseBean class in a number of my projects which is extended by all of my model class objects.  By leveraging the OnMissingMethod() in this BaseBean, I create these implicit accessors, in addition to an implicit constructor init() method as well.

There are multiple examples on the internet that show how you can use OnMissingMethod(), but for the most part, the examples that I have come across do not protect the class from being having infinite previously undefined properties added to it from the outside.  For instance, I could have a class named Person.cfc with define properties of “firstName” and “lastName”, but nothing would stop someone from doing Person.setThisIsNotAProperty(true) and that value would be dynamically added to the Person instance. While some might argue that the flexibility that this approach offers is a positive thing, I prefer to lock my classes down just a bit more.  For instance I like the fact that I can open up a model class and see exactly what properties it can contain with clearly defined <cfproperty/> tags.   From a maintainability standpoint the idea of “mystery” dynamic properties strike me as just wrong.

For my implicit constructor, I take an approach where I loop through any named arguments that were passed, and if the name matches a property that I have defined with <cfproperty/>, then it will be passed to a setter method.  Any arguments that are passed that are not defined as a property are discarded.

So let’s take a look at what this looks like.  First I will paste the entire BaseBean, and below we will break it apart to discuss what is going on.

<cfcomponent output="false">

	<cffunction name="init" access="public" output="false" returntype="BaseBean">
		<cfset var i = "" />
		<cfset initializePropertyList() />
		<cfloop list="#this.propertyList#" index="i">
			<cfif StructKeyExists(arguments,i)>
				<cfset set(i,arguments[i]) />
		<cfreturn this />

	<cffunction name="initializePropertyList" access="private" output="false" returntype="void">
		<cfset var i = "" />
		<cfif NOT StructKeyExists(this,"propertyList")>
			<cfset this.propertyList = "" />
			<cfif ArrayLen(GetMetadata(this).properties)>
				<cfloop from="1" to="#ArrayLen(GetMetadata(this).properties)#" index="i">
					<cfset this.propertyList = ListAppend(this.propertyList,GetMetadata(this).properties[i].name)>
		<cfreturn />

	<cffunction name="onMissingMethod" access="public" output="false" returntype="any">
		<cfargument name="missingMethodName" type="string" required="true" />
		<cfargument name="missingMethodArguments" type="struct" required="true" />

		<cfset var property = "" />

		<cfif ReFindNoCase("^[gs](et)",arguments.missingMethodName)>
			<cfset property = ReReplaceNoCase(arguments.missingMethodName,"^[gs](et)","") />
			<cfset initializePropertyList() />

			<cfif ListFindNoCase(this.propertyList,property)>
				<cfif StructIsEmpty(arguments.missingMethodArguments)>
					<cfreturn get(property) />
					<cfset set(property,arguments.missingMethodArguments[1]) />
					<cfreturn this />
				<cfthrow message="The class #GetMetadata(this).name# does not have a property named #property# so the implicit #arguments.missingMethodName#() method is not available." />

		<cfreturn />

	<cffunction name="get" access="public" output="false" returntype="any">
		<cfargument name="property" type="string" required="true" />
		<cfset var value = "" />

		<cfset value = variables[arguments.property] />

		<cfreturn value />

	<cffunction name="set" access="public" output="false" returntype="void">
		<cfargument name="property" type="string" required="true" />
		<cfargument name="value" type="any" required="true" />

		<cfset variables[arguments.property] = arguments.value />

		<cfreturn />


Before we walk through the actual flow, I would like to point out the method initializePropertyList() that you see here:

<cffunction name="initializePropertyList" access="private" output="false" returntype="void">
	<cfset var i = "" />
	<cfif NOT StructKeyExists(this,"propertyList")>
		<cfset this.propertyList = "" />
		<cfif ArrayLen(GetMetadata(this).properties)>
			<cfloop from="1" to="#ArrayLen(GetMetadata(this).properties)#" index="i">
				<cfset this.propertyList = ListAppend(this.propertyList,GetMetadata(this).properties[i].name)>
	<cfreturn />

By using the GetMetadata() function, we are able to introspect our instance and create a list of property names which we can reference in our other methods.  This is what enables us to enforce the rules that I described above in which we make sure that a property exists before setting it in the constructor or through an implicit accessor.  You will see in both the init() and OnMissingMethod() methods that we call this method to ensure the list of properties is available before testing against it.

With that established, let’s take a look at the init() constructor method:

<cffunction name="init" access="public" output="false" returntype="BaseBean">
	<cfset var i = "" />
	<cfset initializePropertyList() />
	<cfloop list="#this.propertyList#" index="i">
		<cfif StructKeyExists(arguments,i)>
			<cfset set(i,arguments[i]) />
	<cfreturn this />

After making sure that our propertyList has been initialized, we loop through that list and if there is a matching named argument, we call the set() method which sets that value into the variables scope.

So with the object initialized, let’s take a look at our accessors.  For our example, let’s say that we have a Person object and we are doing to set our firstName property like this: person.setFirstName(“Dave”).  Since that setter method doesn’t exist in our Person class, the OnMissingMethod() below will be invoked.

<cffunction name="onMissingMethod" access="public" output="false" returntype="any">
	<cfargument name="missingMethodName" type="string" required="true" />
	<cfargument name="missingMethodArguments" type="struct" required="true" />

	<cfset var property = "" />

	<cfif ReFindNoCase("^[gs](et)",arguments.missingMethodName)>
		<cfset property = ReReplaceNoCase(arguments.missingMethodName,"^[gs](et)","") />
		<cfset initializePropertyList() />

		<cfif ListFindNoCase(this.propertyList,property)>
			<cfif StructIsEmpty(arguments.missingMethodArguments)>
				<cfreturn get(property) />
				<cfset set(property,arguments.missingMethodArguments[1]) />
				<cfreturn this />
			<cfthrow message="The class #GetMetadata(this).name# does not have a property named #property# so the implicit #arguments.missingMethodName#() method is not available." />

	<cfreturn />

We are then doing a regular expression test to see if the missing method names starts with either “get” or “set”.  If so, we derive the name of the target property by stripping the “get” or “set” off of the string.  After making sure that our propertyList has been defined, we then look in that list to see if the target property actually exists in the instance.  If it does, the request is then routed on to either the getter or the setter depending on whether arguments were passed to it.  Otherwise an exception will be thrown to let the developer know that he or she has attempted to access an undefined property.

One thing you might notice is that when our conditional block routes the request to the setter, we return an instance of the class itself.  This allows us to do method chaining like this:  person.setFirstName(“Dave”).setLastName(“Shuck”).  it should be noted that this is not the approach that Adobe took with ColdFusion 9.

So let’s take a look at it in action.  Here is our Person.cfc class.

<cfcomponent output="false" extends="BaseBean">

	<cfproperty name="id" type="string" />
	<cfproperty name="firstName" type="string" />
	<cfproperty name="lastName" type="string" />


As you can see in the <cfcomponent/> tag, we are extending our BaseBean which is all that we need to have a workable domain class object.

When we put this together and access it in our code, we can do this:

<cfset person = CreateObject("component","Person").init(id=7,firstName="Dave", lastName="Shuck") />

<cfdump var="#person#" />

Once we have an instance, we can modify those properties like so:

<cfset person.setId(8).setFirstName("Johnny").setLastName("Rotten") />

I have not actually tested this on Railo, but I can confirm that it works in OpenBlueDragon (including GAE version), and ColdFusion versions 8 and 9.

Setting up Apache, OpenBD, Railo and ColdFusion – Part 3 – Installing ColdFusion and customizing the connector

This is the third part in a series of posts on setting up ColdFusion, OpenBlueDragon, and Railo all on the same machine using Apache webserver to listen for all requests and direct traffic.  Part 2 can be found here.  You will find links to all parts of this series at the bottom under “Related Content”

With that said, this part in the series focuses on installing ColdFusion and configuring the webserver adapter in such a way that only specific Virtual Hosts will be set up to pass requests to the ColdFusion server. First we want to start by creating a localhost site that is specific to ColdFusion 8 in Apache.  I tend to keep all my sites under /www which is actually symlinked to a ‘www’ in my home directory. I find a few benefits in this.  First, I usually keep my home partition in shape and carry it around with me from distro to distro, so I always have my sites in tact with me.  Secondly since it is a symlink to my home and not in a system folder I don’t need special permissions to write in it.  I will be following along this path, but if you keep your sites elsewhere, then you can adjust as necessary.

$ mkdir /www/localhost.cf8

Now, we will want to create an Apache virtual host definition for this site.  To do so, create a new configuration file like this:

$ sudo gedit /etc/apache2/sites-available/localhost.cf8

Paste the the following, then save and close.

<VirtualHost *>
	ServerName localhost.cf8

	<Directory /www/localhost.cf8/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	DocumentRoot /www/localhost.cf8

Now we need to make sure that file is included when Apache starts, so run the following command which will make a symlink to the sites-enabled directory.

$  sudo a2ensite localhost.cf8
Site localhost.cf8 installed; run /etc/init.d/apache2 reload to enable.

As you can see from the localhost.cf8 configuration file, we are expecting our ServerName to be localhost.cf8, so we now need to add that entry to our hosts file.

$ sudo gedit /etc/hosts

Append “localhost.cf8″ (with no quotes!) to the line that begins with  Save the file and close. At this point you should be able to restart Apache and hit that empty site in your browser like this: Now that we have a site ready, it is time for us to install ColdFusion.  Once you have downloaded the .bin installation file from Adobe, browse to that directory in a terminal window.  You may need to chmod the file to be executable, then launch it like this:

$ chmod +x coldfusion-801-lin.bin
 sudo ./coldfusion-801-lin.bin

Choose 1 for English.  After the welcome screen, hit enter to continue.  Next type “Y” and hit enter to agree to the terms and conditions. We are now faced with the installation type prompt. For this example of setting up a development environment, we will choose option 3 “Developer Edition” For our purposes we are going to choose the “Server configuration” option.  However, it should be noted that you could quite easily choole the J2EE WAR file option and install into Tomcat as we did with OpenBlueDragon in Part 2 of thise series. Since we are installing from scratch, we will choose “No” (2) on the next option which is asking if there is an existing version of ColdFusion 8 installed on this machine. For this installation we do not want any of the extra options to install such as Documentation, LiveCycle, Search Services, nor do we want to start on system init since this is on my laptop and I may not always want ColdFusion to start at boot.  So we will uncheck all options like you see in this image and continue. We are going to accept the default installation path of /opt/coldfusion8 Again, since this is a fresh installation, we are going to say “No” (2) to the prompt asking if there are earlier versions of ColdFusion on this computer. Now we start to work our way into the webserver configuration which will tie ColdFusion into Apache.  To start, choose “Add Web Server Configuration” (1) when prompted, then choose “Apache” (1).  In the following prompt asking for the Apache directory that contains your http.conf file, enter /etc/apache2 as you see in the picture below: For the location of the Apache program binary file, enter /usr/sbin/apache2.  For the Start/Stop script, enter /etc/init.d/apache2.  You will see both of these choices here: We now return to the first webserver configuration menu and this time choose “Continue with installation” (4). Then we will need to enter the location of the webroot.  We are going to enter the directory of our new site /www/localhost.cf8 that we created above as you can see in the picture below.  We will also be asked which user we would like ColdFusion to run as.  I typically like to use my own user account for this so I don’t end up with permissions issues where I have difficulty accessing files generated by ColdFusion Now enter your admin password, and if you choose to user RDS, enter the  password for it as well. It will now show you the general options you have selected and you will hit enter to start the actual installation.  Once it completes and prompts you to “Press Enter to exit the installer”, do so and then start the server with this:

$ sudo /opt/coldfusion8/bin/coldfusion start

During this stage you should see messages that it is configuring the webserver successfully, assuming we entered everything properly above.  Once you are returned to a prompt, it is time to go customize the Apache connector stuff just a bit.  open the httpd.conf file like this:

$ sudo gedit /etc/apache2/httpd.conf

In that you will find the connector stuff that ColdFusion added starting with a line: #JRun Settings. Before we take the next steps, let me explain a bit what we are going to do.  We are going to seperate this into two pieces: the module loader, and the actual connector declaration.  As it stands right now, and request coming through the webserver would be handled by ColdFusion and we don’t really want that.  To make it more granular, we are going to create an include file that can be added to any site that will add the connector only to that particular site.  With that said, let’s walk through it.   *Cut* (not copy!) these two lines of that connector info:

# JRun Settings
LoadModule jrun_module /opt/coldfusion8/runtime/lib/wsconfig/1/mod_jrun22.so

Now we are going to paste those into a new file that will go in our mods-available directory.  Click on the “New” button in gedit and paste that text in.  Now save that file as  /etc/apache2/mods-available/cf8.load. Now *cut* (not copy!) the remainder of the connector out that looks like this:

<IfModule mod_jrun22.c>
    JRunConfig Verbose false
    JRunConfig Apialloc false
    JRunConfig Ignoresuffixmap false
    JRunConfig Serverstore /opt/coldfusion8/runtime/lib/wsconfig/1/jrunserver.store
    JRunConfig Bootstrap
    #JRunConfig Errorurl url <optionally redirect to this URL on errors>
    #JRunConfig ProxyRetryInterval 600 <number of seconds to wait before trying to reconnect to unreachable clustered server>
    #JRunConfig ConnectTimeout 15 <number of seconds to wait on a socket connect to a jrun server>
    #JRunConfig RecvTimeout 300 <number of seconds to wait on a socket receive to a jrun server>
    #JRunConfig SendTimeout 15 <number of seconds to wait on a socket send to a jrun server>
    AddHandler jrun-handler .jsp .jws .cfm .cfml .cfc .cfr .cfswf

Create a new file named /etc/apache2/cf8connector. Pasted that text in and save the file. Lastly, you may not have this issue, but I found that the DirectoryIndex attribute of Apache was not adding index.cfm to the list of default files. Due to this I actually added it to my httpd.conf. After all of the steps above, my httpd.conf file only has a single line:

DirectoryIndex index.cfm

Now… it is time to do just a little more Apache config and we are ready to go!  Rember that cf8.load mod file we created earlier?  Let’s turn that on:

$ sudo a2enmod cf8

Now, let’s to add an include call to our localhost.cf8 virtual host configuration.  Open up /etc/apache2/sites-available/localhost.cf8 and add the “Include c8connector” line that you see below:

<VirtualHost *>
        ServerName localhost.cf8

        <Directory /www/localhost.cf8/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        DocumentRoot /www/localhost.cf8
        Include cf8connector

Once you have saved that file, it is time to restart Apache and test out all of our hard work!

$ sudo /etc/init.d/apache2 restart

Now, using your browser, go to http://localhost.cf8/CFIDE/administrator/ and you will see the following: Now… if you are following along from earlier in this series, you should now be able to successfully connect to OpenBD at http://localhost.bd and ColdFusion 8 at http://localhost.cf8 On the next part of this series, we will bring Railo into the mix and finally have all three running on our system.

Setting up Apache, OpenBD, Railo and ColdFusion – Part 2 – Installing Tomcat/Apache/OpenBD

This is the second part in a series of posts on setting up ColdFusion, OpenBlueDragon, and Railo all on the same machine using Apache webserver to listen for all requests and direct traffic.  Part 1 can be found here

Remember, the steps below have some commands specific to Linux, and more specifically to Debian/Ubuntu, but the concepts in general should have at least some similarites across any supported platform, especially you Mac folks.

To start out, we need to make sure that we have the Apache webserver installed with optional “dev” package.  Additionally, later on we will need to be compiling, so let’s make sure that you have the build-essential package as well

$sudo apt-get install apache2 apache2-threaded-dev build-essential

Next I installed the Sun Java 6 JDK

$sudo apt-get install sun-java6-jdk

Next you want to go download Tomcat.  I chose to use Tomcat 6, specifically v. 6.0.18, which the current release as of this posting.  Now, using a terminal cd into the directory where you saved the dowloaded file and do the following:

$ sudo cp apache-tomcat-6.0.18.tar.gz /opt/
$ cd /opt
$ sudo tar xvzf apache-tomcat-6.0.18.tar.gz
$ sudo mv apache-tomcat-6.0.18 tomcat6
$ sudo rm apache-tomcat-6.0.18.tar.gz

Next we will need to edit the Tomcat startup script, but to do so, we need to go get a little information first.  We need to ensure that we know what the current Java home is on your machine.  There are surely easier ways of accomplishing this, but here is the series of steps I took.

~$ which java
$ ls -l /usr/bin/java
lrwxrwxrwx 1 root root 22 2009-01-30 23:00 /usr/bin/java -> /etc/alternatives/java
$ ls -l /etc/alternatives/java[return] lrwxrwxrwx 1 root root 36 2009-01-30 23:00 /etc/alternatives/java -> /usr/lib/jvm/java-6-sun/jre/bin/java 

From that last line I can see that the default Java lives at /usr/lib/jvm/java-6-sun/ (which too is a sym link, but that’s ok!). Now that we have our Java home we can edit the Tomcat startup script

$sudo gedit /opt/tomcat6/bin/catalina.sh

Paste the following lines in just after the big comment block at the top. Make sure that if your Java path looked different than mine did, you will want to adjust accordingly.

JAVA_OPTS="-server -Xms1024M -Xmx1024M -XX:PermSize=256m -XX:MaxPermSize=256m 
    -Duser.language=en -Duser.country=US -Dfile.encoding=UTF-8 

Now we want to download the Apache Tomcat connector source so that we can pass requests from Apache webserver to Tomcat. This is the one I grabbed: http://apache.org/dist/tomcat/tomcat-connectors/jk/source/ Next you will want to browse to the directory that you downloaded that file into using in the terminal. Next run:

$ tar xvzf tomcat-connectors-current-src.tar.gz
$ cd tomcat-connectors-1.2.27-src/native
$ ./configure --with-apxs=/usr/bin/apxs2
$ make
$ sudo make install

Now we will need to create a jk mod file to be included by Apache so that it loads the adapter when Apache starts. In your terminal cd to /etc/apache2/mods-available. In this directory we will create a file named jk.load.

$sudo gedit jk.load

In that file pasted the following, then save and exit:

LoadModule jk_module /usr/lib/apache2/modules/mod_jk.so
# Where to find workers.properties
# Update this path to match your conf directory location (put workers.properties next to httpd.conf)
JkWorkersFile /etc/apache2/workers.properties
# Where to put jk shared memory
# Update this path to match your local state directory or logs directory
JkShmFile     /var/log/apache2/mod_jk.shm
# Where to put jk logs
# Update this path to match your logs directory location (put mod_jk.log next to access_log)
JkLogFile     /var/log/apache2/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel    info
# Select the timestamp log format JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

Now we want to create a symbolic link to this file in /etc/apache2/mods-enabled so that it is loaded when Apache starts.

$ cd /etc/apache2/mods-enabled
$ sudo ln -s ../mods-available/jk.load ./jk.load 

Next we want to create a virtual host in Apache to catch the requests. We are going to create a new file in the sites-available directory.

$ cd /etc/apache2/sites-available
$ sudo gedit localhost.bd 

In that file, paste the following, save, and exit:

<VirtualHost *>
	ServerName localhost.bd
	JKMount /* worker1

Now we need to create a symbolic link to that file from sites-enabled so that it is available when apache2 starts.

$ cd /etc/apache2/sites-enabled
$ sudo ln -s ../sites-available/localhost.bd ./001-localhost.bd

Next we want to make sure that we can get to that site in a browser, so we are going to add ‘localhost.bd’ to our hosts file. Open up /etc/hosts, and append ‘localhost.bd’ (without quotes!) to the line that begins with Now we need to define that “worker1″ that we just referenced above in the VirtualHost definition. We will do that by creating a workers.properties file in our apache2 directory.

$ cd /etc/apache2
$ sudo gedit workers.properties

Insert the following. Then save and exit:

# Define 1 real worker using ajp13
# Set properties for worker1 (ajp13)

Next we want to create a webapp in Tomcat for Apache to send to.

sudo gedit /opt/tomcat6/conf/server.xml

In that config, we need to define the localhost.bd site, or ‘webapp’. Add the following:

<Host name="localhost.bd"  appBase="webapps"
      unpackWARs="true" autoDeploy="true"
      xmlValidation="true" xmlNamespaceAware="false">
	<Context path="" docBase="localhost.bd/" reloadable="true" privileged="true" antiResourceLocking="false" anitJARLocking="false">

Now we need to create the local.bd directory under Tomcat. Note: as you get more comfortable and adventurous, you may want this to be a symbolic link to another place on your machine where your source resides, but for now we are keeping it as simple as we can. Once the directory is created, we will download the OpenBlueDragon war file and extract it.

$ sudo mkdir /opt/tomcat6/webapps/localhost.bd
$ cd /opt/tomcat6/webapps/localhost.bd
$ sudo wget http://openbd.viviotech.net/downloader.cfm/id/64/file/openbd.war
$ sudo jar xvf openbd.war 

And with that step, and after restarting Apache, and starting Tomcat, OpenBlueDragon will now be available.

$ sudo /etc/init.d/apache2 restart
$ /opt/tomcat6/bin/catalina.sh start

Open up your browser and go to: http://localhost.bd/bluedragon/administrator

You should be created with your OpenBD admin page like this:

Setting up Apache, OpenBD, Railo, and ColdFusion – Part 1

This is Part 1 of a muilt-part blog post demonstrating how run OpenBlueDragon, Railo, and ColdFusion all on the same machine, and all using the Apache webserver with individual Virtual hosts using different CFML engines.  But, before we get into it, here is a little background.

For the past several months now I have stepped over to Windows on my laptop after years of not using it regularly.  It was actually the first time I had actually used Vista, actually and was quite an interesting experience.  First, as much as I love the Linux environment, I really expected to loathe being in Windows daily.  I was surprised at how much Vista *didn’t* suck.  With all the raging passion against it in general I suppose that I had low expectations, but nevertheless I really thought it was pretty decent in general.  However it has a more sluggish, constricting feel to it in comparison to Linux, so I have decided I have paid my dues and it is time to go back to using an OS that is truly fun to use, rather than one that just wasn’t as crappy as I thought it would be.

After lots of experimentation with various flavors, hands-down Ubuntu is the most painless and most comfortable Linux distros for me personally.  My reasoning for that is vast and probably belongs in another blog entry, so I will attempt to keep from straying any further off the topic!  That said, last night I decided to try gOS which is a really neat distro built off of Ubuntu.  Although the UI is Gnome based, it has more of a Max 10.5.x feel to it.  I think Mac folks would feel right at home taking this environment for a spin.

Immediately after the first boot of my shiny new OS I started trying to put my development environment back together.  I decided that I wanted to have ColdFusion, Railo, and OpenBD all on from the beginning, with all requests first passing through Apache httpd.  Quite some time ago, Aaron Lynch put together some steps to install Smith Project w/Tomcat/Apache, which we have used several times since, primarily setting up Railo. I am not sure I ever would have waded my way through it without his early experimentation and documentation.  On this iteration, I loosely followed those instructions, opting for several packages from the repos, and updated versions of software, and have documented my steps as the are somewhat different in areas. For my current environment, I first installed OpenBD.  I then followed this by installing ColdFusion and setting it up to user the default JRUN connector that is set up during the installation.  Lastly, I installed Railo as another webapp in Tomcat and tied that into Apache as well.

In the following posts, you will see these steps in detail.

Setting up Apache, OpenBD, Railo and ColdFusion – Part 4 – Installing Railo under Tomcat with Apache webserver

This is Part 4 in an ongoing series demonstrating how to install OpenBlueDragon, ColdFusion, and Railo concurrently with all requests passing through the Apache webserver.  You will find links to all parts of this series at the bottom under “Related Content”

In this post we will be building upon our existing environment in which we installed Apache webserver, Tomcat 6, OpenBlueDragon, and ColdFusion, by adding Railo to the mix.  If you have been following along thus far, we are well past the hard stuff at this point.  To install Railo, which is deployed as a java war file, we are going to repeat and slightly modify many of the steps that we took at the end of Part 2 when we installed OpenBD as a Tomcat web application.

Let’s start by setting up our hosts entry so that our new site resolves to localhost.railo, and set up the Apache webserver to listen for the request.  Open up /etc/hosts, and append ‘localhost.railo’ (without quotes!) to the line that begins with   Next we want to create a new virtual host file for the site under Apache.  Do so like this:

$ cd /etc/apache2/sites-available
 sudo gedit localhost.railo

Paste the following into that new file and save it.

<VirtualHost *>
	ServerName localhost.railo
	JKMount /* worker1

As we learned in Part 2, the line “JKMount /* worker1″ is instructing apache to pass everything to the worker we defined in /etc/apache2/workers.properties.  That worker will then pass the request on to Tomcat port 8009, under which our application actually resides.

Now that we have defined this site, we want to make sure that it gets included on our next Apache restart.  We do this by running the following:

$ sudo a2ensite localhost.railo
Site localhost.railo installed; run /etc/init.d/apache2 reload to enable.

At this point, we need to make sure there is a an application set up in Tomcat to receive our request.  First, let’s start by downloadingthe free Railo Server from their download page.  Look toward the bottom of the page and download  “Railo Custom” and download railo- (or newer version if it is there).  Once we have the file downloaded, we will create a new directory for our application under Tomcat.  We will then copy the Railo war file into it from our download directory.  When we have the war file under /opt/tomcat6/webappas/localhost.railo/ we will extract it, and then safely delete the war file itself.  These steps can be seen here:

$ sudo /opt/tomcat6/webapps/localhost.railo
 sudo cp railo- /opt/tomcat6/webapps/localhost.railo/
 cd /opt/tomcat6/webapps/localhost.railo
 sudo jar xvf railo-
 sudo rm railo-

Since we now have the application in place, it is time to modify the Tomcat server config so that it knows to send the appropriate requests to it.  Do the following:

$ sudo gedit /opt/tomcat6/conf/server.xml

Look for the section that we added previously for our OpenBlueDragon localhost site “localhost.bd”.  We are going to copy that section, modifying the “name” and “docBase” attributes to match our new application and paste it just below the localhost.bd site.  When you are done they should look like this:

<Host name="localhost.bd"  appBase="webapps" unpackWARs="true"
	autoDeploy="true" xmlValidation="true" xmlNamespaceAware="false">
	<Context path="" docBase="localhost.bd/" reloadable="true" privileged="true" antiResourceLocking="false" anitJARLocking="false" />

<Host name="localhost.railo"  appBase="webapps"  unpackWARs="true"
	autoDeploy="true" xmlValidation="true" xmlNamespaceAware="false">
	<Context path="" docBase="localhost.railo/" reloadable="true" privileged="true" antiResourceLocking="false" anitJARLocking="false" />

With all that done, we are ready to restart (or start) Apache, Tomcat and our new Railo site will be available at http://localhost.railo

So with our original goal of being able to run the three CFML engines, it’s time to see how we did….

Free as in money, not as in pain – InstantSpot moves to ColdFusion 8

As I mentioned in several previous blog entries, on January 12 we re-launched InstantSpot after a complete bottom-to-top rewrite. In addition to a completely new code base, we took the unlikely choice of using Railo as our CFML processing engine.

“Why?” you ask? (You aren’t the first)

The reasons were several, and I will detail a few of the key points that went into our decision.

  • It’s free – InstantSpot is basically a small project of big ideas by two developers doing this out of our own pockets and – how can I put this delicately? – we are poor and cheap! Unfortunately, despite how much we will it to be so, InstantSpot has not made us bazillions of dollars (at least as of the time of this posting). From the beginning we have made an effort not to make InstantSpot a financial burden on our families, as they are already paying dearly in the time we spend tied up in code till all hours of the night, and we like to cut financial corners everywhere we can. A free CFML processing engine? That is an obvious avenue to at least explore.
  • It’s fast – No lie… Railo is fast. From our very first development and tests, it just seemed to blow other engines away in the speed that it processed code. This was backed up test after test. Not only does it shine in the speed of processing code, but it also has a tiny footprint on the server. In our environment running it as a Tomcat application, you almost wouldn’t even guess it was there.
  • It’s CFMX 7 compatible – To us this meant that we didn’t have to code anything differently simply because we chose to use Railo over ColdFusion. We had no issues whatsoever using our normal data model patterns we use in any other application, and we used BER releases of ColdSpring and Mach-II without the slightest hiccup. Eventually we found a couple of small places where we had to make a workaround (3 I can think of), but they were without question edge cases, and there were easy workarounds that didn’t feel as though we were compromising the application.
  • It’s the underdog – If you were to poll the ColdFusion community at large, you would find that many people don’t even know there are any other choices besides Adobe ColdFusion, and many of those may have only heard of New Atlanta’s Blue Dragon. Railo hardly gets a mention in most circles. We thought it might be fun to be an advocate by example and help promote what we felt is a great alternative choice. Additionally, Aaron and I have a tendency to choose the road less traveled, but this certainly fit our m.o.

Sounds reasonable right? Since we made that choice around June of 07 and started moving forward with the rewrite, we have felt overwhelmingly positive about our decision.

All of that began to change about 1:00am January 13.

After making the DNS changes and as we started seeing traffic redirect to the new server, we started seeing absolutely inexplicable errors. The closer that we examined them, it became obvious that we had some *serious* threading issues in our application. We are extremely careful in this regard when it comes to our code, so this was very surprising. However, this *was* brand new code, and of course there could have been a hole somewhere right?

As more traffic started coming in, the errors escalated. We started seeing errors at least every minute, each of which generated a painful new email to both Aaron and me. It became clear quite rapidly that the errors actually had nothing to do with the code. We started seeing errors from both Mach-II and ColdSpring that just simply couldn’t happen. For instance , here is one we started seeing from ColdSpring:

Message	variable [beandefinition] doesnt exist
Tag Context	/www/instantspot/www/coldspring/beans/AbstractBeanFactory.cfc (211)

Really? That is pretty interesting since line 210 is:

<cfset var beanDefinition = getBeanDefinition(arguments.beanName) />

And how about this one from Mach-II?

Message	variable [nextevent] doesnt exist
Tag Context /www/instantspot/www/MachII/framework/RequestHandler.cfc (115)

Oh yeah? Well… this is line 114:

<cfset nextEvent = appManager.getEventManager().createEvent(result.moduleName, result.eventName, eventArgs, result.eventName, result.moduleName) />

Clearly even our worst var-scoping misstep couldn’t have created those errors, and furthermore, these are well-tested frameworks used in hundreds if not thousands of applications. If these threading errors existed in them, Aaron and I would not be the ones discovering it in January 2008. We were also seeing some of our own objects that were attempting to call methods of other objects, and obvious sign of serious threading issues. In two instances, a person’s RSS feed actually contained someone else’s content.

We began to wonder if Railo even recognized var-scoping at all? I pulled up an old blog entry that I had made in which I wrote some code examples that showed an easy example of a var-scoping error and ran it against Railo. I set up a Railo scribble pad, and ran the test. It did pass that test, which tells me that Railo at least manages var scoping on a cursory level. However, under the load of our application it appeared that we were looking at something bigger than just var-scoping a few object methods.

At this point, there was no longer any question that in order to get out of this tailspin we needed to do something drastic. We quickly decided that the most logical step was to switch to Adobe ColdFusion 8. The cheap-gene which is so deeply embedded in our DNA had to be thrown out the window, and we had to act by getting a ColdFusion 8 license and getting it implemented asap. One immediate concern that came to mind was how much we had modified the Railo WEB-INF in order to do some of the URL handling that we had implemented as we not only use mod_rewrite in Apache, but we also use another Java application in the mix as well. After installing ColdFusion 8 Standard and digging into the /wwwroot/WEB-INF, we found that we could painlessly apply the same pieces to our ColdFusion application, and with some very small changes, we had InstantSpot running in our development environments.

After doing some heavy but rapid testing throughout all of our application, we felt that we could make the switch. Even if an error or two was discovered later, the benefits would strongly outweigh the utter nonsense we were dealing with at that time. So around midnight last Wednesday night, we pushed up the ColdFusion implementation of our application, crossed our fingers, held our breath, flipped the switch, and……

….. silence.

After the application initialized, suddenly there was peace… no errors… no emails… just an application purring along as it was intended. In fact, we had to push up a test template with broken code to ensure that our error notification was still working. Since that time we have not seen a single error occur in our application with well over 100K page requests since the move.

I want to be clear that this post is not meant to be an attack on Railo. I am sure that Gert and crew work extremely hard and I tend to believe that Railo will mature to a nice alternative if they keep up their efforts that they have shown to date. However I do hope that this post serves as a warning as we found that there are huge implications with using it as it stands today.

Railo, Structures, Keys and Case

After hundreds and hundreds of development hours, Aaron and I are on the verge of releasing the completely rewritten InstantSpot to beta this week.  I plan on a more detailed post of the technology later, but in a nutshell, we are releasing it as a Linux/Apache2.2.x/Tomcat/Railo2/Mach-II application.

Until today, we have found only one place that we have had to make a Railo workaround and that was using some pretty fringe functionality doing XMLRPC for desktop blogging.

Last night I ran into the second point where we have to make a workaround, although from an architecture standpoint, again it is pretty slight.  This time, it came about as I was changing our Mach-II config to use the ColdspringProperty rather than the ColdspringPlugin.  When I did, the application died a horrible death with errors stating that it couldn’t find our imported ColdSpring XML files that are defined in our main ColdSpring XML.

After tearing into it this bit, I found that the DefaultXMLBeanFactory.cfc calls a recursive method to build a structure of the main config XML, with additional keys for each imported XML file which are the full physical path to the file itself.   I could see that just prior to adding to the structure, the case was correct.  However, when I look at the key that was created, it was converted to lower case.

Later in the process as the Mach-II ColdspringProperty.cfc tries to loop through these keys, the config files cannot be found if there are any capital letters that exist in the real path.

I then tried modifying the ColdSpring code to use structure bracket notation for adding the key, rather than using StructInsert(), since this is one known way of preserving key case in ColdFusion.  However, I found that no matter what I did, the structure key that is created is all lowercase.

Stepping back a bit, I decided to do a simple test which confirmed this functionality:


Here were the results:

number 3
number 2
number 1

As you can see, no matter which approach I took, the key is lowercase.

So, what does this mean for our release?  Not much… back to using the ColdspringPlugin.  However, it is something that I hope the fine developers of the Railo project keep in mind as a future fix.