KCAIUG Meeting Notes - Sept 2014

The quarterly Kansas City Arc Info Users Group (KCAIUG) meeting yesterday was hosted by Stantec in Overland Park, KS. It was encouraging and enlightening, and I took quite a few notes. There were 2 presentation parts, which were equally relevant to some clients work I am involved with.

KC Mapping presented first, with discussion towards planning and development best practices. They emphasized the need to follow appropriate planning and development methods to ensure needs are met and addressed without complication. This is more than critical in GIS applications and analysis, where there are so many moving parts and dependencies. In short, a lack of planning or adherence to any sort of development method, is like owning a home without insurance; you may get away with it for a while, but eventually it is going to cost you. They concluded that no one development is better than the other, and that a hybrid Agile/Waterfall approach always seemed to work best, with some situations requiring more emphasis than the other.

The US Army Corps of Engineers (USACE) followed up re-emphasizing the need for appropriate planning, after a long effort of consolidating systems and processes they had in place, which overlapped with those of Federal Emergency Management Agency (FEMA).

Years ago, FEMA recognized the usefulness of USACE data, which they began replicating across numerous databases for their own use, in their own environment. Similar and sometimes exact copies of applications, systems, databases, and processes within the organizations, were leading to massive expenditures on storage and the personnel needed to maintain it all. The lower ranks within FEMA and USACE recognized that they were unnecessarily duplicating efforts within the USACE National Levee Database (NLD) and the FEMA Midterm Levee Inventory (MLI), and quietly initiated a bit of inter-agency collaboration to see how they might relieve some of the burden this short-sighted approach was creating.

The large-scale destruction witnessed from hurricanes in 2007, and the associated awkward responses from ranking officials within both groups, prompted Congress to mandate collaboration between the two organizations. The door was opened, and the stage set for a project that would integrate MLI with NLD. The new integrated backend and dynamic data model eliminated the error-prone and disconnected situation replication and non-cooperation had produced, enabling consolidated applications, systems, and storage, benefiting both organizations on all levels.

Currently, they are working on (and demonstrated) a web application that will be consumed by individuals in the office, in the field, and the general public. It will provide for greater efficiency of systems, higher data accuracy, further reducing workloads. The solution is being produced through a mash-up of open-source solutions, such as MapServer and OpenLayers2, combined with the recognized solutions of Java and Oracle Spatial, providing for a more flexible solution that outperforms a straight-line ESRI solution. This final part is a prevailing theme that I have recognized in most recent approaches to GIS applications; especially, web applications.

In summary… always plan and follow some sort of method, collaborate against the odds, and don’t put all your eggs in one basket.

Sears Delivery Sux

We thought that having Sears deliver our new bedding would be easier that trying to arrange than to pickup ourselves. We were very wrong. The delivery date turned out to be a fiasco, and a test of our patience and understanding of stupidity.

It all started about 1 pm in the afternoon….

1302 — Missed call from (Delivery Phone Number 1)

1358 — Message from (Delivery Phone Number 1) — The delivery driver is 2 hours ahead of schedule, and would like to deliver.

1359 — Missed call from (Delivery Phone Number 2)

1400 — Missed call from (Delivery Phone Number 2)

1406 — Message from (Delivery Phone Number 2) — The driver is 2 hours ahead of schedule, and would like to deliver.

1414 — Message from Sears Main Delivery Number — The delivery driver is 2 hours ahead of schedule and would like to deliver.

1435 — Message from (Delivery Phone Number 3) — The delivery driver is 2 hours ahead of schedule and would like to deliver.

1600 — I received all messages, when I arrived at home at the scheduled time for delivery!

1605 — I called the (Delivery Phone Number 3) number back, and the person indicated that he had wanted to deliver 2 hours early, but was unable to reach me. I explained that I was not able to access my phone, but was here at the scheduled time, awaiting delivery. He indicated that he was on his way to deliver, sounding a little put out, and that he had come by and nobody was home. I told the delivery driver that I was here at the scheduled delivery time and would be here. He said he would call me back in a few minutes with an ETA.

1610 — I called Sears back and the CSR explained that the delivery driver was now running a couple hours behind, but the delivery would still occur within the delivery window, but the delivery driver would call me in a few minutes to let me know an ETA.

1800 — I called Sears, asking if the delivery was still happening, and what was the ETA. The CSR “Cassidy” put me on hold and came back on the line to explain that they spoke with the delivery driver and they should be calling me in a few minutes with the ETA.

1900 — I called Sears, explained this story, and asked about the ETA, also explaining that I had already called 3 times now, and that nobody had bothered to call back. The CSR “RJ” put me on hold and came back on the line to explain that they spoke with the delivery driver and they should be calling me in a few minutes with the ETA.

1930 — I called Sears, explained this story and asked what was the ETA, also explaining that I had already called 4 times now, and that nobody had bothered to call back. The CSR on hold and came back on the line to explain that there was a note that said nobody was home when the attempted deliver. I told her that was not true and that I would like to speak with a manager immediately. I was placed on hold for another 5 minutes only to be told that no manager was available. I told her that I would wait on hold for a manager, and was placed on hold again for 25 minutes.

A manager (Gani) finally answered.

As I was waiting, I had called the delivery warehouse number (Warehouse Delivery Coordinator) again on another phone. The individual I spoke with earlier indicated that the driver said he had tried to deliver, but we were not home. We were home at 4 pm!!! I went through the whole story again, and he said he would see what he could do and would call me back in a few minutes.

I returned to the conversation with Gani, who indicated he was going to get in touch with delivery services and see if he could do something about this, but that I should wait on the delivery warehouse to call me back.

2030 — Spoke with (Warehouse Delivery Coordinator) and the person provided nothing but excuses. He indicated that sometimes their delivery drivers are running early, and it was my fault for not answering the phone when I supposed to. At 4 pm, my delivery was checked back in to the warehouse. I asked why it was checked in to the warehouse at 4 pm, when it was supposed to be delivered at 4 pm. He yelled that the delivery driver tried to deliver, but I was not home. ( I was home at 4 pm!!!) He said he was trying to work something out, but I was not listening. I told him he was giving me nothing but excuses for not making the delivery at the time specified. He said he would do what he could and even “deliver the damn thing” himself, if he had to.

2100 — Gani called back and asked if I heard back from (Warehouse Delivery Coordinator). I indicated that I had and that I was not happy with the situation at all. He said the best he can do is have somebody from the warehouse call me sometime in the next 24 hours.

2130 — (Warehouse Delivery Coordinator) called and said that he was at the warehouse and was going to make the delivery himself. I asked if it was a result of the Customer Service group. He said he was doing this on his own, and had not heard from Customer Service at all.

2230 — Personally delivered by (Warehouse Delivery Coordinator)

First Environmental Steps - Restoring a County GIS (Part 3)

Upon taking on the position of GIS Manager, I spent the first few days familiarizing myself with the hardware and software currently in place, and what licensing was available on the same. I also took some time to meet with all of the various Department Heads and Officials I would be working with to get a better grasp of the political structures, boundaries, and my role in all of it.

The first of those meetings was with Information Services (PCIS). As a department of the County, it was made clear that while PCIS sometimes provided assistance to the Office of the Assessor (an independent office), but were not there for support. I was a bit surprised by all of this, and their apparent desire to continue that way, but knew that was not going to work out well. There was no way anything was going to get off the ground without some collaboration with them, since they were really the only other “technical” people in the entire organization. Through various means (and likely my constant badgering), we eventually threw that time-honored and highly restrictive tradition out the window.

Another issue that came up during that first meeting outlined one of the first tasks at hand. Asking whether there was any sort of network and database models available, they first looked at me as if I had a 3rd eye, and then asked what I needed that for. After explaining that it was going to be bit difficult to maintain this structure without knowing a little bit about how things were connected around there, they proceeded to sketch it out for me verbally. Nothing was documented. The problem did not simply exist in the department that I now managed, but was endemic in the organization. I later learned that nobody bothered to document anything for one rather simple and selfish reason; job security. I resolved to change that attitude; if nowhere else, within the GIS Department, hoping others would follow the example.

Since there was so much to be documented, I was not quite sure where to start, so began with what seemed the most natural place to start, an overview of the hardware and software in use. The workstations were still on Windows XP, running ArcMap v9.1; the geo-database residing on a Windows 2003 server, running IBM DB2 v8.1 and ArcSDE v9.1. The web server was on the same version of another server, running Apache Tomcat (don’t recall the version), and ArcIMS v9.1. In contrast, ESRI has just released v9.3.1.

The web application was a custom situation ESRI had assembled for the Assessor a few years earlier. It used frames, a bit of Java and Javascript to provide real estate information for the County, through a data or map search interface. This site was dependent upon ArcSDE, as well as loads created by views to data stored in the Collector, Recorder, Planning & Zoning, and Computer Assisted Mass Appraisal (CAMA) system for the Assessor. It worked relatively well with minor intervention, but sorely needed a facelift; if nothing else, to get rid of those frames. Unfortunately, the documentation for that was also missing in action; but, later found.

Licensing was quite fortunate. We held 2 ArcEditor licenses, ArcGIS for Server Basic (for the ArcIMS v9.1), and ArcGIS for Server Enterprise. I was pleased to see the later two and immediately brought it to the attention of the Assessor, explaining the capability. She was happy to hear, and equally disappointed that none who had held the GIS Manager position previously, had attempted to implement the ArcGIS for Server she had been paying licensing fees for. We decided to make an effort to make that happen.

Finally, a Standard Operating Policy & Procedure document was needed immediately, to start documenting how things were supposed to work. One of the first line items in that document: ALL processes and procedures will be documented prior to, or at the time of implementation. While that was not an issue int he department I managed, it was a sore sticking point for the only other GIS staff in Planning & Zoning, who steadfastly refused to document anything. There was not much that could be done about that, since that position was officially in another office. All that I could do was continue to emphasize the value of doing so. It never stuck, throughout the life of the project, so I just did it.

Background - Restoring a County GIS (Part 2)

I accepted the position of GIS Manager, just as I began my final year working towards a Bachelor of Science in Physical Geography (special emphasis on GIS and karst environments, along with a Minor in History). Little did I know that everything I had learned or been a part of in the past 20 years was about to be put to use.

The position reported directly to the County Assessor, a rural Republican woman from the north end of the County. She had held the seat for about 6 years I think, after being appointed from within the Office by the Governor, following the resignation of the previous Assessor. She was hard-nosed and demanding, while at the same time perfectly clear and reasonable in her expectations. Much like me, she preferred getting down to business with straight and honest answers to questions, and had very little tolerance for folks that beat around the bush. As a result, we got along really well, after testing each other a few times, and I was determined to make good things happen for her, the Office, and the County GIS.

The previous GIS Manager had left the position about 6 months prior to my arrival and only about a year in the position. I later discovered that this was a scenario that had repeated itself numerous times in the past. GIS folks signed on to the position long enough to hold the title, then left for a position in the private sector. There had been no attempt to retain anyone in the position, through commensurate compensation, and there was no interest in doing so. Consequently, the GIS was in a crumbling and dilapidated state, and not really much use to anyone except the persons maintaining and analyzing the records.

A GIS Analyst had only recently signed on, after having been absent from the profession for several years. She was doing her best maintain the data and day-to-day business; that is, in spite of a total lack of guidance, direction, documentation, or procedure, from the previous GIS Manager. I was concerned she would follow the same path as many others that had been in that position, but was fortunate to have her give the situation a chance and stay on through 4 years of the restructuring.

Throughout the County, there were only a couple of others involved with GIS. The GIS Coordinator within the Planning & Zoning Department mostly worked independently on projects associated with his department. Although he really had no responsibility to our office, he enthusiastically joined in most of the projects we rolled out during the next 5 years. The Director of Information Services was an immense help in forwarding our objectives too. He had originally been brought on years before, to initiate a GIS for the County. He had not worked in GIS for quite a few years, but understood the basic concepts; as well as the importance of a structured and well-documented approach to building and maintaining a useful GIS.

While this situation probably would have been quite a challenge for someone just finishing college, I had enough background to make a good run of it. I had already been in the business world for 20 years and just then leaving an Analyst role I had been in for 2 years with a financial software firm. Prior to that, I spent 5 years working with a team in my own company providing creative services, primarily in photography and event production, and pre-social media web marketing and promotional support. For the 13 years before that time, I proved my technical worth, rising through the ranks of a semi-local financial firm in various technology roles from quality assurance, helping structure production standards, statistical analytics, database management, data modeling, software and hardware support, software design and development, and even a little web production in the early days of HTML.

Initial Thoughts - Restoring a County GIS (Part 1)

During the past 5 years, I spent most of my waking hours rebuilding and restructuring a dilapidated Geographic Information System (GIS) for a Platte County in Missouri.

My team and I restructured systems and data to be more efficient, reliable, and accurate, while fully documenting processes and procedures, and organizing policy towards a functioning business solution using the ESRI suite of products, ArcGIS. Indeed, we only recently positioned our GIS to enable the Assessor and the County to begin to reduce its reliance on taxation and begin generating revenue through a fee structured subscription environment, capable of delivering geographic and non-geographic data to any client in the world, 24 hours a day, 365 days per year.

Resources for GIS were extremely limited, so it was very hands on, and I did most of the work on this system, personally. Along with a GIS Analyst under my supervision, the GIS Coordinator n the Planning & Zoning Department, as well as a bit of consulting on system design, architecture, and setup support from the Information Services Department.

In spite of numerous obstacles, both internal and external, we successfully upgraded all ArcGIS software from version 9.1 through v10.2, and implemented ArcGIS for Server v10.2, while at the same time, migrating this situation from an aging IBM DB2 v8.1 database on a Windows 2003 server to a Microsoft SQL Server 2008 R2 database on a Windows 2008 server.

This multi-part series will document those adventures in full, from beginning to the bitter end, when a newly-elected Republican Assessor, closely affiliated with the Tea Party, made one of several uninformed decisions. In his opinion, an [underpaid] GIS Manager was an unnecessary expense, because “nobody really uses maps anymore;” ironically, just a few days later he realized he had made a mistake. The GIS Manager did not just make maps.

Fortunately for him, the system was well documented and permitted the GIS Analyst to easily step in and start with a good foundation. That was one of project goals, after all; that anyone could easily step in and assume command of the GIS, with little effort. 

Essentially, this is a “Post Mortem” on the PCAO Geographic Information System Project, in every sense of the word, in the hopes of providing some insight to any other GIS Manager or Project Team that may face similar circumstances.