Tuesday, December 31, 2013

ArcGIS 10.1 Performance Issues

Since we installed ArcGIS 10.1, we have been experiencing extreme performance issues. We couldn't prove it just by panning and zooming so I had to create a comparison charts to compare different environments.

​We managed to do several testing on the speed of ArcGIS Desktop layer loading between several geodatabase, compared Different Clients (10, 10.1 SP1) vs Oracle and SQL Server.​​ The hardware is relatively the same 16 GB RAM with 8 Core CPU running on Windows 2008 R2 SP1. That is except for the SQL Server DB which is running on 8 GB Windows 2008 R2 SP1. These have exactly the same configurations.
​The log file below shows only loading the a set data set with a geometric network

Scenario A: From Client ArcMap 10.1 to 10.1 Database (Oracle 11g)  
[  1   ] About to add WaterDistributionDataSet
[  1   ]  finished in 0.1 seconds
[  2   ] About to set the source of WaterDistributionDataSet
[  2   ] About to get the feature class object  WaterDistGeomNetwork_Junctions
[  2   ]  finished in 11 seconds
[  3   ] About to get the feature class object  SystemValve
[  3   ]  finished in 0.1 seconds
[  4   ] About to get the feature class object  AirValve
[  4   ]  finished in 0.1 seconds
[  5   ] About to get the feature class object  ServiceValve
[  5   ]  finished in 0.1 seconds
[  6   ] About to get the feature class object  FlowControlValve
[  6   ]  finished in 0.1 seconds
[  7   ] About to get the feature class object  PressureReliefValve
[  7   ]  finished in 0.1 seconds
[  8   ] About to get the feature class object  Fitting
[  8   ]  finished in 0.1 seconds
[  9   ] About to get the feature class object  Junction
[  9   ]  finished in 0.1 seconds
[  10   ] About to get the feature class object  Hydrant
[  10   ]  finished in 0.1 seconds
[  11   ] About to get the feature class object  Pump
[  11   ]  finished in 0.2 seconds
[  12   ] About to get the feature class object  WServicePoint
[  12   ]  finished in 0.1 seconds
[  13   ] About to get the feature class object  Meter
[  13   ]  finished in 0.1 seconds
[  14   ] About to get the feature class object  Tank
[  14   ]  finished in 0.1 seconds

Scenario B: ​From Client ArcMap 10.1 to 10 Database (Oracle 11g)  
[  1   ] About to add WaterDistributionDataSet
[  1   ]  finished in 0.1 seconds
[  2   ] About to set the source of WaterDistributionDataSet
[  2   ] About to get the feature class object  WaterDistGeomNetwork_Junctions
[  2   ]  finished in 13.4 seconds
[  3   ] About to get the feature class object  SystemValve
[  3   ]  finished in 0.7 seconds
[  4   ] About to get the feature class object  AirValve
[  4   ]  finished in 0.7 seconds
[  5   ] About to get the feature class object  ServiceValve
[  5   ]  finished in 0.7 seconds
[  6   ] About to get the feature class object  FlowControlValve
[  6   ]  finished in 0.7 seconds
[  7   ] About to get the feature class object  PressureReliefValve
[  7   ]  finished in 0.8 seconds
[  8   ] About to get the feature class object  Fitting
[  8   ]  finished in 0.8 seconds
[  9   ] About to get the feature class object  Junction
[  9   ]  finished in 0.7 seconds
[  10   ] About to get the feature class object  Hydrant
[  10   ]  finished in 0.7 seconds
[  11   ] About to get the feature class object  Pump
[  11   ]  finished in 1.5 seconds
[  12   ] About to get the feature class object  WServicePoint
[  12   ]  finished in 0.7 seconds
[  13   ] About to get the feature class object  Meter
[  13   ]  finished in 0.7 seconds
[  14   ] About to get the feature class object  Tank
[  14   ]  finished in 0.7 seconds

Scenario C: From Client ArcMap 10 to 10.1 Database (Oracle 11g) 
[  1   ] About to add WaterDistributionDataSet
[  1   ]  finished in 0 seconds
[  2   ] About to set the source of WaterDistributionDataSet
[  2   ] About to get the feature class object  WaterDistGeomNetwork_Junctions
[  2   ]  finished in 1.9 seconds
[  3   ] About to get the feature class object  SystemValve
[  3   ]  finished in 0.1 seconds
[  4   ] About to get the feature class object  AirValve
[  4   ]  finished in 0.1 seconds
[  5   ] About to get the feature class object  ServiceValve
[  5   ]  finished in 0.1 seconds
[  6   ] About to get the feature class object  FlowControlValve
[  6   ]  finished in 0.1 seconds
[  7   ] About to get the feature class object  PressureReliefValve
[  7   ]  finished in 0.1 seconds
[  8   ] About to get the feature class object  Fitting
[  8   ]  finished in 0.2 seconds
[  9   ] About to get the feature class object  Junction
[  9   ]  finished in 0.1 seconds
[  10   ] About to get the feature class object  Hydrant
[  10   ]  finished in 0.1 seconds
[  11   ] About to get the feature class object  Pump
[  11   ]  finished in 0.3 seconds
[  12   ] About to get the feature class object  WServicePoint
[  12   ]  finished in 0.2 seconds
[  13   ] About to get the feature class object  Meter
[  13   ]  finished in 0.1 seconds
[  14   ] About to get the feature class object  Tank
[  14   ]  finished in 0.1 seconds
 ​ 
Scenario D: From Client ArcMap 10.1 to 10.1 Database (SQL Server 2008 R2) [Virtual Machine]
[  1   ] About to add WaterDistributionDataSet
[  1   ]  finished in 0.1 seconds
[  2   ] About to set the source of WaterDistributionDataSet
[  2   ] About to get the feature class object  WaterDistGeomNetwork_Junctions
[  2   ]  finished in 1.5 seconds
[  3   ] About to get the feature class object  SystemValve
[  3   ]  finished in 0.2 seconds
[  4   ] About to get the feature class object  AirValve
[  4   ]  finished in 0.2 seconds
[  5   ] About to get the feature class object  ServiceValve
[  5   ]  finished in 0.2 seconds
[  6   ] About to get the feature class object  FlowControlValve
[  6   ]  finished in 0.2 seconds
[  7   ] About to get the feature class object  PressureReliefValve
[  7   ]  finished in 0.2 seconds
[  8   ] About to get the feature class object  Fitting
[  8   ]  finished in 0.2 seconds
[  9   ] About to get the feature class object  Junction
[  9   ]  finished in 0.2 seconds
[  10   ] About to get the feature class object  Hydrant
[  10   ]  finished in 0.2 seconds
[  11   ] About to get the feature class object  Pump
[  11   ]  finished in 0.3 seconds
[  12   ] About to get the feature class object  WServicePoint
[  12   ]  finished in 0.2 seconds
[  13   ] About to get the feature class object  Meter
[  13   ]  finished in 0.2 seconds
[  14   ] About to get the feature class object  Tank
[  14   ]  finished in 0.2 seconds

Notice that surprisingly​ ​SQL Server while on a virtual machine scored the best performance more than 80% of the 10.1 configuration. This depends on so many factors and configuration but apparently it proves that there is something fishy with 10.1 and Oracle. I would recommend using SQL Server since it complies much better with ESRI technology. 

Wednesday, December 25, 2013

ArcGIS for Server Web Adaptor

You may have stumbled upon this new component that ESRI added in 10.1.  You see it there, in the installation menu, not sure if you should install it or not. You don't even know what does it do. I remember skipping the Web Adaptor first time, I installed ArcGIS for Server successfully and things worked just fine without it. But that was before going to production.

Prior 10.1, ArcGIS for Server (hereafter known as just Server) depends on Internet Information Services (IIS) to publish system web sites which are required for communications between Server components. Server uses these sites to publish and manage GIS Services. However, coupling ArcGIS for Server to IIS like that created a dependency on Microsoft, which prevented ease of upgrade to other platforms.

ArcGIS for Server requires a Web Server Software in order to function. It is where the system web sites are published. That is why in 10.1, ESRI shipped the product with a free built-in Web Server (Apache TomCat 7.0.27) and allowed all system related sites to be published on that built-in Web Server. They even choose a unique port for that purpose, 6080. This allowed ESRI to simplify the installation and easily maintain one architecture across multiple platforms.

After you complete the installation, you can access your services and manage them through this port over HTTP. Assume your Server host name is GIS-SERVER

You use this link to view all services
http://GIS-SERVER:6080/arcgis/rest/services 

This will view a particular service (LANDCOVER) REST description
http://GIS-SERVER:6080/arcgis/rest/services/landcover/MapServer

This will view the service using javascript api
http://GIS-SERVER:6080/arcgis/rest/services/landcover/MapServer?f=jsapi

This gives you access to the ArcGIS for Server Manager
http://GIS-SERVER:6080/arcgis/manager

And so many other uses.

Although this approach seems clean and decoupled, there is however a problem with it. You are really limited with what you can do with this built-in web server. You don't have the luxury to modify the web server configuration, set permissions, change the port or allow authentication. Moreover, you have to enable the 6080 port on your firewall if you want to allow someone outside your network to access it. There are other security issues involved, like Cross Domain. Cross Domain is when a web site or a page requests data from another page which is not hosted on the same domain and port. If you will start publishing web applications and pointing to your services through this port 6080, you will need to enable the cross domain to allow your applications to communicate with other domains. This is a big security risk, as malicious attackers might use this hole to inject scripts on your website that communicate with a remote site and retrieve/send vital information. The Web Adaptor comes in here to fix this.

The Web Adaptor is a web site that acts like a bridge between the existing Apache built-in web servers and your own dedicated Web Server software. It redirects the traffic between the two and allow you to freely configure your Web Server without affecting the existing one. You can publish your Web Adaptor on IIS or any other Web Server using your choice of port. This is usually the default port which is 80. You can also choose to install the Web Adaptor on a separate server making it a dedicated Web Server and point to existing GIS setup. This requires you to open port 6080 between the Web Server and the GIS Server. The GIS Server is the server where you installed ArcGIS for Server.



Click to enlarge



We will discuss more about ArcGIS for Server in future posts. Stay tuned and subscribe.