The Unorthodox Way to Compress a Replication-Enabled Versioned GeoDatabase to State 0

If you landed here chances you lost hope of all the links and by-the-book ways to compress your geodatabase. You probably have a versioned geodatabase that no matter how many times you compress, no matter how many users you kick out, how many connections you kill; The state is not set to zero.

You have one version SDE.DEFAULT and it is still pointing to state 8549347 or whatever, this state referencing thousand of edits. So you  are stuck with have a huge performance degrade on your database because of all the queries to the A and D table. Usually replication might cause this, so you might have few hidden versions pointing to state 0. Therefore compress command will get confused when these versions with state 0.


BEFORE YOU DO THIS, BACKUP YOUR DATABASE, DO AN ORACLE DUMP OR A SQL SERVER BACKUP. ALTHOUGH THIS THING WORKED FOR ME THREE TIMES I CAN'T GUARANTEE JACK SHIT.


Buckle up, we are about to open the hood of the SDE Technology and loose few bolts in the SDE engine, change a fuse or two, start the engine manually and put the hood back.


1- Fireup your DBMS, connect as SDE user.

2- Open the SDE.VERSIONS

3- You may skip this to 4 if you just want to do the fix, else continue reading if you want to know how this works

Name: The Version Name
Owner: Who created the version
Version_ID: An ID for the version
Status: A status that tells whether this version if private, public, protected or even system
State_ID: To what state this version is pointing, the state
Description: Text, description of the version
Parent_Name: This is a stupid field, bad ERM design, there is a parent_version_id so we can get the name of the parent. Unless they did it like to avoid joining, thus speed up the query response time, then they are smart and I am sorry.
Parent_Owner: again, stupid field, never mind.
Parent_Version_ID: The parent version id there you go.
creation_time: when did this version is created

4- Now you might have different values, but look at the whole structure it should be the same

5- The geodatabase is not compressing because there are two states pointing to zero which makes the compress command thinks everything is rosy. So take your screwdriver lets screw this DB.

6- We will do a manual reconcile to those two hidden versions to point to the same state id as the default (that is reconciling basically) Change the state 0 in those two version to the state_id of the default as follows

7- Save and close

8- Go run the compress command

sdeversion -o compress [-N]  -u sde -p sde_password -i arcsde_service -s server_name
-D database

You should now see your state tree set to zero.

Water Thumb Drive

Digital Thumb drives can hold Gigabytes of data and transfer up to whatever USB 3.0 can handle.

Water thumb drives could hold Exabytes of data (the number of sand grains on Earth and Mars combined) and transfer up to whatever the molecular bonds can support.

Save one bit in a Hydrogen atom and we'll get there.

Arc2Google Official Page

Arc2Google users grew more than I expected, therefore I'm gonna dedicate a page for any new releases of this tool.

Currently I will support ArcGIS 10.1 and above, Apologies for not supporting 9.3/9.3.1 as I don't have those versions anymore. 

Arc2Google Official Page

Arc2Google Current version is 10.1.0.3

Feel free to email me at HUS DOT MHD AT GMAIL DOT COM

Web Services, best invention since sliced bread

Probably you read about it in a tech blog or heard it in a conference.

If you end up here from a google search, you're probably searching for an easy definition for a web service.

Consider the following scenario:

Scenario #1
You at home watching TV in the living room, your mother is in the kitchen making dinner. She is out of cheese, so she asks you to get some from the store. You leave the house, buy some cheese, return home and hand the cheese.

Scenario #2
You are outside jogging in the park, your mother is at home making dinner. She is out of cheese, so she calls your cell, you answer, she asks you to get some cheese, so on your way you go to the store, buy some cheese, return home and hand the cheese.

First scenario is an application for local function calling. You have a software installed locally on your machine (kitchen and living room) that does something (getting the cheese)

Second scenario is a case where that function is online and you can access it from anywhere (home and the park) using a communication protocol (cell phone).


If a software can be called from anywhere it is called a web service. It is that simple, but some IT folks like to use words to make them sound sophisticated. Like REST and SOAP and SAAS etc...

It is its simplicity which makes it beautiful with so many applications.

The Prestige in Programming

Please read this quote from the movie Prestige very carefully.

"Every great magic trick consists of three parts or acts. The first part is called "The Pledge". The magician shows you something ordinary: a deck of cards, a bird or a man. He shows you this object. Perhaps he asks you to inspect it to see if it is indeed real, unaltered, normal. But of course...it probably isn't. The second act is called "The Turn". The magician takes the ordinary something and makes it do something extraordinary. Now you're looking for the secret... but you won't find it, because of course you're not really looking. You don't really want to know. You want to be fooled. But you wouldn't clap yet. Because making something disappear isn't enough; you have to bring it back. That's why every magic trick has a third act, the hardest part, the part we call "The Prestige".

Every programmer has his own tricks on how he make his application stands out. Everyone see the output program or script or functionality and like it, let them enjoy the output, don't tell them anything else.

Once your programming prestige is revealed, that is it. You will start hear things like

"Oh, that's how he did it. I thought It's more complicated. "
"Aha you used that feature, everyone can do that" 
"No, my level of thinking and implementation was way beyond this"
"Instagram? I thought of that application before but didn't implement it"
"No my idea is way more sophisticated." 


Programmers are like magicians, they shouldn't reveal their tricks. Create tricks and keep them to yourself. Learn from others.

ArcFM Server, Do You Need it?

Lots of people requested me to write about this technology, so here it goes.

Good thing about this blog is it doesn't take sides, if I didn't like a product I simply say so. So when you hear something from me (good or bad) its probably coming from my sole experience with a product.

What is ArcFM Server? 
Lets start with what is ArFM. ArcFM has been created on top of ArcGIS platform by Miner and Miner as desktop application for utility users. Since ESRI was not focusing on Utility users, M&M Found a niche and tackled it with a solid product. Made a good money out of this.  Schneider Electric found this gem and bought it.

ArcFM Server is an extension for ArcGIS Server (now 10.1), allows you to support ArcFM functionalities to your GIS Services. They probably didn't say this in the brochures and flyers but ArcFM Server is USELESS without ArcFM Desktop Editor. You need the desktop version to configure your geodatabase, upgrade it to support ArcFM etc.. Without it you are crippled as horse with no hoof wall.

Here is a live demo of ArcFM Server, it is now on 2.4

Good thing about ArcFM Server?
At my company, we have implemented ArcFM Server for one reason. We didn't want to customize anything. We want a product that we can install, configure and voila!. We have arcfm desktop fully configured with feeder manager and all, we implemented ArcFM Server and bang! we have all our electric traces right on ArcFM server page with some basic nice features with no single line of code.

Bad thing #1 Lack of Support and Training
If you got error in the installation or configuration ArcFM server (and yes you will) good luck with that, you are on your own. The forum is filled with unanswered questions. No training yet offered for ArcFM server.

Bad thing #2 Silverlight
I said this to the guys at schneider electric and I'm gonna say it again. There is a god damn elephant in the room and it is called silverlight. Why on heaven are you still supporting this? The only reason we are going web 2.0 so that we can consume our services from multiple sources. Silverlight is not supported on any of the tablets or phones. HTML5 however is supported. Either build us native applications on mobile to consume these services.

Bad thing #3 Trace Tools are so slow
ArcFM Server Tracing tools take on average 40 seconds to execute. That is not acceptable in our business so we end up creating our own from scratch. My custom trace achieves the same result with only 0.5 seconds to run. 98% Less!  So yeah that kinda killed the good thing.

So I'll give this product a year or two to mature, and until ArcFM have a vivid strategy for this product.

Mxdperfstat 10.1 Download MXD Performance Statistics

If you are here chances that you were like me searching for an ArcGIS 10.1 version of this wonderful tool. Well, there isn't any, but I managed to make it work by copying some 10 dependency dlls. (Version , System and few others, took me awhile to figure them out)

If you don't know what this tool is, this is it's description from ESRI site

MXDPERFSTAT (ArcGIS 10 and 93) can help diagnose typical MXD document performance problems, e.g. 
• Inefficient scale dependency 
• Slow symbology 
• Large features 
• Projection on the fly 
• Potential database tuning 

System Requirements: 
1. Microsoft .Net Framework 
2. ESRI .NET Assembly 10 or 9.3.1 
a. ArcGIS Desktop with .Net support or 
b. ESRI Engine Runtime 
3. ESRI license: 
a. ArcGIS Engine runtime or 
b. ArcGIS Desktop

This tool will work on ArcGIS 10.1, extract and use,

Download here

If you want the 9.3 or 10 version of this tool click here.

Fetch URL Text from Android/Java Application

I searched a lot for a straight forward clean multithreaded code to do this simple task, read data from a URL and get the output. All I found were segments of codes that force close.

So I wrote a simple class do this task.

Create a new Class name it FetchURL and past this whole thing into it.

//Hussein Nasser 19-4-2013
//www.husseinnasser.com 
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
 public class FetchURL {

 private String output;
 private String url;
 
 public FetchURL()
 {
  output = "";
 }
 
 public String getOutput()
 {
  return output;
 }
 
 
 public void Run(String u)
 {
   url = u;
    Thread t =  new Thread() {
         
       public void run() {
                   
         // Toast.makeText( mycontext , "running thread bitch", Toast.LENGTH_LONG).show();
                   
                    URL textUrl;
               try {
                 
                textUrl = new URL(url);
                
                BufferedReader bufferReader = new BufferedReader(new InputStreamReader(textUrl.openStream()));
                 
                String StringBuffer;
                      String stringText = "";
                while ((StringBuffer = bufferReader.readLine()) != null) {
                 stringText += StringBuffer;
                }
                      bufferReader.close();
                 
                      output = stringText;
                    
               } catch (Exception e) {
                // TODO Auto-generated catch block
                //e.printStackTrace();
                 
                output= e.toString();
               }
              
                }
            };
             
           t.start();
      try {
   t.join();
  } catch (InterruptedException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
  
 }
 
 
 
 
}

To use the code in your main activity

//Create an Instance of our Class
     FetchURL fu = new FetchURL();
     
     //run this shit (this will pause your current activity until you get the result, this should return 123 
     fu.Run("http://geshout.com/version.txt"); 
     
     //You are clear, your activity is now active and get your string and do Whatever you want 
     String o = fu.getOutput();
     
     //Cheers!
     Toast.makeText(this, "output " +   fu.getOutput(), Toast.LENGTH_LONG).show();
     

This code allows you to read and execute PHP URL, almost anything that returns HTML I guess, anyway report any bugs to me. This code uses threading to execute the network capability And oh, make sure you have the internet permission on your Android Manifest.

.NET Microsoft Performance Counter does not match Task Manager

I have been struggling with this for a while now. I was asked to display the CPU usage for a list of windows servers remotely from one machine. So I wrote a piece of code that does just that using the Performance Counter. Only to find out that it returns inaccurate CPU data. I tried reading more than once, taking the normal distribution of a list of values nothing worked.

After a while it appears that you have to read .netValue once then sleep a while then read it again . Yes that is it. This fixed the problem.

Here is the code to get the real value of CPU matching the Task Manager

    Public Function getCPUPerformance(ByVal remoteHost As String) As String

        Dim userHandle As New IntPtr(0)

        LogonUser("username", "domain", "password", LOGON32_LOGON_INTERACTIVE, LOGON32_PROVIDER_DEFAULT, userHandle)

        Dim identity As New WindowsIdentity(userHandle)

        Dim context As WindowsImpersonationContext = identity.Impersonate()

        Dim cpuUsage As PerformanceCounter

        cpuUsage = New PerformanceCounter("Processor", "% Processor Time", "_Total", remoteHost)

        Dim val1 As Single = cpuUsage.NextValue()

        val1 = cpuUsage.NextValue()

        Thread.Sleep(50) 'You have to Sleep, otherwise you will get kinky results    

  

        val1 = cpuUsage.NextValue()

        Return val1

    End Function

ESRI: Registration Failed, Could not write to disk.

ESRI, ESRI ESRI...

You get everybody confused with your messages.

Why it happens?

Chances that you are here because you encountered this error and you don't know how to fix it. Well the fix is easy but ESRI like to waste your time instead of displaying a little more informative message it just slam whatever the OS returns right back at you.

Anyways, this error usually appear because you were trying to install a new extension on your ArcGIS desktop environment (I assume 10 and above). It ESRIRegAsm.exe to register their dlls, whatever launches this executable must have Administrator privileges because 

The Fix

Run as Administrator. 

If you are running ESRIRegAsm.exe manually, run the Command prompt (cmd) as administrator.

If this error pops up because you installed some exe setup file, then run this setup file as administrator. 

If you are running an MSI file (Windows Installer, they look like this ) , for some reason you cannot right click and run as administrator, so you can use this method.

P.S.: Yes, you have to run as administrator even if you are an administrator on that machine, yeah, don't ask.