Persistent Connections

English is not my first language, so I have to remind myself of definitions from time to time. So mind me if you see me define a phrase before I elaborate on it. To persist something is to keep it existing. So what does a persistent connection refer to? and why this is something software engineers should know about?

TCP is the de-facto protocol of the network communication. When we want to send data between node A and node B we establish a TCP connection, be it a simple visit to google.com or an Oracle database connection, under the hood its all TCP.  

Lots of things involve establishing the connection. Handshaking, acknowledgment, making sure the parties in the connection are in fact who they say they are etc. We don't need to discuss details of what exactly happens in this post, but one thing you should know, opening TCP connection is expensive. 

So imagine this scenario. You visit http://www.nationalgeographic.com website, this uses the HTTP protocol which underneath open a TCP connection to the national geographic server. So lets go back in time to 1996, when HTTP/1.0 was just released. This would happen when you visit that page:

Open TCP Connection national geographic website (this has many steps remember)

Read Index.html and send it back to the browser

Close TCP Connection

For each Image in the page

     Open TCP Connection

     Read Image from the server disk, send back to browser

     Close TCP Connection

Next image

Images are just an example of things we need to load, back then there were much more nastier resources. 

So now imagine the overhead of opening and closing the connections. The network congestion from all acknowledgments being sent back and forth and the wasted processing cycles both the server and the client have to endure. That is why persistent connections became popular, open a connection, and leave it open while we send everything we have, once we are done we can close it. Here is a modern visit of nationalgeographic.com in 2016, HTTP 1.1

Open TCP Connection national geographic website  

Read Index.html and send it back to the browser

For each Image in the page

     Read Image from the server disk, send back to browser

Next image

... Do more 

... Do more

Close TCP Connection

Now, I know this is very specific to browsers and web servers but the same story is true for database connections. In my years of experience as a programmer working with databases, I developed a habit, and am sure most of you did too, of opening a connection, sending a query and then closing a connection. That might be fine and barely noticeable if you have like 10 users working with your application. However, as you scale up, you will start noticing performance degradation.

Another thing you gain of persistent connections is PUSH events. This is how WhatsApp is able to freak you out by instantly delivering your wife's message "Where are you!" to your phone the moment she hit the send key on hers. WhatsApp do that by having a live open connection to their server from your mobile (not exactly TCP though, a much more efficient protocol called XMPP, we can touch upon that on some other post). 

Disadvantages?

We mentioned the advantages of persistent connection, but are there any disadvantages? Yes, there is no free lunch apparently.

When using persistent connection, you keep the connection alive on both the client and server, so you start eating up more memory with more connections opened. So you have to be smart about closing those idle connections.

Another problem came with persistent connections that is specifically for TCP. In a nut shell, now that we started to keep connections alive on the server, attackers came up with this idea, "Hey, what if we made the server run out of memory by establishing millions of connections and never replay back?" Thus DDOS attacks were born. Again we can touch more on this on another post. 

-Hussein

Process Isolation

You may have heard of this term in one of the conferences or tech talks. If you have worked with ArcGIS Server, you probably have seen the option to use High Process Isolation vs Low Process Isolation when configuring the GIS instances. So what does this mean and why does is it important to isolate processes?

So when we say "Isolate", we get a general idea of what that might be. To keep processes separate from each other. So you would think that one advantage of process isolation is to protect the process data from being accessed by other processes, and that is correct. But is that all?

Let's say you want to write a program to implement two functionalities (A and B).  Functionality A is low intensive (perhaps its just printing the result or updating the UI) and B is high intensive (querying the database, processing, rendering  etc..). So one approach is to write code to implement both A and B in one application.  Both Code A and B will be running in one process. 

Another approach is write the Code for A and the Code for B and use Process.Start() to execute code A in one process and code B in another process. Here is where the processes are isolated. 

In the first approach if both A and B runs in the same process, you risk B eating on your memory. If anything goes wrong in code B and it crashes, the entire process will crash taking code A down with it and users will lose your application entirely. However in the second approach, if B crashes, A will still be running and it can easily relaunch B, resurrect it back to life.

-Hussein

ArcObjects SDK available for download !

After years finally users can freely download and work with the ArcObjects SDK for both .NET and Java! Previously this was only available to partners and customers who request the bundle DVD.

Here are the steps.

1. Go to Arcgis.com
2. Register a trial 60 days account
3. Sign in and on the right you can have the link.

Now that you have ArcObjects SDK, go build cool solutions.

Don't forget we have two Youtube series on IGeometry discussing how to get started with ArcObjects. One with VB.NET and another with C# , more than 30+ hours of content.

Enjoy

Extending ArcObjects (VB,NET)

https://www.youtube.com/watch?v=XrZs1rwmOwg&list=PLQnljOFTspQXqYsWJG8o-eJpGlvzww9lE

ArcObjects by Example (C#)

https://www.youtube.com/watch?v=nn4mtxdi19A&list=PLQnljOFTspQVMRyBp7UOt8pU7wygBmEja&src_vid=XrZs1rwmOwg&feature=iv&annotation_id=annotation_401451585

See the Great American Eclipse of August 21, 2017

My colleague Mike Zeiler is fascinated with Maps. What is astonishing is how he is able to author to serve his hobby. Solar Eclipses.

He travels around the world to see every eclipse, he is among the few who actually does that. Next year the eclipse will be in United States - Oregon, so he doesn't have to travel too far this time.

This is his website, go check out the cool maps he built and pay attention to the count down timer to the next eclipse.

http://www.greatamericaneclipse.com/

Enjoy

ArcGIS for Server Architecture Explained

A youtube episode (part of docker series) where we discuss the ArcGIS for Server architecture. We talk about web servers (web adaptors) GIS Servers, config stores and how each component play a role in the ArcGIS server architecture.

Enjoy

Docker and ArcGIS

As a Geogeek, I always want to augment cool new technologies with my line of work, ArcGIS. Docker is proving to be the future in terms of simplifying deploying and scaling.

This Youtube series is my attempt to start to unravel the marriage between these two technologies.

This is something new and different in this channel. IGeometry has always been about programming with ArcGIS, we talk about how to build cool software with ArcGIS technology. However, we never talked about how does existing bleeding edge technologies blend with the ArcGIS platform. So here it is.

I'm very interested in Docker and I have actually used it in a small project with ArcGIS. I want to start this series to try to understand where Docker fits in the ArcGIS stack. We might eventually end up doing some implementations of Docker and ArcGIS. If and when possible.

You might say that ArcGIS runs on Windows and Docker runs on Linux, so how can you integrate the two? That is not entirely true, we have a lot of pieces in the ArcGIS technology that actually run on linux and thus supports Docker containers. We will explore that in the coming episodes.

Let us see how this goes.

Enjoy

https://www.youtube.com/watch?v=9oA_h9ol3dY

ArcGIS Add-ins C# (IGeometry) - 12 - Reception Coverage

IGeometry

---

ArcGIS Add-ins C# (IGeometry) - 12 - Reception Coverage Posted: 20 Mar 2016 11:27 PM PDT

You are subscribed to email updates from IGeometry. To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States

ArcGIS Add-ins C# (IGeometry) - 10 - BARS

We created a new tool to add a device (phone, tablet etc.) and get the current signal and the connected tower, also draw it on the map!

We checked in to github

https://github.com/hnasr/phonatech.git

Class v. Object: The simplest explanation possible

• Take a cupcake pan and assume it has only one hole for one cupcake.
• Prepare a cake mix with vanilla flavor and pour it in the pan.
• Put the cupcake pan in the oven. And wait until it is cooked.
• Take it out of the oven and you now have vanilla cupcake.
• Prepare a cake mix with raspberry flavor this time and put it in the same pan.
• Put the cupcake pan in the oven. And wait until it is cooked.
• Take it out and you now have raspberry cupcake.
• Put each Cupcake on its own plate. You only have two plates.
• Similarly try bake a lemon cupcake, but wait, you don't have a plate to put it on so you can't bake just yet.
• Eat the raspberry cupcake.
• Now bake a lemon cupcake, take it out of the oven and put it on the empty plate where the raspberry cake was.

Illustrated:

The Cupcake pan is the class.

Cupcake();

The vanilla cupcake is the an object made from the class.

Cupcake vanillaCupcake = new Cupcake();

The raspberry cupcake is a second object made from the same class.

Cupcake raspberryCupcake = new Cupcake();

Class is the same, objects are made from a class, objects from the same class can have different attributes.

vanillaCupcake.Flavor = "Vanilla";
raspberryCupcake.Flavor = "Raspberry";

The time the cupcake was in the oven cooking is the object initialization time. How much time it take to create an object, store it in memory and all this shit.

creationTime = Now; //08:01:05 AM 
Cupcake vanillaCupcake = new Cupcake();
initializationTime = Now - creationTime; //08:01:07 AM  
//initializationTime is 2 seconds

The lemon cupcake could be created but no plate for it, this is insufficient memory to create an object.

Cupcake lemonCupcake = new Cupcake(); //ERROR

Eating the raspberry is basically destroying the object and removing it from the memory.

raspberryCupcake = null;

Now that we have an extra plate, we can put our lemon cake in it. We have enough memoery to create another object.

Cupcake lemonCupcake = new Cupcake();
lemonCupcake = "Lemon"

Cupcake are just to explain what class and object are but if you really want to understand it, I do believe OOP is learned in a real life project context. So I started a Youtube channel to teach OOP within a context of a real life project where we build a project from scratch and add functionality episode by episode.

I don't even have ads on this thing, I have a day job and I do this in my free time. I just genuinely want to share my knowledge out there.

Here if any one is interested https://www.youtube.com/playlist?list=PLQnljOFTspQX-XZChZg-b06MJ27nAJQS9

My post on reddit: 

-Hussein

ArcGIS Add-ins C# Series

We are back! 

Brand new episode, talking about ArcGIS for Desktop, customizations, describing the setup, Extending ArcObjects vs Add-ins and discussing what we are going to do in the coming episodes! 

Enjoy Guys
Stay Awesome!