My most ever challenging project

For some reason this topic came up as a question in quite a few interviews. Though I remember "my most challenging project" rather well, I tend to leave it out from the discussions, due to several reasons (and as it sounds too fantastic).

It was just after Y2K (2000) era, when humans realized that the world wasn't going to destroy itself due to the limitations of the date problem in early computers...Our journey with Karma had just ended and I was hunting hight and low, when a fellow friend, who was the CEO of a major IT Project company, in Turkey (I am publishing this article with his kind permission). I had signed up to work on project basis, ranging from networking hardware to complex IT tender projects

Soon enough a "Mission Impossible" project landed on my desk. Turkish Directorate of Motorways -(TCK was and still is one of the major gov't tenderers in Turkey) had recently opened a tender for data connectivity project for the Istanbul's two Bosphorus Bridges datacentre's.(As there are toll booths on both bridges, the data used be collected to huge tapes then manually shipped to the hq datacenter, which of course meant a huge vulnerability).

At first glance in 2018, the project may seem quite simple, but in early 2000 era of Turkey, where fiber connectivity was only a myth. Most of the city’s internetworking backbone was built on dialups and copper cables which were connected over tens of pops.. Due to the extremely low quality cabling, excessive pop usage, speeds over 2 Mbps (async ofc!) was only a dream. Whilst the Telecom was still trying to hurdle with the leased copper lines, though it was available in selected locations, ISDN was part of the dream; even then 4-5 128kbit lines didn’t solve anything.

So the TCK had a huge bottleneck as trying to link both bridges’ datacenters into one, generally meant for the offering companies to use at least 2x 2 Mbit copper Leased lines over approx. 20 pop locations. Of course this means huge hardware as the day had it’s limitations. Estimated value for the project was around $ 2m.

Of course, first alternative that came to my mind was to lay a 5 km of fiber cable under the Bosphorus Sea. Though it seemed reasonable, it still was risky as Istanbul is known for it’s unsolicited constructions, yes, even underwater and as the govt’s offices were so disconnected from each other, they usually wouldn’t even bother letting the other party know that they are going to dig in their turf, until they damage something :)) ; plus, Bosphorus Sea itself isn’t a soft cookie. It’s known for its strong currents, which generally meant extra shielded fiber wires needed to be used with a lot of slack. When we added the costs it came close to $ 2m, which didn’t really leave much profit for us.

Wireless? What wireless? We are talking about year 2002. How can you get a secure wireless singal over 5 km and achieve 2Mbit data connectivity ?

I didn’t know it was possible and most of you out there, still do not know that it is possible !  

With a long flight to Canada and vast R & D, I was convinced that it was possible; at least it was possible to send 155 Mbit at full duplex over 4 km and the company was promising that weather conditions such as rain, fog, snow didn’t matter. After the nda, we shared our project with the manufacturer who mainly served on military contract, which were actually fascinated with the idea. They were so fascinated, they even decided to support the project !

The challenges were that the direct line distance from A to B was about 5 kms (4.9) and the range of the device was limited to 3 km (2.8 guaranteed). This simply meant we needed a pop somewhere in the middle, perhaps on a cell tower or something even higher ! Also due to the technology of the products all the transreceivers had to see one and other over a certain degree range.

You still don’t know what I am talking about do you ? OK, no need for mystery. There’s a technology that exists since the late 90’s and is widely used where cable or radio signal connectivity isn’t possible :)) Yeah I know. I drooled too.

It’s called FSO – Free Space Optics. You can read all about what it is here. Similar technology is currently used today to transmit live HD video from the ISS to earth and enabling us to view it over YouTube live. Though our goal was to obtain the “AFAP” (as far as possible + as fast as possible) fSONA can obtain around 2.Gbit data rate over shorter distances today.

What FSO is:

Free Space Optics (FSO), also called Free Space Photonics (FSP) or Optical Wireless, refers to the transmission of modulated visible or infrared (IR) beams through the atmosphere to obtain broadband communications. FSO systems can function over distances of several kilometers. As long as there is a clear line of sight between the source and the destination, and enough transmitter power, communication is theoretically possible. Like fiber, FSO uses lasers to transmit data, but instead of enclosing the data stream in a glass fiber, it is transmitted through the air.

In A Nutshell - FSO transmits invisible, eye-safe light beams from one "telescope" to another using low power infrared lasers in the teraHertz spectrum, where capacity can reasonably be expected to reach 10 Gbps. The light beam carries whatever optical transmission signal (layer 2 or MAC) and protocol framing a manufacturer chooses to market, typically SONET/ATM and 10/100/1000 Ethernet. Plus, unlike other free space communication systems, FSO doesn’t require licensing.

I admit it took me a while to convince my friend who actually had put me up to this task as this technology was quite unknown then and it still is pretty much unknown in today’s IT world.

The most amusing part was that all equipment and the installation cost was merely $ 400k ! On the day of the tender, it was quite crowded, probably around 30 companies, some are tendering, the rest just curious enthusiasts.

As expected, first offer was from one of the leading IT project companies and their offer was to go use 2 x 2Mbps LL coppers over 20 pops and hops which guaranteed 1.5 Mbps over the low quality and totally untrustable backbone of Istanbul at a round figure of $ 1.7m. The reason for 2x LL was to achieve the redundancy as well as to load balance the traffic. (But, I can’t help still thinking, wtf is the point of having 2x copper wires going through the same route, same potholes etc. as they are bound together for damage !)

All of the offers in the specific tender were based on copper wire technologies and they just played around the prices on their active/passive hardware via the discounts on their GPLs. The price was as low as $ 1.4m, until we stepped in. Our starting offfer was around 20% lower than the closest competitor and a huge gap of 144 Mbps in full duplex !

I still remember that moment quite well as all the heads suddenly turned towards us like a herd of penguins!! There were quite few synical grins by the “pro’s of the trade” like it was some kind of a joke being told. After a 5 minute presentation, TCK had to decide that there’s no reason for a 2^nd round to tender. After selection of a substute, there was a huge rowl in the room people screaming in pain and distress ! 😂😂😄😄

As a rookie conqueror, I remember that moment quite well. With a simple proactive thinking and deep research I had managed to defeat the top 10 brands and the pro-IT companies which were left clueless ; which several of them went far enough to threaten us with legal proceedings due to their misery!

It was until next Monday that my victory celebrations lasted, when the company owner, who reminded me that I was the leader of the project which didn’t only consist of the sales part of the project, but also the supervision of the installation as I had agreed previously.

Our project proposal actually consisted of placing 2x beacons on the highest point of the two Bosphorus Bridges as well as installing 2x beacons on a antenna tower (approx. 45m high) at the center location as a bridge point. (and I always thought the firm had technicians to install the devices!!!).

First one to bail out was the manufacturer, then the company’s other techies 😀😀😄 as the distance to place the beacons were as high as 170m from sea level (approx. 70m from the carriage way to the peak of the carrier towers) on a tiny platform as wide as 3 x 4 m !! Though I tried to explain to my fellow technicians that all towers consisted of elevators to take them up, I guess I wasn’t very successful.

Frankly, as being one of the fortunate ones to experience the 1999 Izmit earthquake (7.6 magnitude) on a high tower apartment, I knew what it feels like to be in a high altitude when the ground is moving. As both of the Bosphorus Bridges are suspension bridges, they relatively move around even when you are on the carriage way. But being on a 12m² platform, 70m higher and in open-air…….. … --- …

Admittedly I was an adrenaline junkie until that day and I had never had experienced any anxiety whatsoever.

Fortunately it took approx. 4 months for the goods to be shipped from Canada to Turkey; enough time to get used to the idea of working at high altitude. Luckily I had managed to convince two professional mountaineers from Middle Eastern Technical University to assist me on my quest….

Until that day, I had thought I had completed my “most challenging project”….

It was sometime in 2002 a cloudy Spring day that was scheduled for the climb on the primary tower of Fatih Sultan Mehmet bridge. It took us around an hour to carry the equipment to the leg of the bridge, where we loaded them onto the service elevator for the ascension 😇😇

When the climb began, I had alreay calculated the wind, the vibration but what I hadn’t taken into consideration was the flexibility ratio of the steel-reinforced concrete tower! Honestly, I have lived that moment in my nightmares for several years.

When we were in the final chamber, it was much smaller than I had expected. Though scene was spectacular, I realized there was no reason to wear the safety helmet as incase of a mishap, it wouldn’t really matter on which part of my body I would land on 😆😆😆😆 Although we were securely harnessed to the rails by double safety belts (to move around you need to detach/attach one belt at a time) due to the gushing winds as well as the flexing tower (I would have never guessed that concrete would flex so much) I could hardly standup (yeah also due to the height LOL!). For a second or so, for a reason that I could not comprehend I had wished to try base jumping from that point. I guess it would have been more fun.

Of course as we hadn’t unpacked the goods on the ground level, we had to cancel the event and head back to the ground level to re-prepare. On the 2^nd trial it was more comforting than the first mockup as the boxes were unpacked and everything was nearly ready to go. After 2 hours of drilling the reinforced concrete, we managed to fixate the beacon on the tower and install the network cable behind it all the way down the elevator shaft. I even had managed to get few shots on my 2MP Kodak camera !

No, the rest wasn’t child’s play. Though I didn’t have to climb to the top of the bridging tower in the center, I had to do the climbing once again on the old Bosphorus Bridge, which was about the same height, the concrete was crumbly and it rocked like a cradle. After a month of constant climbs, tests, I had managed to complete the project on time and hand it over to the offfice doodes who think themselves are real technicians 😈😈😈

I think it was around then I had got the idea to get married and settle down...sigh...I wish I was on top of that tower instead now (at least I had harnesses)….

Was it worth it ? 

Hell Yeah ! Let's go again !

References:

fSONA - http://www.fsona.com

Wikipedia - https://en.wikipedia.org/wiki/Free-space_optical_communication