Saturday, November 21, 2026
HomeCloud ComputingSave Time and Possibly Your Job with Full Stack Observability

Save Time and Possibly Your Job with Full Stack Observability

[ad_1]

Again in October of 2017, I may have actually used an observability suite.

We had simply migrated the entire Cisco developer web site, developer.cisco.com, from our in-house managed datacenter area to an AWS area, US West. All of the QA, integration, and consumer acceptance testing had gone with no hitch. SSL certs had been utilized and dealing as anticipated. We went reside with the positioning over a weekend. There have been no complaints for a couple of days, and we thought we had simply overseen a totally profitable migration.

Then I bought a ping. Our VP was displaying an SVP the positioning on their telephone. The VP’s telephone may deliver up the positioning no downside, however the SVP’s telephone simply couldn’t resolve the web page. Scrambling to determine what had occurred, we had been checking web site entry logs, database logs, and having everybody on the group hit the positioning from numerous gadgets. No pleasure. Nobody internally may replicate the problem. However then we did begin to get a trickle of exterior experiences of individuals experiencing the identical failure.

Daily for every week, I used to be poking across the web to determine simply what was the reason for the nook situation. Our engineers had been making an attempt to ID the place the issue was occurring. Lastly, I’m having lunch with a colleague, and I ask him to see if he can get to our web site from his telephone. He couldn’t. I attempt on my telephone. I can.  We actually have the identical make and mannequin of telephone, so I’m scratching my head. We head again to the workplace, and he comes by a bit later to let me know that he was capable of hit the positioning later with no downside.

Lastly, it dawned on me: at lunch we had been each on our cell provider’s service, however within the workplace we’re on Wi-Fi. I requested him to show off Wi-Fi. Now he can’t get to the positioning! Lastly, a workable lead. I get to looking out and discover out that with some cell carriers and with a selected model of the telephone, the mixture of SIM settings plus the provider community configuration was set to solely resolve websites that had IPv6 addresses. “That’s humorous,” I assumed, “we had been IPv6 enabled at our previous datacenter. Certainly AWS can be enabled for IPv6.” Seems, they had been… largely. They had been not for the configuration of VPC we wanted to make use of within the area to which we had migrated.

It took a lift-and-shift to maneuver our set up to a special AWS area, and eventually the SVP (and different customers!) may now get to our web site.

What I Wanted However Did Not Have

You may be asking, “How does this lengthy story relate to full stack observability? Even when that they had all of the monitoring instruments in place, they’d’ve nonetheless wanted the luck to determine this one out.”  Granted, this was at all times going to be a tough situation to run down. However FSO would have accelerated our potential to rule out false alerts sooner, and even instantaneously. We’d not have needed to pore over logs or verify databases. We wouldn’t have needed to do handbook site visitors checking. Or dig into the code to see what may be occurring. We’d have recognized that these areas had been purple herrings and we might have narrowed our focus rather more shortly to the consumer facet. We’d have been capable of see if the requests had been attending to our CDN and the place the returns had been failing, and arguably with the appropriate device we would have gotten a feed immediately from our VPC that stated, “Shopper can’t resolve IPv4 addresses.”

I’ve been in software program growth for 20 years, and anybody that has been writing — and extra importantly, debugging — code for that lengthy will inform you that the extra visibility you’ve gotten into the code the simpler and faster it’s to seek out and repair a problem. Immediately, with the abstracted and layered complexity of functions, discovering a fault is commonly extraordinarily difficult. Throw in microservice architectures, and you’ve got challenges not simply with the bodily layers impacting the applying (community, compute, storage) however the virtualized ones like container volumes. Each single a part of an software deployment, from the community, to the consumer, to the app, has an influence. You want visibility to points on all the, full stack.

Purposes, and the individuals who keep them, are higher served after we can see and measure what’s happening, good or unhealthy. If Accounting’s internet software is operating gradual once they’re making an attempt to shut out 1 / 4, is the problem considered one of community bandwidth, or is it a persistently crashing software node? We must always be capable of determine that in seconds with a mixture of streaming telemetry information from the community and software information from the mesh supervisor. If we’re actually savvy, we might even be capable of determine faults proactively by feeding in information on conditions the place we all know we would have – like spikes in database hits, or consumer load, each of which might require scaling up pods, for instance.

The excellent news is that observability applied sciences and tooling retains getting higher at offering us deeper perception so we are able to make higher selections extra shortly. With machine studying and AI added to the combo, we’re beginning to see self-healing networks, processes, and functions. These instruments will give us extra time to innovate, and require much less time from individuals making an attempt to determine why a bigshot can’t entry an software.

Sadly, there may be not (but) a magic bullet to comprehend full stack observability. It requires conscientious design and implementation from individuals engaged on the community to these coding the functions. This work results in tooling and instrumentation at numerous ranges, offering the visibility and metrics wanted to achieve observability. We predict it’s value getting on top of things on the applied sciences and processes of observability.

To study extra, I like to recommend planning to cease by The DevNet Zone at Cisco Reside US this yr (both in particular person or nearly). You possibly can study quite a bit about what Cisco is doing to facilitate full stack observability from community monitoring automation and software insights with AppDynamics, all the way in which to the content material supply area and the consumer. Make sure to try my workshop, Instrumenting Code for AppD, Thursday, June 16 at 9:00am PDT.

And take a look at periods like these:

Learn extra about Observability:

I’ll see you at Cisco Reside!


We’d love to listen to what you assume. Ask a query or depart a remark beneath.
And keep related with Cisco DevNet on social!

LinkedIn | Twitter @CiscoDevNet | Fb | Developer Video Channel

Share:



[ad_2]

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments