3 Considerations for Container Debugging

Debugging is a major, critical challenge for software developers. The shift to highly distributed operations in containerized cloud environments over the last year of the pandemic has further complicated the debugging process for many DevOps teams.

In this article, Liran Haimovitch, co-founder and CTO at Rookout, offers three ways to overcome the challenges of managing debugging in containerized environments.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

Local IDEs Are Probably Not a Good Fit for Distributed Repositories

One of the first things organizations typically realize when making the shift to cloud-native, containerized environments is that it is unwise to try replicating on-premises operations in these new, and often, hybrid, environments. The debugging process is no exception to this rule, when developers are working remotely and uploading code to a local integrated development environment (IDE). In fact, Haimovitch says, using a remote debugger when debugging within a local in IDE is not practical or possible.

“When applications are running remotely, oftentimes connecting a debugger is not possible due to security restrictions or other technical limitations,” Haimovitch says. “Production-grade remote debugging solutions are built for this purpose, and can save developers time; from going through the typical code change to CI/CD to release processes.”

Think Big-Picture Observability

Observability is increasingly seen as a necessary component in the debugging process, as a way to rely on inferences often derived from AI-generated data points to gain valuable insights into application performance — especially those distributed in containerized cloud environments. In this way, Haimovitch recommends DevOps teams use an “observability pipeline, or similar construct, to determine when and how to debug applications.” With this approach, application logs, metrics and traces are sent as events “… from running services, and are sent through an observable management interface for alerting, routing and searching before output in their proper destination(s),” says Haimovitch. “Think of it as a centralized, one-stop shop for observability,” he says.

Debugging solutions should also be combined with application-performance monitoring (APM) tools and service meshes, which are increasingly essential for properly managing Kubernetes application deployments. APMs, Haimovitch says, can be used to monitor system operation and performance to triage operations issues faster. “Remote debuggers get you to a deeper level of information than what APM’s provide, and are a powerful force when combined together,” says Haimovitch.

Meanwhile, service meshes can offer deeper observability capabilities, as well as help improve the reliability of networking and security, Haimovitch says. Since networking and security often represent problem areas that impact application performance, the use of service mesh specifically, in this way, further supports debugging processes.

Developer- and Container-Centric Debugging Tools

Successfully implementing all of the above, of course, is contingent on adopting the right tools that, among other things, automate many of the processes. Take into consideration developers’ personal favorite debugging tools, and that those might not necessarily offer the right fit; DevOps’ team leads must also take into consideration developers’ preferences and requests during the tool selection process.

“DevOps teams should adopt tools that expand current capabilities, but that also can fit easily within the developer’s current workflows,” says Haimovitch.

Debugging tool selection also needs to take observability, service meshes and APMs into account when considering the debugging processes. “When adopting a new tool, you want it to be a step up from what is being used today in terms of insight into issues and discovery of new and previously unseen information,” Haimovitch says.

At the same time, developers’ needs remain paramount. “Teams should adopt solutions that can shift the identification of defects further to the left in the software delivery process. The sooner defects can be spotted and remediated, the cheaper they are to fix,” Haimovitch says. “Solutions should be continually pushing the identification of those defects toward the development phases, and out of production where they affect your customers.”