How legacy applications block digital transformation

Note: this is part 4 of an on-going series about digital transformation in the time of pandemics. You can read part 1 here. Part 2. Part 3.

Drivers of legacy modernization

It’s clear that legacy applications are a roadblock to digital transformation. What are the drivers for legacy modernization?

Legacy systems in both life and P&C are built on outdated architectures with high maintenance costs, inherent inflexibility, redundant features, lack of connectivity and low efficiency. Complex application and process logic is often hard-coded and undocumented.”
Market Trends: How Cloud Managed Service Providers Address the Insurance Industry, Gartner Oct. 2019

From the demand side, we see supporting digital business (aka digital transformation) and agility to respond to a more rapidly changing environment are key factors in the DX journey. These are the users and management requirements to move away from legacy code. From the perspective of IT, five factors are all critical:

• Technical debt: applications may function cor- rectly, but still be written in obsolete languages, run on out of date runtime platforms, and use technologies like COM that are potential vectors for malware or just plain hard to
• Aging technology: Although Visual Basic 6 (not VB.NET) has been out of date since the start of the 21st century, countless VB6 applications are still in production, some in excess of 1M lines of code (LOC). They may still rely on 3rd party components from vendors long gone. The same is true for languages like PowerBuilder, Delphi, ASP and ASP.NET classic, and more.
• Time, cost, and risk of change: Old legacy apps normally have been maintained by multiple generations of developers with no central control over architecture, coding standards, or engineering patterns. They are often found to include bad practices like untyped variables, globals, tightly coupled code, and little documentation or useful com- ments. At some point, it’s safe to assume that no single employee understands the application source code anymore. Change—even a trivial change—creates risk and takes longer than it should due to a lack of any automated testing or even a regression test suite. This was perfectly exemplified by the global angst that set in as IT prepared themselves for Y2K updates to ancient mainframe applications. Except today it’s client-server.
• Unsupported platforms: Not only is VB6 out of vendor support, but so are the primary platforms it was deployed on: Windows XP, and—just recently—Windows Regulated industries are typically prohibited from using OOVS tools or platforms and can be subjected to fines or other sanctions when caught. Worse than that is the lack of vendor support means these out of support platforms are magnets for bad actors to target with malware. Yet all too frequently it’s the vital applications running on those OOVS platforms that keep the enterprise using them. Time to modernize.
• Skills shortage: Just as the mainframe COBOL world saw their work- force age out some time ago, the experienced workforce from the 90s build out of client-server desktop applications is rapidly aging out of the workforce, and there is little to replace them with. Languages like VB, Delphi, and PowerBuilder—used extensively to build mission- critical client-server applications over a decade or longer—aren’t being taught in computer science curricula today. Neither new entrants to the IT workforce nor experienced mid-level or senior developers want to be saddled with supporting these obsolete languages. Interim staffing agencies who can provide maintenance programmers with the requisite experience can charge usurious hourly rates, further increas- ing the pain of maintaining these legacy applications.

7 ways to modernize legacy applications

How to get off legacy applications? From the simplest to the most complex, here are the seven ways Gartner categorizes modernization approaches (Choose the Right Approach to Modernize Your Legacy Systems, Gartner Aug 2019):

• Encapsulate: wrap legacy apps and expose APIs
• Rehost: aka lift and shift—redeploy existing app without recompiling (no real changes)
• Replatform: aka lift and reshape--move the existing app to a new plat- form without changing code, features, or functions
• Refactor: restructure and optimize existing code without adding func- tionality to remove technical debt
• Rearchitect: ala re-engineer—alter existing source code manually or via code transformation/migration to enable shifting to cloud platform and fully exploit new capabilities
• Rebuild: aka rewrite, redesign—rewrite existing application from scratch while maintaining basic purpose
• Replace: aka drop and shop—retire or eliminate app or component and replace with something better including

Analyzing application portfolios

The first step in attacking the legacy problem is understanding the scope of the problem. Inventorying and categorizing the entire portfolio of applica- tions is critical. Let’s examine two approaches to portfolio analysis

The TIME approach

Gartner recommends enterprise IT management categorize all legacy applications according to a system with the acronym TIME, which stands for Tolerate, Invest, Migrate, Eliminate. The axes of analysis are business fitness and technical fitness5. An example of a medium size portfolio plotted against these two dimensions might look like this the following diagram.

Gartner’s description and recommendation for each category is as follows (Use TIME to Engage the Business for Application and Portfolio Triage, Gartner, 2019):

Tolerate (Medium to high technical fitness; medium to low business fitness): Generally these apps do not pose significant risk for the enterprise nor are they particularly vital. They should be left alone, although monitored in case they fall into a category requiring replacement or re-engineering.

Invest (Medium to high technical and business fitness): These are the best and most important apps in the portfolio. Typically they are newer and are not dependent on obsolete languages, platforms, or components. Continue to maintain these apps with a commitment to preventing any further technical debt.

Eliminate (Medium to low technical and business fitness): These apps beg the question “Why is this still running?” Apps in this category represent a significant liability to the organization, in that they are neither providing real value and at the same time requiring expensive maintenance or exposing risk vectors. Decommissioning, retiring, or replacement with COTS or moving necessary functionality into applications in the Invest category is recommended. It is also necessary to understand and preserve the data that these applications use, moving that to another silo as appropriate.

Migrate (Medium to high business fitness, medium to low technical fitness): In a way, these are the crown jewels of the application portfolio, because they are already critical to the organization but blocking fully realizing the benefits of digital transformation. Typically these applications have the following characteristics:

• Large: applications in this camp can be from 500KLOC (1000 lines of code) to 5 million LOC and up. Their size limits any single engineer from having full knowledge of the application logic and
• Complex: they contain large amounts of code to handle business rules and processes. Because they were built incrementally over years, it’s difficult or impossible to isolate the actual code for individual pro- cesses and
• Old: many of these apps can date back to the 1980s or 90s, some even have core code originally written for Microsoft
• Multiple authors: developers for these applications came and left over the years. Each brought their own style, patterns, prejudices, and preferences for coding, making the resulting application a sort of fruit salad of software engineering
• Old technologies and languages: VB6 apps made extensive use of COM and 3rd party components, most of which are no longer available or supported. COM, ADO, Crystal Reports, and proprietary UI classes can make these applications difficult to run properly on current hardware/ OS
• Weakly typed, tightly coupled: Concerns are not isolated. Functions perform multiple tasks. Variables are poorly named and late-bound, making run time error checking problematic. Exceptions are not logged or isolated. Unit tests are non-existent. Regression tests are likewise missing in action. Dead code abounds. Comments are either out of date or missing entirely. Naming conventions are all over the

Yet these same applications are vital to the organization. These are perfect candidates for modernization via automated migration (see below).

PortfolioMAP from Mobilize.Net

Mobilize.Net analyzes application portfolios against similar dimensions as Gartner's TIME system: the amount of technical debt vs the business valueof the application. See the matrix below.

What do we mean by "technical debt?" Technical debt can be defined as any engineering compromise that expedites the release of a piece of software but that causes problems down the road. Martin Fowler says:

Software systems are prone to the build up of cruft - deficiencies in internal quality that make it harder than it would ideally be to modify and extend the system further. Technical Debt is a metaphor, coined by Ward Cunningham, that frames how to think about dealing with this cruft, thinking of it like a financial debt. The extra effort that it takes to add new features is the interest paid on the debt.”
Technical Debt, Martin Fowler May 2019 (from https://martinfowler.com/ bliki/TechnicalDebt.html)

Technical debt—like unpaid interest on a loan—can accumulate over the the years in legacy applications until it becomes overwhelming. At that point the technical quality of the application can become a serious liability to the business owner.

The “Business Value” axis is self-explanatory. Suffice it to say that just because applications are old doesn’t mean they are not mission critical.Using these two axes, apps can be placed into the four quadrants of:

• Support and maintain
• Do nothing
• Shoot and replace
• Modernize

In this case, the quadrant labeled “Modernize” can be further divided, again by evaluating apps against the same dimensions of technical debt and business value.

This portion of the portfolio represents applications with years of un-remediated technical debt, written in obsolete/unsupported languages and platforms (such as VB6 or PowerBuilder), yet has significant business value (unique, competitive advantage, or mission critical). Some of these can be cloud-hosted via “lift and shift” and others can be repaired or rewritten. More typical, however, is the need to preserve proven IP without incurring the costs and risks of a green-field rewrite. Using Gartner’s continuous modernization approach as a guideline, the most rational strategy for unblocking these legacy applications is to use automated migration tools.

Both of these approaches are similar in evaluating all legacy apps against both business and technical dimensions; the specifics of how to measure are best left to individual judgement.

Legacy modernization and digital transformation

Reviewing our four categories of legacy applications above, we can see that the greatest value to cost ratio comes from modernizing the apps in the top left quadrant (see the Mobilize PortfolioMAP diagram) or the “Migrate” quadrant in the Gartner TIME system.

Why? Because these offer the greatest business value and have the worst technical quality. The “Do Nothing” and “Shoot and Replace” categories of apps offers little real business value, so modernizing them is largely wasted resources. “Support and Maintain” are providing real business value, but since they represent the most recently developed and best technical achievement in the portfolio, they are more easily unblocked from DX than the “Modernize” quadrant apps. Let’s look at both of these top two quadrants separately.

"Support and Maintain" quadrant

These applications provide significant business value, so they are a key part of achieving real DX. They are also the best in terms of technical quality (lowest technical debt), so generally that means they are of recent vintage, written in still-current languages and platforms, and with modern software engineering best practices like loose coupling, separation of concerns, OOP, unit tests, and more.

Many if not all of these applications may already be cloud native in simple or complex form.

Simple cloud native architecture

Beginning with the 1990s, organizations rushed to build dynamic web sites that could do more than display images of cats. A host of technologies published by vendors sought to fill the need for tools and components that could allow browser-based code to replicate or even replace classic monolithic applications. ASP, ASP.NET, Java applets, Flash, J2EE, Webforms were all part of the early evolution of web applications, all of which had some common characteristics:

• The UX was rendered in a browser using HTML and CSS
• Logic was largely retained on a web server which processed requests from the client and constructed pages to be returned and rendered
• UI elements were mostly text-based and fixed (non responsive).

Rich cloud native architecture

More recently we see richer, thicker client architectures taking advantage of JavaScript, frameworks like Angular or Vue, UI widget libraries with rich interactive capabilities, and more client-side logic, data validation, and processing. Connections from clients are made with JSON messages to RESTful endpoints, and API ecosystems connect microservices, serverless functions, data layers, and servers running in containers together.

Combined with these architectural elements, modern cloud native apps are built with CIPS components including Azure DevOps, Jira, Visual Studio Code, Github and with practices & tools like Agile, TDD, BDD, DevOps, CI/CD, automated build/test/deploy, etc.

Digital transformation and the "Support and Maintain" apps

Whether simple or complex, most existing web applications can be part of the DX initiative without undue effort. For example, the server side code is already structured to handle sessions and requests, so connecting APIs is relatively straightforward. This opens the application logic and functionality to both inter-company app integration as well as intra-company integration with supply chain partners, customers, and more. Likewise, monolithic apps in this quadrant can be deconstructed/refactored into microservices using the “strangle” approach, which in turn can further simplify extending and integrating them with other systems and processes (see Nicolas Carlo at www.understandinglegacycode.com).

"Do Nothing" quadrant apps

There's little to say about these apps. Leave them alone. Their technical quality does not create risk and their business value is low. Ignore for now.

"Shoot and Replace" quadrant apps

These apps are bad. Little real business value and terrible technical quality. Frequently these are department-level applications created perhaps Microsoft Office automation or tools like FoxPro, Access, or even VB6. They are good candidates for replacement by some kind of Commercial 

"Modernize" quadrant apps

The biggest brakes on your journey to digital transformation are the apps in this quadrant. They have value, they represent vital business rules and processes, but they’re locked away in complex, opaque, and poorly documented code that’s out of date, obsolete, and out of support.

These mission-critical applications can’t be retired because the organization depends on the value that they continue to deliver.

They can’t be easily replaced by COTS. Commercial software application platforms like SAP, Salesforce, and Dynamics allow for both extensive customization and even programability, but they come with big drawbacks when trying to replace existing complex bespoke applications:

• The upfront effort of customization and related development to duplicate existing app functionality can be expensive, carry significant risk, and take months of effort
• Recurring costs for COTS are forever
• Once fully implemented, it may be virtually impossible to ever leave these systems, which now hold all your data in proprietary formats
• Users will need to be retrained
• Processes that can’t be fully duplicated in the commercial offering will have to be revised or eliminated, potentially causing disruption to the enterprise.

For these reasons, most organizations with large complex systems typically look elsewhere to remediate these systems, and ordinarily the first suggestions from IT are to rewrite the application from scratch.

Why software architects believe they must rewrite legacy apps

Ask 20 software architects how to approach legacy code and 19 will tell you it needs to be rewritten from scratch. They’re not always wrong, but likewise they are frequently mistaken. The 20th you ask will say “NO!” loudly to the notion of rewriting, because that engineer has already experienced just what a nightmare rewriting large complex applications can be.

Recently I visited a market-leading product manufacturing company. They had achieved a level of success with PC-based application software their customers used to control the company’s connected devices. But those customers wanted a cloud option, not the current offering written in VB6 and captive to a desktop running its own instance of SQL Server. They wanted status messages on their smart phones; they wanted API integration with other systems; and they wanted access to controllers anytime, anywhere. Not just at their desktop.

So IT began rewriting the software to be cloud native.

As anyone who has gone through a similar process knows, things rarely go as planned. Schedules began slipping. Requirements shifted, leading to scope creep. Delivery dates were promised and missed. Finally the product manager realized he was going to get hammered in the market by competitors while waiting for IT to deliver the promised cloud version.

Wisely, he sought out and found Mobilize.Net who were able to quickly and affordably migrate his existing application to cloud native using AI-assisted automation. That migration gave the company an effective hedge while waiting for IT to deliver the “perfect” application. Meanwhile the migrated application is driving customer adoption, has a mobile app version, and is using APIs to connect existing controllers to other customer systems.

Perfect is the enemy of good.

This saying has been around for years, and is variously understood to mean two opposite ideas:

• Making something “good enough” prevents it from every being great (ie perfect)
• Striving for perfection blocks progress, while good keeps you moving

Green field software development is difficult. The number of high- visibility projects that have tanked is enormous. Like large-scale civil engineering projects¹, writing large software applications from scratch historically carries a high probability of failure². Some interesting statistics from 2019³:

• On average, projects exceed the budget by 27%
• IT failures cost US businesses between $50-150B/year
• Projects with budgets greater than $1M have a 50% higher failure rate than projects with budgets less than $350k.

Why this is so is not entirely clear. Perhaps because of confirmation bias, perhaps because most engineers who underestimate the risk of failure do so because they have not had the chance to participate in a large-scale, multi-year development project. Some possible causes include:

• Changing requirements (ie scope creep)
• Lack of user involvement
• Poor estimation
• Poor project management
• Lack of resources
• Badly defined requirements
• Inadequate testing.

Yet these same engineers all too frequently push back against any alternatives to a full-blown application rewrite, citing (among other factors) the following issues with the current application:

• “Spaghetti” code
• Bad coding practices
• Poor design
• Low quality code
• Too hard to understand
• Doesn’t have all requested features.

All these defects may indeed be present in the legacy application, still it is in production, providing critical business value daily. In spite of all the failings.

Why automated migration beats rewriting legacy applications

Drawing on Gartner’s guidance of continuous modernization, let’s examine how using AI-assisted automated migration tools can reduce the cost, time, and—most importantly—risk of modernizing legacy applications compared to rewriting them.

The migration process is but a single step in the full modernization process, not an absolute endpoint.

Advanced modernization tools such as the Visual Basic Upgrade Companion or WebMAP move legacy workloads from old platforms to new ones. See the table on page 35 for some conversion scenarios.

Automated migration doesn’t require re-coding complex business rules and processes from existing, functioning legacy applications, but rewrites do. In the case of automated migration, the code is translated directly from the source language to the destination language, using deep semantic analysis to avoid introducing functional defects.

Rewriting existing complex legacy applications, on the other hand, requires the development team to perfectly understand the actual functioning of an application that often has little or no documentation, has deeply convoluted code patterns, and may actually function by taking advantage of existing bugs in either the code or the runtime platform. Those applications rarely have full functional regression test suites, so validation often is via direct deployment to the user community; a lack of bug reports generally substitutes for a formal test run.

Automated migration gets a functionally equivalent, modernized application into actual production—replacing the original application—in as little as weeks or months, depending on the size. Since the application source code is now in a modern language, on a modern platform, and possibly in an improved architecture, the IT development team can begin immediately working on improvements and refactoring.

A rewrite, on the other hand, can take years for a large complex application, even without the almost inevitable schedule slips. During that period, the existing legacy production application not only has to continue working for its user base, but will require IT resources to continue to maintain and update it.

Continuous modernization using automated migration

Code migration doesn’t solve all problems instantly. The migration tools recreate original code faithfully to ensure functional correctness and readability. Tightly coupled code will still be tightly coupled. Code-behind patterns will not be converted to MVC. Poor error handling will still be sub-standard.

What code migration does is slash time, cost, and risk. The original app was running in production, providing value, and the migrated app can immediately be put into production and provide value.

How does that address the modernization problem?

Our suggestion is to refactor selectively once the new application is in production. After all, the code works. Refactoring working code with no other goal than to refactor is, frankly, not a good use of scarce IT resources. However, as these types of applications require on-going maintenance and improvement, it opens the door to a continuous modernization process that has real value: during any maintenance work, the development team can refactor code that has to be touched for the update. Classes that are not modified can be left alone—they are working, right? Monolithic functions and methods can be refactored into more discrete units, unit tests can be added, and proper comments written. Dead code—after verification that no valid code paths can touch it—can be deleted. And so on.

More importantly, the app can now be part of a DevOps/CI/CD engineering process that can have substantial benefits to the organization.

Software engineering needs to be pragmatic above all. IT in the enterprise is always under-resourced compared to the volume of demand from the user community. As low-code and no-code development becomes more common (and the tools improve), citizen developers will be able to off-load some of the backlog from conventional IT, allowing the core IT function to spend less time maintaining legacy apps and more time adding value to the organization through innovation and new development.

Next: in our final installment, we'll explore why true digital transformation requires both Agile development and DevOps. Coming soon.

*******************************

¹ The poster child for mega-projects was Boston’s “Big Dig” (Central Artery Tunnel Project) which cost $14.8B and was budgeted at $2.8B, a 428% cost overrun.

² In 2015 the Standish Group published their CHAOS Report, which showed that only 18% of large agile projects and 3% of large waterfall projects. The remainder (82% of agile and 97%! of waterfall) meet user requirements on time and within budget. These failure rates have been fairly consistent for years.

³ https://learn.g2.com/project-management-statistics

⁴ Good list can be found here.