Migrating Clarion to .NET 8

It's no mystery why 3GL and 4GL tools took off in the 1990s. Personal computers (PCs) had made huge inroads into the corporate desktop, supplanting minicomputer workstations from companies like DEC and Data General and the once-ubiquitous IBM 3270 terminals. All of those required either expensive off-the-shelf applications or custom software written by the organization's IT department. IBM's release of the original PC, and the clone wars that followed, were a backlash against the priesthood of IT--moving control of computing power down to the actual users.

Without software computers are just expensive decorations, and writing usable custom (bespoke¹) apps first in DOS and later in Windows was non-trivial. Apple and then Microsoft trumpeted the value of a graphical user interface, but both required programmers to use painfully difficult APIs and "shoot your own foot off" programming languages like C². 

There had to be a better way.

Clarion, PowerBuilder, and Visual Basic were all attempts to address the problem of how to dumb down programming PCs to something below the Einstein-level IQ scenario. Featuring WYSIWYG (what you see is what you get) forms editors and simplified database connections, these were all popular alternatives for building "forms over data" type applications in the enterprise. Lots of apps were built on these platforms, and plenty of them are still alive and kicking.

Long time coming

Clarion dates back to 1986 with--believe it or not--a DOS version. It featured some quaint ideas (in hindsight) that were common at the time, including building screens by writing directly to the video memory of the IBM PC and using a dongle for copy protection. Clarion had its own proprietary ISAM database format ("DAT" files) and came with all the tools you needed including a screen designer, editor, and debugger. The code was interpreted, not compiled. 

The late 80s into the mid 90s were the era of Compiler Wars. Back then I was in the Languages Business Unit at Microsoft and Borland was our chief competitor, handing us our hindquarters almost daily. Clarion wound up being bought by a startup founded by some Borland executives and kept adding features through the 90s. 

Clarion is still alive today following various changes in ownership. True to its roots, it continues to try to simplify building screens over databases and gives developers an alternative to writing pure code. 

Why change?

Programming languages come and go, although some tend to be stickier than others. Many moons ago, ADA was going to revolutionize coding for the US Department of Defense. Smalltalk was going to switch everyone from procedural languages to object oriented ones. There was even a time when C was considered slow, sloppy, and dumbed down compared to assembly language. 

Clarion, like other 4GLs, has its own programming language. It sort of kind of resembles other programming languages like C or Java, but it's not identical. Knowing Spanish might give you a head start on learning Italian, but growing up in Madrid doesn't mean you can get all chatty with the barista you order from in Milan. The similarities are helpful but not sufficient for useful skills.

LIkewise, knowing C or Rust or Java or Python--languages you can actually study at university--won't make you an expert in Clarion. We hear from customers who have legacy portfolios in classic VB, PowerBuilder, and Clarion (among other legacy platforms) that finding developers today is getting next to impossible. Perhaps in the 1990s you could source Clarion expertise as easily as, say, Python gurus today, but that crowd has aged out of the workforce largely. If they were 30 when they learned Clarion in 1990 they'd be 63 today and more interested in their golf handicap than helping you maintain that old app.

Migrating to .NET

Nothing anywhere comes close to .NET for building full stack web (and desktop) applications on top of structured databases. C#/.NET expertise is legion, and competent programmers are available in every developed economy around the world. 

Converting Clarion source code to C# isn't a trivial task. For one thing, much of Clarion source code was auto-generated by the IDE, not an uncommon scenario with 4GL languages. Auto-generated code is rarely as readable or well-written as human code (one hopes), and converting that creates an extra level of effort with associated risks of additional defects. 

Some Clarion data types have no comparable type in C#: the Picture Token type can be a number, currency, date, string, and more. For example:

Var1 STRING(@N2)               ! Two digits, group with commas
Var2 STRING(@E9.1)             ! Nine digits, period as separator, one decimal
Var3 STRING(@S20)              ! String with length of 20
Var4 STRING(@D6)               ! Date format as dd/mm/yyyy
Var5 STRING(@T3)               ! Time format as hh:mmTT
Var6 STRING(@P#-####-####P)    ! Pattern i.e. 1-9999-7777
Var7 STRING(@K<# ^^^ ##K)      ! KeyIn i.e. 04 AUG 23

Our solution is to create helper classes so these examples would get converted to C# as follows:

string Var1 = CreateNumericPictTokenFormat(2);
string Var2 = CreateScientificPictTokenFormat(9, ".", 1);  // "." could also be '.'
string Var3 = CreateStringPictTokenFormat(20);
string Var4 = CreateDatePictTokenFormat(6); // could be worked with enum
string Var5 = CreateTimePictTokenFormat(3); // could be worked with enum
string Var6 = CreatePatternPictTokenFormat("#-####-####");
string Var7 = CreateKeyInPictTokenFormat("<# ^^^ ##");

Other changes are more straightforward, for example a procedure call in Clarion maps pretty straightforward to a method declaration and call in C#:

    PROGRAM
    MAP
AddOne PROCEDURE(short), short
    END 

    CODE

    ! This is the procedure call
    AddOne(4)

AddOne PROCEDURE(short val)
new_val short
    CODE
        new_val = val + 1
    RETURN new_val

Turns into this in C#:

public static class Program
{
    public static void Main(string[] args)
    {
        // This is the procedure call
        AddOne(4)
    }
    
    public static void AddOne(int val)
    {
        int new_val;
        new_val = val + 1;
        return new_val;
    }
}

There's more, obviously. Regular expressions will only get you so far. Converting the forms (screens) is an enormous challenge, one where our proven AI-assisted technology shines. If you want to roll your own conversion strategy, rebuilding the forms in your desktop or web UI framework of choice and then coding up the event handlers will be necessary. To move Clarion from desktop to a proper web application will also require handling a lot of tricky bits like modal dialogs and session state. It will basically be a ground-up rewrite, and those are really hard to pull off on time and on budget. 

Is it worth it?

If you've read this far, you've either got a masochistic streak or some antique Clarion you'd like to get rid of. I'm assuming if were a small app (and thus a small-ish problem) you would have already disposed of it via a rewrite into a modern language. So let's propose, Dear Reader, you have a large complex Clarion app and you'd like to change your relationship status from "committed" to "open to new experiences."

Arguably there are few reasons remaining to keep an application captive to a desktop; setting aside the whole work from home vs return to office debate, web apps are easier to implement DevOps, easier to do Agile, easier to deploy, easier to monitor. One can make a pretty strong argument that mobile is the only place where native apps make any sense today, with a few exceptions.

Thus the strategy for moving off Clarion probably should include a strong bias toward ending with a web application, hosted in the public cloud (again, there are reasons to stay within a completely private network--Defense Department stuff, for example--but over time the valid reasons become like warm days in November³). 

Any large application modernization project is going to be painful because there's no silver bullet. Like my recent kitchen re-do, you'll have to put up with the software equivalent of cooking on a hotplate on top of the washing machine while blotting out the sound of power tools. That's where experience and advanced tools can really pay off. To learn more, give us a call.

------------------

¹For our friends across the pond.

²In the mid-90s, I was in the Visual C++ business unit when we released MFC as an attempt to make writing Windows apps easier. Rarely does one combine the words "C++" and "easier" in any single thought. 

³If you post a comment here about how you're living in some place right now with balmy weather I will delete it. You've been warned. Those of us in the Pacific NW are in the season of suffering.