WordPress – Some Lessons Learned

As you’ve read before in this blog, I’m working on some plugins to display genealogy data on a WordPress site. I have a plugin for Gramps (written in Python) that creates data files, and a plugin for WordPress (written in PHP) to present that data. This currently requires the webmaster to manually copy the resulting files to the server. The next step in the development process is to make the Gramps plugin communicate directly with the WordPress plugin to update the data.

This is one of the most ambitious projects I’ve embarked on as a hobby programmer. I spent the past few days getting up to speed on programming the WordPress REST API. The last hurdle to get past was interesting. The REST API requires creating a nonce, and passing it on all API calls to the server. Since my code needs to update data on the server, these API calls must be in the context of a logged-in session. The first step in the process must, however, be allowed for a not logged-in user since it logs the user into the server.

The problem is that the value of the nonce is determined based on the session cookie, and that cookie is different for a logged-in session and a not logged-in session. I struggled for a while to try to figure out how I could create a nonce for the logged-in session while performing a function in a not logged-in session.

After a few days of trying different things, searching on-line, and at times spinning my wheels, I finally found that one source on-line that explained the solution. Here’s my working signon function code with the necessary code in bold:

function my_update_cookie($logged_in_cookie) {
    $_COOKIE[LOGGED_IN_COOKIE] = $logged_in_cookie;

function tangled_web_start(WP_REST_Request $request) {
               [$this, 'my_update_cookie']);

    $creds = ['user_login' => 
              'user_password' => 
              'remember' => true];

    $user = wp_signon($creds, false);
    if (is_wp_error($user)) {
        return $user;

    return wp_create_nonce('wp_rest');

The key is to add the 'set_logged_in_cookie' action. Once the signon takes place, that action is called and the LOGGED_IN_COOKIE is updated. Once that is done, the correct nonce is then created.

On the python side, the session issues are managed by an instance of the requests.Session() class. Here’s how the call to the above API is done:

# Logon to WordPress site
session = requests.Session()
parms = {'id': logon_creds['userid'],
         'pw': logon_creds['password']}
res = session.post(url=tgturl + 
if res.status_code != 200:
    return 'Login failed'
nonce = res.text
session.headers.update({'X-WP-Nonce': nonce})

Once the nonce is returned, it gets added to the headers for all subsequent API calls.

Now that I’ve got this working, the rest of the development process should be relatively straight-forward. I’ll have to program API’s for the following tasks:

  • Get a list of checksums for the JSON data files on the servers for a given Tangled Web instance
  • Upload a JSON file or image file to the server
  • Update the names index on the server
  • Update the global settings for the instance

Once this is done, my usual Gramps workflow will have one more step. At the end of a session adding and updating people in my Gramps database, I’ll be able to invoke the “Export to Tangled Web” function in Gramps to update the server immediately, instead of having to manually copy and unpack the files on the server and reload the data.

The Tangled Web Project

Genealogy research doesn’t (or shouldn’t) happen in a vacuum. We all want to share the results of our research with others. These days, that usually is done on the internet. I’ve regularly posted the results on my own web site. Ever since using Gramps, I’ve posted my data using the Gramps Narrated Web Site, but I was never been fully satisfied with it. The information wasn’t always the easiest to navigate, some pages were very large, and it wasn’t very mobile friendly.

Web technology steadily changes. For the past four years, I’ve been using WordPress. The genealogy data was the only section of my web site still using static html pages. I knew programming a new web publishing system would be a big project, but a few months ago I dove in. And so now, I’m able to publish my data using that new system. You can see my genealogy data at Boldts & Molls.

I call this project “Tangled Web”. The most common metaphor used in genealogy is the tree. However, that’s a flawed metaphor, useful only in narrow scopes. Relationships are complex, and form a vast web of interrelationships, often quite complicated.

Clicking on Boldts & Molls, you first see an introductory page. That “home” page includes a list of people born on today’s date in history. It also shows a list of the most common names in the genealogy, in the form of a name cloud, with the bigger names indicating the more common names. There’s also a “search” button, where you can search on last name, given name, location, or a range of years.

For individuals, the new system uses a tabbed layout. The “Family” tab shows parents, grand-parents, siblings, spouses, and children. The “Timeline” shows a history of the person, starting with the marriage of the parents. All events recorded in the Gramps database are listed, and birth, marriage, and death events include witnesses and informants. The “Pedigree” shows the pedigree for the person, with instances of pedigree collapse shown. And the “Sources” tab shows the sources and citations, including scans of the primary source records.

One aspect I wanted to emphasize was relationships between people. And so wherever a person is listed on someone’s page, their relationship is shown.

How was this implemented? First, there’s a Gramps plugin, which creates a set of data files. Like all Gramps plugins, this code is written in Python. Next, there’s a WordPress plugin, written in PHP, which processes that data, and maintains the search index. Finally, there’s a good chunk of code written in JavaScript that formats the data in a browser. The latter uses a technique called Ajax, which is becoming more and more popular. The idea is that the content of the web page is written dynamically, under the control of the JavaScript program. The browser doesn’t have to refresh the whole page whenever you click on a link, but just has to request the changed data from the server. This is how pages like Google Maps work. Altogether, it’s about 5000 lines of Python, PHP, JavaScript, and CSS code.

What’s next? My first priority is to write up some documentation, and upload the code to my GitHub page. The code will be available to anyone under a GPL license. For the next version of the code, my goal is to make it easier to update a Tangled Web instance on the server. That is, I want the Gramps plugin to have the capability to communicate directly with the WordPress plugin to update the data changed since the last update.

Finally, I invite other people to contribute to this project. WordPress isn’t the only content management system out there. Although it’s the most commonly used CMS, there are others like Drupal or Joomla!. If you’d like to have a Tangled Web instance running on your self-hosted web site running on a different CMS, feel free to rewrite the PHP code to suit your CMS.

Cheers! Hans

Some Reasons Why I Use Open Source Software

I’ve been using Linux now for some 23 years. My first reaction after loading RedHat 5.1 was “What the heck am I getting myself into?”. But the open source world gradually matured, and now there are very few reasons not to use free or open-source software (FOSS).

I won’t go into all of my reasons for using FOSS. I worked for several decades as a software developer. So I know full well what goes into developing, selling, maintaining, and supporting software. For companies that sell software, the unfortunate reality is that their number one priority is profit. Once a software product is developed, in the worst case, they will fix defects and enhance the product only if there is a business case for it. And of course, to maximize profit, they will hire the lowest cost developers they can.

I think most of us know of software defects that have gone unfixed for years. I myself have been burned buying shoddy software. That’s one reason I really appreciate the good quality, open-source software that I rely on daily.

To illustrate, let’s look at some of the programs in the “Favorites” category of the application menu on my Linux system:

First, there’s Google Chrome. This is clearly one of the biggest success stories in the FOSS world. And it’s very likely that you too already use Chrome. At one time, Microsoft Internet Explorer had a massive dominant position among web browsers. But now, even Microsoft has abandoned IE, and now uses the open-source Chromium engine to power its Edge browser.

Next, there’s LibreOffice Writer. While Microsoft has ceded the battle for browser supremacy, they still vigorously defend their office suite cash cow. However, here’s a case where the FOSS alternative to Microsoft Word is clearly superior. LibreOffice Writer isn’t just as good as Word, it has more functionality than Word. Writer is considered as desktop publishing application, while Word is not.

Then there’s the GIMP, the Gnu Image Manipulation Program. This is a program I use almost daily. It provides almost the same functionality as PhotoShop, but at a price hundreds of dollars cheaper. While a professional photographer or graphic designer can write off the cost of an application like PhotoShop, an amateur like me cannot. I simply can’t justify the high price of PhotoShop given the availability of a powerful program like GIMP that more than satisfies my image processing needs.

MuseScore is an amazing application that every musician or songwriter should know about. This isn’t just a music score editor. While you’re composing your musical creation, you can hear what the piece sounds like. You can have as many instruments as you want, you can change instruments, transpose keys, and once you’re done, you can record the composition. The user interface isn’t the easiest, but I think that probably goes with the territory. Music composition isn’t easy to begin with, and there is definitely a learning curve to get past.

In the realm of genealogy, it’s (unfortunately) especially important to consider the reputation and integrity of any vendor. That’s why open-source is very important in this realm. Gramps is a full-function genealogy application that supports pretty much the whole GEDCOM data model. In addition, it includes a framework for supporting user-written extensions. You can find some of my own Gramps extensions at my GitHub page.

The latter application illustrates an important point about FOSS. It’s not just one company supporting a piece of software. In the open-source realm, it’s a whole community. Frankly, as a programmer, I have no inclination whatsoever to improve the quality of a program if it just adds to the profits of some company. On the other hand, the open-source community is full of people willing to contribute to the benefit of other users. Everyone is free to use the software without obligation, but also, everyone is invited to contribute in any way that they are able to.

I could go on describing other FOSS applications I use either regularly or occasionally. Such as Audacity for audio editing, Kdenlive for video editing, or Komodo edit for developing software. But I think this list should provide enough justification for considering open-source software instead of buying commercial software.

JavaScript – Some Lessons Learned

If you’re not a programmer, move along, there’s nothing to see here. Only programmers will understand what I’m writing about here.

As you might already be aware, I post the results of my genealogy research on this website. So far, I’ve been using the “Narrated Web Site” feature of Gramps. But since I’ve never been totally satisfied with it, I’ve always toyed with the idea of coming up with a better way to present my data. So finally, about a month ago, I started work on a new project, a different way to present my genealogy data.

This is turning out to be a rather ambitious project, with elements coded in Python, PHP, and JavaScript, as well as CSS and HTML. Sometimes it’s hard keeping all the different languages straight. Since I’m using dynamically-generated HTML, most of the challenges are in the JavaScript code, an area that I’m certainly not expert in. In this missive, I discuss some of the lessons I’ve learned while coding, some after much flailing about. I won’t dive too deeply into specific details. You can learn more with the right Google search.

When to register click handlers

First, since I make heavy use of dynamic HTML, I make good use of click handlers. But while coding, I noticed that in some situations, the click handlers weren’t working as expected. It took a bit of digging, and I found a solution I wasn’t totally happy with. Digging further, I then realized something important about registering click handlers when using dynamic HTML: You need to register the click handler after adding the HTML to the DOM. In the cases where they weren’t firing as expected, the handlers were registered first.

Handling history

When using static HTML page, you don’t have to worry about history. The browser does it all automatically. However, with dynamic HTML, you need to manually keep track of the history if you want meaningful navigation through your content. This is achieved by coding an “onpopstate” event handler, as well as calls to functions “pushState()” and “replaceState()”.

After some flailing about, I stepped back and studied the issue in a bit more detail. And I realized it really wasn’t so bad. I just needed to keep track of three specific situations when displaying page content.

First, when moving to a new page, you need to call “pushState” while outputting the new content. That registers the new page in the history stack.

Second, when navigating to a page previously displayed using an “onpopstate” handler (invoked when the user presses the “back” or “forward” button), you just have to output the page content. No further action is needed.

Finally, there’s “replaceState()”. That’s used in situations where you want to update the content currently on the screen, but you don’t want to advance further in the history. For example, you may just want to switch to a different tab in your content. Or perhaps you want to present a table of search results ordered by name instead of date. These are cases where, when pressing the “back” button, you don’t want to go back through these different views. In this type of situations, you call “replaceState()”.

Querying attributes of HTML elements

The final lesson regards querying certain attributes of HTML elements. I’m now getting into an interesting piece of JavaScript programming where I need to know the real sizes of some HTML elements. There are some functions in jQuery to get the size of an element: “outerHeight()” and “outerWidth()”. I tried calling them, and was surprised to get zeroes as the result.

Again, a bit of digging on-line, and I found the answer. Immediately after adding dynamic elements to the DOM, the attributes might not be set yet. Processing the elements must wait until they are ready. So you’ll need to code something like:

jQuery('#mysection').ready( function() {

I tried this, and I did get non-zero values for the height and width. However, the values returned did not take into account the size of an embedded image. Taking this approach further requires waiting until all embedded images are loaded. Grrr!

I ended up deciding that this avenue was just too strewn with potential problems. I stepped back and decided to take a different approach to what I wanted to do, an approach with a lot fewer complications.

In addition, further research turn up another issue: This usage of the “ready()” function is deprecated by jQuery.


There’s a lot to JavaScript coding. It’s an incredibly complicated and potentially intimidating environment. I’m constantly looking things up on-line: in manuals, on-line courses, or other blogs. However, on-line information, while usually correct, is sometimes missing some important detail that a novice JavaScript programmer might not be aware of that’s needed to make full sense of the information.

Cheers! Hans

Take Back Control Of Your News

Face it, Facebook sucks at news. We all know that. This was all too painfully clear over the past year, with fake news stories being spread like wildfire throughout the social media. Facebook understands the issue and is taking steps to mitigate the problems. Whether or not these steps are enough is, however, questionable.

Personally, I’ve never been much of a Facebook fan. I use it since it’s pretty much a fact of life these days. Like it or not, it’s now the way to connect with people. My main issue is that Facebook decides what we should see in our feeds with precious few options to customize our preferences. (For the past few weeks, it seems that Facebook has decided not to put any items from my liked groups and pages into my news feed.)

But still, a lot of people get their news from Facebook. One study found that a majority of Americans get their news from Facebook. This is a frightening result considering the control Facebook has over what we read. What can we do?

A few days ago, I came across an article that reminded me of a protocol that has dropped in popularity over the past few years, but still has relevance: Why RSS Still Beats Facebook and Twitter For Tracking News.

RSS stands for “Rich Site Summary” (or “Really Simple Syndication”). RSS provides a standard way for news sites and blogs to present information. Using an RSS aggregator, you can view news and opinions from the sources you trust, not just the stuff Facebook thinks you want to read.

To get started, check out one popular RSS aggregator, Feedly. It’s what I now use to read news on my tablet. You can easily search for news feeds and select the ones you’re interested in. Alternatively, if you know the URL for a blog’s RSS feed, you can paste it into the search field and add it manually to your list. For example, to see the RSS feed for this blog, click here. If you want to read my ramblings as soon as they’re written, just add the URL of that page to your aggregator.

It’s time to take back control of your news feed. Don’t count on Facebook for news. Go take advantage of RSS aggregators, and view the news you want.

Cheers! Hans

Adventures in WordPress

I’m an old school programmer. I remember a time when internet access was slow and we could see images slowly appear on web pages. At the time, understanding the technical aspects of HTTP and HTML were important to properly balance out design considerations and performance.

But times change. And I’m getting too old to worry too much about the nitty-gritty details. In my professional life, I dealt with one content management system, Zope. We were trying to develop a system based on Zope and Plone, but for various reasons, the effort wasn’t successful. I’m a big fan of Python. But sometimes it seems that, because the language is so easy to use, systems built using Python can get bloated very easily.

About five years ago, I learned PHP. It’s an ugly language with a less than stellar reputation. But it is ubiquitous and widely supported. Likewise, there are aspects of WordPress that grate with my old-school programmer creds. But it is widely used and supported. Sometimes you just have to be pragmatic.

For about 6 years, I was the webmaster for our church, the Kingston Unitarian Fellowship. Up until recently, the site was hard-coded HTML using SSI and Javascript. And it took some effort to make the site mobile-friendly. But back in January, the board directed me to use WordPress for the church website. It was not entirely surprising since we talked a bit about it before. And it made sense since other people in the congregation would then be able to update content. That was an important consideration for me since I knew it was only a matter of time before we would end up resigning our membership.

Starting from zero knowledge, I had most of the site converted within a matter of hours. Over the next week, I gained the knowledge to move over the dynamic content, and fix a few other glitches. And after a few months, I learned the “right” ways to do certain things, such as where to load the custom CSS and Javascript files.

With that new knowledge and experience, I decided my own web site needed a revamping. It most definitely was not mobile-friendly, and I hadn’t done much with the site for years other than occasionally update the genealogy section. But there was some content that sometimes prompted visitors to drop a few bucks in my “tip jar”. So I set out to bring my web site solidly into the 21st Century.

My site had literally hundreds of static pages. How to approach such a task? First, photo albums took up a large number of pages. I decided to implement the photo albums using a custom WordPress plugin with Ajax loading of the photos. Conversion of the content was made easier with a custom script.

The next biggest group was a set of about 200 pages, each one for a specific area in the city of Toronto. Again, I wrote a custom plugin with Ajax loading of the individual pages, and converted the pages using a custom script.

The rest had to be handled manually. That put a lot of pages in the main menu. So many in fact, that I reached a hard limit, making it difficult to update the menu. I ended up adding smaller sub-menus included at the top of some pages. Over time, I’ll do more of that to make the main menu more manageable.

Finally, I copied over blog postings from my blogs hosted on Blogger.

There are always trade-offs with any project, such as a web site. I like the freedom you get with a hard-coded site. But that takes much more of an effort. I’m not a big fan of the choices of theme you get with a content management system, but I can live with the options provided by the default scheme that comes with the current version of WordPress.

Cheers! Hans

Three Disruptive Technologies

This is the column I wrote intended for the May 2017 edition of our church newsletter. Given recent events, that issue will be my last. With that in mind, I decided not to include this column. Instead, I offer it up here, on my own personal blog:

For almost two years, I’ve had the privilege of acting as editor for KUFLinks. In that role, I’ve taken the liberty of writing a monthly column under the banner “Communications”, most of which have been on one particular topic, technological change. In this missive, I look at three pivotal changes, that have been quite disruptive on society.

First, when editing KUFLinks, I use a piece of software called “LibreOffice Writer”, which is well-suited to desktop publishing. This program is a member of a large class of software known as “free software”. Although much of this software is available at no cost, the word “free” primarily refers to freedom. That is, you have the freedom, granted by the software license, to do what you want with it. Either without restriction, or with one specific limitation that in practice doesn’t affect the users of the software. (There is fierce debate between these two camps, but the details are not relevant to this discussion.)

Most of you aren’t aware of this, but free software underlays much of what we do today. The most popular web browsers, Chrome and Firefox, are based on free software. Most of the software running the internet is too, from the operating systems, to the web servers, databases, and content management systems. If you use a smart phone or tablet, you’re using products based on free software. Even the WordPress software running the KUF web site is free software.

Second, let’s go back a few centuries to the invention of the printing press in Europe. This of course led to immense change in European society. Of interest to Unitarians is the story of one man, Michael Servetus, who used the printing press to publicize his views. In doing so, he got a lot of people mad at him, especially the Catholic church. Servetus expected a safe haven from the Calvinists in Geneva, but unfortunately, they too were not happy with him. Later, Servetus came to be considered the first Unitarian.

Third, we come back to the present. Many of us still remember a time without the internet. Looking back, it now seems strange that we had to look up information in books, often having to wait until the chance to visit the local library or book store. In many cases, we had to travel some distance to find the right repository of information. When doing research, patience was definitely a virtue.

But of course today, everything is on-line, often just a simple search away from the convenience of our home, either on a desktop computer, or on a mobile device. And if we want to connect with other people with the same interests or values, that too is a simple matter of pressing few buttons.

I can say a whole lot more on each of these three disruptive technologies, but I’ll leave it at this. For now.

Cheers! Hans

Is It Now Time To Leave Facebook?

What do we do about Facebook? Many of us use Facebook every day. For many of us, it’s a great way to connect with friends and acquaintances, and to see what’s going on in our communities. I organize a monthly ukulele jam, and Facebook is one of the ways I use to publicize the jams, both on the Kingston Ukulele Society page, as well as other pages.

Unfortunately, Facebook continues to tinker with the filtering algorithms used in deciding what we should see. And this tinkering means that there’s less likelihood that we’ll see what we really want to see.

Facebook does offer a way for us to tailor what we see in our newsfeed. For me, I have my settings configured to see “All Updates” from the vast majority of my Facebook friends, but only the “Status Updates”, “Photos”, and “Music and Videos” from them. The photo at right shows how to change the settings. However, some people have reported that this option is no longer available to them. Since it normally takes a while for updates to roll out to users, it’s inevitable that the rest of us will lose this capability too.

In a Youtube video, Derek Muller explains Facebook’s algorithm that decides what we should see. But here’s my concern: Facebook can never truly understand what’s really important to me. For example, I have some Facebook friends that I have little day to day interaction with, but still I’m interested in everything they post. No algorithm can figure that out.

Of course, Facebook can do whatever it wants. In fact, as a public company, they have a duty to ensure that their stock-holders get the best possible return on their investment. Even if it means reducing the level of usefulness to its users. We all need to realize this fact of business.

But Facebook is also treading a fine line. While doing what they can to maximize share value, they also can’t risk alienating its users. If Facebook becomes less useful to us, what’s the point? Already, there are reports of teenagers leaving Faceook in droves, moving to mobile messaging apps. If Facebook can’t guarantee that I’ll see exactly the things I’ve asked for in my settings, what postings will I miss out on? And also, what assurance can I have that people will see my notifications of upcoming ukulele jams? Big companies can afford to pay Facebook the big bucks needed to ensure that everyone sees their posts. I can’t.

Can I afford to leave Facebook in favor of an alternative social networking site? I’m on Google+, as are many of my friends and acquaintances. But most of them aren’t active on that site. Today, I signed up to Pinterest, but it’s not clear if that’s an acceptable alternative. And I’ve never quite seen the point of Twitter. Today, Facebook still offers me the ability to tailor my newsfeed, but what happens when they take that feature away from all of us?

We’re all left with a dilemma. We all visit Facebook to stay connected and see what’s happening in our communities (geographic or interest). But unless there’s a mass migration, we can’t simply jump to an alternative social networking site. So we’re all stuck with Facebook. As for me, I’ll do my bit to post more on Google+, and less on Facebook. If enough of us do that, perhaps we can tip the balance in favor of the alternative. Or convince Facebook to put more emphasis on the needs and wants of its users.

Cheers! Hans

A Short History of Free-Form RPG

Free-form RPG is back in the news with the announcement of free-form D, F, P, and H-specs. In this short essay, I look back at the history of free-form syntax in RPG.

It all started more than twenty years ago, back when RPG IV was being designed. At the time, there were two distinct schools of thought. On one side, there were people who insisted RPG should have a fully free-form syntax. On the other, were those who strongly believed that RPG should remain in its traditional fixed-form layout. The result was a compromise that leaned heavily towards a traditional fixed-form syntax, but with some free-form elements, such as keywords on the new definition specification and expressions in an extended factor two entry.

My first task in the implementation of the RPG IV compiler was coding the new D-spec, along with its keywords. Later, we were able to convince our planner that keywords should also be allowed on the F-spec. Later still, keywords on the H-spec became an obvious design change.

The extended factor two opcodes, such as EVAL, IF, DOW, etc., were a riskier proposition. At first one developer was assigned the task, and later another to help out the first. But as time went on, it was clear that they weren’t making any progress. Late in the development cycle, I was assigned that piece of the language. Scrapping the work already done and starting from scratch, I finished the feature well in time for the initial release of RPG IV.

With the release of RPG IV, the development team was scaled back, leaving two developers working on new features. Because the extended factor two calcs were such a hit, one proposed enhancement was a fully free-form syntax for C-specs. However, we believed that it would require 100% of all development and testing resources for one release. When considering potential enhancements, this item would always get pushed well down the priority list.

This changed during the planning cycle for V5R1. I realized that common coding standards had changed somewhat during the past few years. I realized that a new free-form C-spec did not need to include all traditional C-spec features. Conditioning indicators weren’t needed now that we had the IF opcode. Resulting indicators were largely deprecated in favor of built-in functions. And some opcodes were no longer needed, such as IFxx and the MOVE opcodes. We just needed a few additional built-in functions to fill in some gaps. With this strategy, free-form calcs could be implemented relatively easily, along with other useful functional enhancements.

But of course, this proposal was not without controversy. In order to get the proper design, we had approval from management to discuss the proposal publicly, on a popular RPG related mailing list. Many people loved the idea, but some were vehemently against the proposal. Personally, although I knew we had something good, I was so disheartened with the criticism that I gave the feature a 50/50 chance of making it to release. However, in 2001 it was released, and over time, most critics came around to accept the syntax.

With V5R1 and /FREE released, we started thinking about moving towards a free-form syntax for other spec types. However, while free-form syntax made a lot of sense for calc specs, we could see little benefit for the rest of the language. Other enhancements of a more functional nature were always considered more important.

In the summer of 2003, the iSeries group in the Toronto Lab could not escape the “staffing actions” rampant throughout the Software Group, and the RPG compiler development team was reduced to one person. I was moved to a new team responsible for PL/X, a compiler used internally within IBM for the development of mainframe software.

So now we come to the Fall of 2013, 12 years after the release of  free-form calcs, 18 years after the release of RPG IV. A few weeks ago came the announcement of free-form D, F, P, and H-specs in RPG. Although I’m happy to see this come about, I’m also puzzled. It seems all too anti-climactic, too little too late to save an anachronism of a programming language. In a previous missive entitled Is RPG Dead? The Autopsy, I list half a dozen features common to modern programming languages but missing from RPG. Note that free-form syntax is not included in that list. At the time I wrote that, significant parts of the language were still fixed-form, but I didn’t consider the lack of a fully free syntax significant.

So what’s the big deal about the additional free-form syntax? More than ten years ago when I was part of the RPG development team, we always looked for useful functional features to add to the language. That is, features that would help improve programmer productivity, or allow programmers to do things they couldn’t do easily before. But that seemed to change about ten years ago. In the last release that I had any involvement with, one feature added was XML operations within the language. It wasn’t as if programmers couldn’t handle XML before since there were already XML API’s in use. But as far as I could tell, the only reason the XML opcodes were added was because COBOL was getting XML operations, and someone within the management team decided that RPG needed them too. I considered it a goofy feature, but I was too tired and jaded to argue. (Now that JSON is commonly used in places where XML was previously used, should RPG now include JSON opcodes?)

Lately, the major enhancements to RPG seem to be accompanied by major coverage in the iSeries press. Consider that silly Open Access feature, which seems primarily designed for ISV’s helping to modernize old monolithic applications, which in most cases probably should be rewritten from scratch instead. And now with the new free-form specs, a lot of pundits are writing about how the feature will make the language more acceptable to other programmers, while ignoring the functional deficits previously mentioned. That is, these days, it seems like planning RPG content is more of a public relations exercise geared towards managers of client RPG shops rather than providing real improvements making the job of RPG programmers easier. As RPG falls further and further behind the modern programming languages, I suspect the PR show will become more and more prominent.

In my opinion, although I think the new free-form specs are a nice improvement to RPG, they will not make RPG more palatable to the programmers of Java, Python, PHP, etc. A dozen years ago, we saw little justification for the feature, and I don’t see what has changed in the meantime. If anything, RPG has become less and less relevant as applications programmers continue to discover the productivity gains to be had by using modern interpreted languages like Python and PHP.

Operation Red-Nose Project

About 13 months ago, I started working on a project to replace the software used by the local Operation Red-Nose (ORN) organization. Up until then, they had been using an old DOS-based system that was long past its best-before date. In fact, they were only using part of the system, and even that had major usability issues. The project was suggested by my supervisor at the MCF Practice Firm, a volunteer at ORN. The purpose of the project was for me to pick up some specific programming skills. At the time, I couldn’t complete the project since I got a job offer five weeks into the program. But that job didn’t last, and I was able to return to the ORN project. (During the summer, federal funding for the practice firms was withdrawn, and so MCF was forced to close.)

For the ORN project, I decided on CakePHP and MySQL. I chose PHP and MySQL since they are pretty much ubiquitous in the I.T. realm. For the framework, I quickly narrowed down the choices to CodeIgniter and CakePHP. I chose the latter since it strongly encouraged the use of an MVC design. The ORM also impressed me, with its ability to get all the data associated with a transaction in one operation (typically). That is, all the SQL joins were taken care of automatically.

In a nutshell, the ORN application manages many aspects of the operation, in particular, the event nights during the annual campaign. The system keeps track of volunteers. During an event night, volunteers are checked in to the system and assigned to teams. Operators take calls from clients and enter their ride requests into the system. The dispatcher assigns the requests to the available teams, and tracks the progress of the requests. And at the end of the evening, the treasurer tracks the donations.

An essential part of the development process was demonstrating the system to the ORN volunteers and getting feedback. After each weekly meeting, I’d normally have several pages of suggested improvements. The main issue was understanding how the volunteers would learn the capabilities of the system, which wasn’t always easy for a programmer geek like me. As the programmer, I knew how to get things done. But for others, especially those with little computer experience, it wasn’t always clear.

Add to that some language issues. ORN Kingston is managed through La Route du Savoir, an organization that provides training and services to French speaking people in the Kingston area. Although everyone at ORN speaks English, one person had a bit of trouble with some of the more technical language I used. Often, someone had to translate my technical language into a form that that person could understand, and her requirements into language I could understand. But we managed, and the result was a system that everyone was happy with, even if it took a few weeks after the start of the 2012 campaign to get all the required features in place and the defects fixed.

Is there more work to do on the new Kingston ORN system? Sure. The data entry form for the telephone operators can be improved. I took a few phone calls myself, and readily saw one problem with the layout of the entry fields. For the 2013 campaign, I’ll rearrange the entry fields to reflect the order that questions are asked of the callers. That is, if the first question is “Where are you?”, the pickup location should be the first item on the form. In general, that form should have a list of questions for the operators to ask to ensure that nothing is missed.

How did CakePHP work out? In retrospect, I think it was a perfectly reasonable choice. At no time was there anything that CakePHP couldn’t handle. There was one annoying issue with session timeout. However, upgrading to version 2.2 addressed that problem. The upgrade didn’t go without some effort, though. When I started the project, I didn’t always follow the preferred naming convention. For version 2.2, I had to rename my controller classes.

What about MySQL? The application currently uses 16 tables, which is puny compared to other production systems. Although I prefer more robust DBMS’s, I could clearly see how MySQL has matured over the past few decades. The InnoDB engine (now the default) is sufficiently robust for this type of application, supporting foreign keys and transactions, vital features in even the smallest of databases. In contrast, today in the year 2013, there are still production MySQL databases that use the old MyISAM engine. In my opinion, any database administrator who entrusts vital company data to a MyISAM database should be fired immediately.

If I were to start the project today from scratch, would I do things differently? Styles of programming for web applications are always changing. For the ORN project, I designed it as a fairly traditional server-based web application, with just a bit of client-side programming. But the trend is to move more processing to the client, with the view and controller components on the client and the model on the server. But for this type of application, I don’t think I’d do things any differently.